.\" Automatically generated by Pod::Man 2.22 (Pod::Simple 3.14) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is turned on, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .ie \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . nr % 0 . rr F .\} .el \{\ . de IX .. .\} .\" .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). .\" Fear. Run. Save yourself. No user-serviceable parts. . \" fudge factors for nroff and troff .if n \{\ . ds #H 0 . ds #V .8m . ds #F .3m . ds #[ \f1 . ds #] \fP .\} .if t \{\ . ds #H ((1u-(\\\\n(.fu%2u))*.13m) . ds #V .6m . ds #F 0 . ds #[ \& . ds #] \& .\} . \" simple accents for nroff and troff .if n \{\ . ds ' \& . ds ` \& . ds ^ \& . ds , \& . ds ~ ~ . ds / .\} .if t \{\ . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' .\} . \" troff and (daisy-wheel) nroff accents .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' .ds 8 \h'\*(#H'\(*b\h'-\*(#H' .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] .ds ae a\h'-(\w'a'u*4/10)'e .ds Ae A\h'-(\w'A'u*4/10)'E . \" corrections for vroff .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' . \" for low resolution devices (crt and lpr) .if \n(.H>23 .if \n(.V>19 \ \{\ . ds : e . ds 8 ss . ds o a . ds d- d\h'-1'\(ga . ds D- D\h'-1'\(hy . ds th \o'bp' . ds Th \o'LP' . ds ae ae . ds Ae AE .\} .rm #[ #] #H #V #F C .\" ======================================================================== .\" .IX Title "SquidAccessDlfConverter 3pm" .TH SquidAccessDlfConverter 3pm "2008-11-19" "Lire 2.1.1" "LogReport's Lire Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" Lire::Proxy::SquidAccessDlfConverter \- convert squid logs to dlf format .SH "DESCRIPTION" .IX Header "DESCRIPTION" squid_access2dlf expects an access.log log file, as produced by the \s-1SQUID\s0 Web Proxy Cache (http://www.squid\-cache.org/) on its stdin. .PP These log files are whitespace separated, columns are .PP .Vb 2 \& time elapsed remotehost code/status bytes method URL rfc931 \& peerstatus/peerhost type .Ve .PP A typical logline looks like e.g.: .PP .Vb 2 \& 979992041.366 502 192.168.1.160 TCP_IMS_HIT/304 216 GET \& http://example.com/home2.html \- NONE/\- text/html .Ve .PP Meaning of fields is: .IP "time" 4 .IX Item "time" A Unix timestamp as \s-1UTC\s0 seconds with a millisecond resolution. .IP "elapsed" 4 .IX Item "elapsed" The elapsed time considers how many milliseconds the transaction busied the cache. .IP "remotehost" 4 .IX Item "remotehost" The \s-1IP\s0 address of the requesting instance, the client \s-1IP\s0 address. .IP "code/status" 4 .IX Item "code/status" This column is made up of two entries separated by a slash. This column encodes the transaction result. .Sp The cache result of the request contains information on the kind of request, how it was satisfied, or in what way it failed. The status part contains the \&\s-1HTTP\s0 result codes with some Squid specific extensions. Squid uses a subset of the \s-1RFC\s0 defined error codes for \s-1HTTP\s0. .RS 4 .IP "\s-1TCP_HIT\s0" 4 .IX Item "TCP_HIT" A valid copy of the requested object was in the cache. .IP "\s-1TCP_MISS\s0" 4 .IX Item "TCP_MISS" The requested object was not in the cache. .IP "\s-1TCP_REFRESH_HIT\s0" 4 .IX Item "TCP_REFRESH_HIT" The requested object was cached but \s-1STALE\s0. The \s-1IMS\s0 query for the object resulted in \*(L"304 not modified\*(R". .IP "\s-1TCP_REF_FAIL_HIT\s0" 4 .IX Item "TCP_REF_FAIL_HIT" The requested object was cached but \s-1STALE\s0. The \s-1IMS\s0 query failed and the stale object was delivered. .IP "\s-1TCP_REFRESH_MISS\s0" 4 .IX Item "TCP_REFRESH_MISS" The requested object was cached but \s-1STALE\s0. The \s-1IMS\s0 query returned the new content. .IP "\s-1TCP_CLIENT_REFRESH_MISS\s0" 4 .IX Item "TCP_CLIENT_REFRESH_MISS" The client issued a \*(L"no-cache\*(R" pragma, or some analogous cache control command along with the request. Thus, the cache has to refetch the object. .IP "\s-1TCP_IMS_HIT\s0" 4 .IX Item "TCP_IMS_HIT" The client issued an \s-1IMS\s0 request for an object which was in the cache and fresh. .IP "\s-1TCP_SWAPFAIL_MISS\s0" 4 .IX Item "TCP_SWAPFAIL_MISS" The object was believed to be in the cache, but could not be accessed. .IP "\s-1TCP_NEGATIVE_HIT\s0" 4 .IX Item "TCP_NEGATIVE_HIT" Request for a negatively cached object, e.g. \*(L"404 not found\*(R", for which the cache believes to know that it is inaccessible. .IP "\s-1TCP_MEM_HIT\s0" 4 .IX Item "TCP_MEM_HIT" A valid copy of the requested object was in the cache and it was in memory, thus avoiding disk accesses. .IP "\s-1TCP_DENIED\s0" 4 .IX Item "TCP_DENIED" Access was denied for this request. .IP "\s-1TCP_OFFLINE_HIT\s0" 4 .IX Item "TCP_OFFLINE_HIT" The requested object was retrieved from the cache during offline mode. The offline mode never validates any object, see offline_mode in squid.conf file. .IP "\s-1UDP_HIT\s0" 4 .IX Item "UDP_HIT" A valid copy of the requested object was in the cache. .IP "\s-1UDP_MISS\s0" 4 .IX Item "UDP_MISS" The requested object is not in this cache. .IP "\s-1UDP_DENIED\s0" 4 .IX Item "UDP_DENIED" Access was denied for this request. .IP "\s-1UDP_INVALID\s0" 4 .IX Item "UDP_INVALID" An invalid request was received. .IP "\s-1UDP_MISS_NOFETCH\s0" 4 .IX Item "UDP_MISS_NOFETCH" During "\f(CW\*(C`\-Y\*(C'\fR" startup, or during frequent failures, a cache in hit only mode will return either \s-1UDP_HIT\s0 or this code. Neighbours will thus only fetch hits. .IP "\s-1NONE\s0" 4 .IX Item "NONE" Seen with errors and cachemgr requests. .RE .RS 4 .RE .IP "bytes" 4 .IX Item "bytes" The size is the amount of data delivered to the client. Mind that this does not constitute the net object size, as headers are also counted. .IP "method" 4 .IX Item "method" The request method to obtain an object. Methods are: .Sp .Vb 10 \& method defined cachabil. meaning \& \-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \& GET HTTP/0.9 possibly object retrieval and simple searches. \& HEAD HTTP/1.0 possibly metadata retrieval. \& POST HTTP/1.0 CC or Exp. submit data (to a program). \& PUT HTTP/1.1 never upload data (e.g. to a file). \& DELETE HTTP/1.1 never remove resource (e.g. file). \& TRACE HTTP/1.1 never appl. layer trace of request route. \& OPTIONS HTTP/1.1 never request available comm. options. \& CONNECT HTTP/1.1r3 never tunnel SSL connection. \& ICP_QUERY Squid never used for ICP based exchanges. \& PURGE Squid never remove object from cache. \& PROPFIND rfc2518 ? retrieve properties of an object. \& PROPATCH rfc2518 ? change properties of an object. \& MKCOL rfc2518 never create a new collection. \& MOVE rfc2518 never create a duplicate of src in dst. \& COPY rfc2518 never atomically move src to dst. \& LOCK rfc2518 never lock an object against modifications. \& UNLOCK rfc2518 never unlock an object. .Ve .IP "\s-1URL\s0" 4 .IX Item "URL" This column contains the \s-1URL\s0 requested. Please note that the log file may contain whitespaces for the \s-1URI\s0. In the Lire \s-1DLF\s0 the \s-1URL\s0 is split up in proto, host and path. .IP "rfc931" 4 .IX Item "rfc931" The eighth column may contain the ident lookups for the requesting client. .IP "peerstatus/peerhost" 4 .IX Item "peerstatus/peerhost" results from neighbouring caches. .IP "type" 4 .IX Item "type" The content type of the object as seen in the \s-1HTTP\s0 reply header. .SH "EXAMPLES" .IX Header "EXAMPLES" To process a log as produced by Squid: .PP .Vb 1 \& $ squid_access2dlf < squid.log .Ve .PP squid_access2dlf will be rarely used on its own, but is more likely called by lr_log2report: .PP .Vb 1 \& $ cat /var/log/squid.log | lr_run lr_log2report squid_access .Ve .SH "VERSION" .IX Header "VERSION" \&\f(CW$Id:\fR SquidAccessDlfConverter.pm,v 1.8 2008/06/12 15:04:32 wraay Exp $ .SH "AUTHORS" .IX Header "AUTHORS" Joost Bekkers Joost van Baal Wessel Dankers .SH "COPYRIGHT" .IX Header "COPYRIGHT" Copyright (C) 2001 Joost Bekkers Copyright (C) 2001\-2003 Stichting LogReport Foundation LogReport@LogReport.org .PP This program is free software; you can redistribute it and/or modify it under the terms of the \s-1GNU\s0 General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. .PP This program is distributed in the hope that it will be useful, but \s-1WITHOUT\s0 \s-1ANY\s0 \s-1WARRANTY\s0; without even the implied warranty of \&\s-1MERCHANTABILITY\s0 or \s-1FITNESS\s0 \s-1FOR\s0 A \s-1PARTICULAR\s0 \s-1PURPOSE\s0. See the \&\s-1GNU\s0 General Public License for more details. .PP You should have received a copy of the \s-1GNU\s0 General Public License along with this program (see \s-1COPYING\s0); if not, check with http://www.gnu.org/copyleft/gpl.html.