.\" Automatically generated by Pod::Man v1.37, Pod::Parser v1.32 .\" .\" Standard preamble: .\" ======================================================================== .de Sh \" Subsection heading .br .if t .Sp .ne 5 .PP \fB\\$1\fR .PP .. .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. | will give a .\" real vertical bar. \*(C+ will give a nicer C++. Capital omega is used to .\" do unbreakable dashes and therefore won't be available. \*(C` and \*(C' .\" expand to `' in nroff, nothing in troff, for use with C<>. .tr \(*W-|\(bv\*(Tr .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' 'br\} .\" .\" If the F register is turned on, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . nr % 0 . rr F .\} .\" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .hy 0 .if n .na .\" .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). .\" Fear. Run. Save yourself. No user-serviceable parts. . \" fudge factors for nroff and troff .if n \{\ . ds #H 0 . ds #V .8m . ds #F .3m . ds #[ \f1 . ds #] \fP .\} .if t \{\ . ds #H ((1u-(\\\\n(.fu%2u))*.13m) . ds #V .6m . ds #F 0 . ds #[ \& . ds #] \& .\} . \" simple accents for nroff and troff .if n \{\ . ds ' \& . ds ` \& . ds ^ \& . ds , \& . ds ~ ~ . ds / .\} .if t \{\ . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' .\} . \" troff and (daisy-wheel) nroff accents .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' .ds 8 \h'\*(#H'\(*b\h'-\*(#H' .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] .ds ae a\h'-(\w'a'u*4/10)'e .ds Ae A\h'-(\w'A'u*4/10)'E . \" corrections for vroff .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' . \" for low resolution devices (crt and lpr) .if \n(.H>23 .if \n(.V>19 \ \{\ . ds : e . ds 8 ss . ds o a . ds d- d\h'-1'\(ga . ds D- D\h'-1'\(hy . ds th \o'bp' . ds Th \o'LP' . ds ae ae . ds Ae AE .\} .rm #[ #] #H #V #F C .\" ======================================================================== .\" .IX Title "MIRMON 1" .TH MIRMON 1 "2016-07-23" "perl v5.8.8" "User Contributed Perl Documentation" .SH "NAME" mirmon \- monitor the state of mirrors .SH "SYNOPSIS" .IX Header "SYNOPSIS" .Vb 1 \& mirmon [-v] [-q] [-t timeout] [-c conf] [-get all|update|url url] .Ve .SH "OPTIONS" .IX Header "OPTIONS" .IP "\fB\-v\fR" 4 .IX Item "-v" Be verbose ; \fBmirmon\fR normally only reports errors and changes in the mirror list. .IP "\fB\-q\fR" 4 .IX Item "-q" Be quiet. .IP "\fB\-t\fR \fItimeout\fR" 4 .IX Item "-t timeout" Set the timeout ; the default is \fI300\fR. .IP "\fB\-get\fR all | update | url " 4 .IX Item "-get all | update | url " With \fBall\fR, probe all sites. With \fBupdate\fR, probe a selection of the sites ; see option \f(CW\*(C`max_poll\*(C'\fR below. With \fBurl\fR, probe only the given \fIurl\fR, which must appear in the mirror\-list. .IP "\fB\-c\fR \fIname\fR" 4 .IX Item "-c name" Use config file \fIname\fR. The default list is .Sp .Vb 1 \& ./mirmon.conf $HOME/.mirmon.conf /etc/mirmon.conf .Ve .SH "USAGE" .IX Header "USAGE" The program is intended to be run by cron every hour. .PP .Vb 1 \& 42 * * * * perl /path/to/mirmon -get update .Ve .PP It quietly probes a subset of the sites in a given list, writes the results in the 'state' file and generates a web page with the results. The subset contains the sites that are new, bad and/or not probed for a specified time. .PP When no 'get' option is specified, the program just generates a new web page from the last known state. .PP The program checks the mirrors by running a (user specified) program on a pipe. A (user specified) number of probes is run in parallel using nonblocking \s-1IO\s0. When something can be read from the pipe, it switches the pipe to blocking \s-1IO\s0 and reads one line from the pipe. Then it flushes and closes the pipe. No attempt is made to kill the probe. .PP The probe should return something that looks like .PP .Vb 1 \& 1043625600 ... .Ve .PP that is, a line of text starting with a timestamp. The exit status of the probe is ignored. .SH "CONFIG FILE" .IX Header "CONFIG FILE" .Sh "location" .IX Subsection "location" A config file can be specified with the \-c option. If \-c is not used, the program looks for a config file in .IP "* \fB./mirmon.conf\fR" 4 .IX Item "./mirmon.conf" .PD 0 .IP "* \fB$HOME/.mirmon.conf\fR" 4 .IX Item "$HOME/.mirmon.conf" .IP "* \fB/etc/mirmon.conf\fR" 4 .IX Item "/etc/mirmon.conf" .PD .Sh "syntax" .IX Subsection "syntax" A config file looks like this : .PP .Vb 30 \& +-------------------------------------------------- \& |# lines that start with '#' are comment \& |# blank lines are ignored too \& |# tabs are replaced by a space \& | \& |# the config entries are 'key' and 'value' pairs \& |# a 'key' begins in column 1 \& |# the 'value' is the rest of the line \& |somekey A_val B_val ... \& |otherkey X_val Y_val ... \& | \& |# indented lines are glued \& |# the next three lines mean 'somekey part1 part2 part3' \& |somekey part1 \& | part2 \& | part3 \& | \& |# lines starting with a '+' are concatenated \& |# the next three lines mean 'somekey part1part2part3' \& |somekey part1 \& |+ part2 \& |+ part3 \& | \& |# lines starting with a '.' are glued too \& |# don't use a '.' on a line by itself \& |# 'somekey' gets the value "part1\en part2\en part3" \& |somekey part1 \& |. part2 \& |. part3 \& +-------------------------------------------------- .Ve .Sh "required entries" .IX Subsection "required entries" .IP "project_name \fIname\fR" 4 .IX Item "project_name name" Specify a short plaintext name for the project. .Sp .Vb 2 \& project_name Apache \& project_name CTAN .Ve .IP "project_url \fIurl\fR" 4 .IX Item "project_url url" Specify an url pointing to the 'home' of the project. .Sp .Vb 1 \& project_url http://www.apache.org/ .Ve .IP "mirror_list \fIfile-name\fR" 4 .IX Item "mirror_list file-name" Specify the file containing the mirrors to probe. .Sp .Vb 1 \& mirror_list /path/to/mirror-list .Ve .Sp If your mirror list is generated by a program, use .Sp .Vb 1 \& mirror_list /path/to/program arg1 ... | .Ve .Sp Two formats are supported : .RS 4 .IP "* plain : lines like" 4 .IX Item "plain : lines like" .Vb 3 \& us http://www.tux.org/ [email] ... \& nl http://apache.cs.uu.nl/dist/ [email] ... \& nl rsync://archive.cs.uu.nl/apache-dist/ [email] ... .Ve .IP "* apache : lines like those in the apache mirrors.list" 4 .IX Item "apache : lines like those in the apache mirrors.list" .Vb 2 \& ftp us ftp://ftp.tux.org/pub/net/apache/dist/ user@tux.org ... \& http nl http://apache.cs.uu.nl/dist/ user@cs.uu.nl ... .Ve .RE .RS 4 .Sp Note that in style 'plain' the third item is reserved for an optional email address : the site's contact address. .Sp Specify the required format with option \f(CW\*(C`list_style\*(C'\fR (see below). The default style is 'plain'. .RE .IP "web_page \fIfile-name\fR" 4 .IX Item "web_page file-name" Specify where the html report page is written. .IP "icons \fIdirectory-name\fR" 4 .IX Item "icons directory-name" Specify the directory where the icons can be found, relative to the \fIweb_page\fR, or relative to the \&\s-1DOCUMENTROOT\s0 of the web server. .Sp If/when the \fIweb_page\fR lives in directory \f(CW\*(C`.../mirmon/\*(C'\fR and the icons live in directory \f(CW\*(C`.../mirmon/icons/\*(C'\fR, specify .Sp .Vb 1 \& icons icons .Ve .Sp If/when the icons live in \f(CW\*(C`/path/to/DOCUMENTROOT/icons/mirmon/\*(C'\fR, specify .Sp .Vb 1 \& icons /icons/mirmon .Ve .IP "probe \fIprogram + arguments\fR" 4 .IX Item "probe program + arguments" Specify the program+args to probe the mirrors. Example: .Sp .Vb 1 \& probe /usr/bin/wget -q -O - -T %TIMEOUT% -t 1 %URL%TIME.txt .Ve .Sp Before the program is started, \f(CW%TIMEOUT\fR% and \f(CW%URL\fR% are substituted with the proper timeout and url values. .Sp Here it is assumed that each hour the root server writes a timestamp in /path/to/archive/TIME.txt, for instance with a crontab entry like .Sp .Vb 1 \& 42 * * * * perl -e 'print time, "\en"' > /path/to/archive/TIME.txt .Ve .Sp Mirmon reads one line of output from the probe and interprets the first word on that line as a timestamp ; for example : .Sp .Vb 3 \& 1043625600 \& 1043625600 Mon Jan 27 00:00:00 2003 \& 1043625600 www.apache.org Mon Jan 27 00:00:00 2003 .Ve .Sp Mirmon is distributed with a program \f(CW\*(C`probe\*(C'\fR that handles ftp, http and rsync urls. .IP "state \fIfile-name\fR" 4 .IX Item "state file-name" Specify where the file containing the state is written. .Sp The program reads this file on startup and writes the file when mirrors are probed (\-get is specified). .IP "countries \fIfile-name\fR" 4 .IX Item "countries file-name" Specify the file containing the country codes; The file should contain lines like .Sp .Vb 2 \& us - United States \& nl - Netherlands .Ve .Sp The mirmon package contains a recent \s-1ISO\s0 list. .Sp \&\fIFake\fR domains like \fIBackup\fR, \fIMaster\fR are allowed, and are listed first in the report ; lowercase-first fake domains (like \fIbackup\fR) are listed last. .Sh "optional entries" .IX Subsection "optional entries" .IP "max_probes \fInumber\fR" 4 .IX Item "max_probes number" Optionally specify the number of parallel probes (default 25). .IP "timeout \fIseconds\fR" 4 .IX Item "timeout seconds" Optionally specify the timeout for the probes (default 300). .Sp After the last probe is started, the program waits for + 10 seconds, cleans up and exits. .IP "project_logo \fIlogo\fR" 4 .IX Item "project_logo logo" Optionally specify (the \s-1SRC\s0 of the \s-1IMG\s0 of) a logo to be placed top right on the page. .Sp .Vb 2 \& project_logo /icons/apache.gif \& project_logo http://www.apache.org/icons/... .Ve .IP "htm_head \fIhtml\fR" 4 .IX Item "htm_head html" Optionally specify some \s-1HTML\s0 to be placed before . .Sp .Vb 2 \& htm_head \& .Ve .IP "htm_top \fIhtml\fR" 4 .IX Item "htm_top html" Optionally specify some \s-1HTML\s0 to be placed near the top of the page. .Sp .Vb 1 \& htm_top testing 1, 2, 3 .Ve .IP "htm_foot \fIhtml\fR" 4 .IX Item "htm_foot html" Optionally specify \s-1HTML\s0 to be placed near the bottom of the page. .Sp .Vb 4 \& htm_foot \&
\& \&
.Ve .IP "put_histo top|bottom|nowhere" 4 .IX Item "put_histo top|bottom|nowhere" Optionally specify where the age histogram must be placed. The default is 'top'. .IP "min_poll \fItime-spec\fR" 4 .IX Item "min_poll time-spec" For 'min_poll' see next item. A \fItime-spec\fR is a number followed by a unit 's' (seconds), or 'm' (minutes), or 'h' (hours), or 'd' (days). For example '3d' (three days) or '36h' (36 hours). .IP "max_poll \fItime-spec\fR" 4 .IX Item "max_poll time-spec" Optionally specify the maximum probe interval. When the program is called with option '\-get update', all sites are probed which are : .RS 4 .IP "* new" 4 .IX Item "new" the site appears in the list, but there is no known state .IP "* bad" 4 .IX Item "bad" the last probe of the site was unsuccessful .IP "* old" 4 .IX Item "old" the last probe was more than 'max_poll' ago. .RE .RS 4 .Sp Sites are not probed if the last probe was less than 'min_poll' ago. So, if you specify .Sp .Vb 2 \& min_poll 4h \& max_poll 12h .Ve .Sp the 'reachable' sites are probed twice daily and the 'unreachable' sites are probed at most six times a day. .Sp The default 'min_poll' is '1h' (1 hour). The default 'max_poll' is '4h' (4 hours). .RE .IP "min_sync \fItime-spec\fR" 4 .IX Item "min_sync time-spec" Optionally specify how often the mirrors are required to make an update. .Sp The default 'min_sync' is '1d' (1 day). .IP "max_sync \fItime-spec\fR" 4 .IX Item "max_sync time-spec" Optionally specify the maximum allowable sync interval. .Sp Sites exceeding the limit will be considered 'old'. The default 'max_sync' is '2d' (2 days). .IP "always_get \fIregion ...\fR" 4 .IX Item "always_get region ..." Optionally specify a list of regions that must be probed always. .Sp .Vb 1 \& always_get Master Tier1 .Ve .Sp This is intended for \fIfake regions\fR like \fIMaster\fR etc. .IP "no_randomize" 4 .IX Item "no_randomize" Mirmon tries to balance the probe load over the hourly mirmon runs. If the current run has a below average number of mirrors to probe, mirmon probes a few extra, randomly chosen mirrors, picked from the runs that have the highest load. .Sp If you don't want this behaviour, use \fBno_randomize\fR. .IP "no_add_slash" 4 .IX Item "no_add_slash" If the url part of a line in the mirror_list doesn't end in a slash ('/'), mirmon adds a slash and issues a warning unless it is in quiet mode. .Sp If you don't want this behaviour, use \fBno_add_slash\fR. .IP "list_style plain|apache" 4 .IX Item "list_style plain|apache" Optionally specify the format ('plain' or 'apache') of the mirror\-list. .Sp See the description of 'mirror_list' above. The default list_style is 'plain'. .IP "site_url \fIsite\fR \fIurl\fR" 4 .IX Item "site_url site url" Optionally specify a substitute url for a site. .Sp When access to a site is restricted (in Australia, for instance), another (sometimes secret) url can be used to probe the site. The of an url is the part between '://' and the first '/'. .IP "env \fIkey\fR \fIvalue\fR" 4 .IX Item "env key value" Optionally specify an environment variable. .IP "include \fIfile-name\fR" 4 .IX Item "include file-name" Optionally specify a file to include. .Sp The specified file is processed 'in situ'. After the specified file is read and processed, config processing is resumed in the file where the \&\f(CW\*(C`include\*(C'\fR was encountered. The include depth is unlimited. However, it is a fatal error to include a file twice under the same name. .IP "show" 4 .IX Item "show" When the config processor encounters the 'show' command, it dumps the content of the current config to standout, if option \&\f(CW\*(C`\-v\*(C'\fR is specified. This is intented for debugging. .IP "exit" 4 .IX Item "exit" When the config processor encounters the 'exit' command, it terminates the program. This is intented for debugging. .SH "STATE FILE FORMAT" .IX Header "STATE FILE FORMAT" The state file consists of lines; one line per site. Each line consists of white space separated fields. The seven fields are : .IP "* field 1 : url" 4 .IX Item "field 1 : url" The url as given in the mirror list. .IP "* field 2 : age" 4 .IX Item "field 2 : age" The mirror's timestamp found by the last successful probe, or 'undef' if no probe was ever successful. .IP "* field 3 : status last probe" 4 .IX Item "field 3 : status last probe" The status of the last probe, or 'undef' if the mirror was never probed. .IP "* field 4 : time last successful probe" 4 .IX Item "field 4 : time last successful probe" The timestamp of the last successful probe or 'undef' if the mirror was never successfully probed. .IP "* field 5 : probe history" 4 .IX Item "field 5 : probe history" The probe history is a list of 's' (for success) and 'f' (for failure) characters indicating the result of the probe. New results are appended whenever the mirror is probed. .IP "* field 6 : state history" 4 .IX Item "field 6 : state history" The state history consists of a timestamp, a '\-' char, and a list of chars indicating a past status: 's' (fresh), 'b' (oldish), 'f' (old), \&'z' (bad) or 'x' (skip). The timestamp indicates when the state history was last updated. The current status of the mirror is determined by the mirror's age and a few configuration parameters (min_sync, max_sync, max_poll). The state history is updated when the mirror is probed. If the last update of the history was less than 24 hours ago, the last status is replaced by the current status. If the last update of the history was more than 24 hours ago, the current status is appended to the history. One or more 'skip's is inserted, if the timestamp is two or more days old (when mirmon hasn't run for more than two days). .IP "* field 7 : last probe" 4 .IX Item "field 7 : last probe" The timestamp of the last probe, or 'undef' if the mirror was never probed. .SH "INSTALLATION" .IX Header "INSTALLATION" .Sh "general" .IX Subsection "general" .IP "* Note: The (empty) state file must exist before mirmon runs." 4 .IX Item "Note: The (empty) state file must exist before mirmon runs." .PD 0 .IP "* The mirmon repository is here :" 4 .IX Item "The mirmon repository is here :" .PD .Vb 1 \& https://svn.science.uu.nl/repos/project.mirmon/trunk/ .Ve .IP "* The mirmon tarball is here :" 4 .IX Item "The mirmon tarball is here :" .Vb 1 \& http://www.staff.science.uu.nl/~penni101/mirmon/mirmon.tar.gz .Ve .Sh "installation suggestions" .IX Subsection "installation suggestions" To install and configure mirmon, take the following steps : .IP "* First, make the webdir :" 2 .IX Item "First, make the webdir :" .Vb 2 \& cd DOCUMENTROOT \& mkdir mirmon .Ve .Sp For \fI\s-1DOCUMENTROOT\s0\fR, substitute the full pathname of the document root of your webserver. .IP "* Check out the mirmon repository :" 2 .IX Item "Check out the mirmon repository :" .Vb 2 \& cd /usr/local/src \& svn checkout REPO mirmon .Ve .Sp where .Sp .Vb 1 \& REPO = https://svn.science.uu.nl/repos/project.mirmon/trunk/ .Ve .Sp or download the package and unpack it. .IP "* Chdir to directory mirmon :" 2 .IX Item "Chdir to directory mirmon :" .Vb 1 \& cd mirmon .Ve .IP "* Create the (empty) state file :" 2 .IX Item "Create the (empty) state file :" .Vb 1 \& touch state.txt .Ve .IP "* Install the icons in the webdir :" 2 .IX Item "Install the icons in the webdir :" .Vb 2 \& mkdir DOCUMENTROOT/mirmon/icons \& cp icons/* DOCUMENTROOT/mirmon/icons .Ve .ie n .IP "* Create a mirror list ""mirror_list"" ;" 2 .el .IP "* Create a mirror list \f(CWmirror_list\fR ;" 2 .IX Item "Create a mirror list mirror_list ;" Use your favorite editor, or genererate the list from an existing database. .Sp .Vb 3 \& nl http://archive.cs.uu.nl/your-project/ contact@cs.uu.nl \& uk http://mirrors.this.org/your-project/ mirrors@this.org \& us http://mirrors.that.org/your-project/ mirrors@that.org .Ve .Sp The email addresses are optional. .ie n .IP "* Create a mirmon config file ""mirmon.conf"" with your favorite editor." 2 .el .IP "* Create a mirmon config file \f(CWmirmon.conf\fR with your favorite editor." 2 .IX Item "Create a mirmon config file mirmon.conf with your favorite editor." .Vb 9 \& # lines must start in the first column ; no leading white space \& project_name .... \& project_url .... \& mirror_list mirror_list \& state state.txt \& countries countries.list \& web_page DOCUMENTROOT/mirmon/index.html \& icons /mirmon/icons \& probe /usr/bin/wget -q -O - -T %TIMEOUT% -t 1 %URL%TIME.txt .Ve .Sp This assumes the project's timestamp is in file \f(CW\*(C`TIME.txt\*(C'\fR. .IP "* If you have rsync urls, change the probe line to :" 2 .IX Item "If you have rsync urls, change the probe line to :" .Vb 1 \& probe perl /usr/local/src/mirmon/probe -t %TIMEOUT% %URL%TIME.txt .Ve .IP "* Run mirmon :" 2 .IX Item "Run mirmon :" .Vb 1 \& perl mirmon -v -get all .Ve .Sp The mirmon report should now be in 'DOCUMENTROOT/mirmon/index.html' .Sp .Vb 1 \& http://www.your.project.org/mirmon/ .Ve .IP "* If/when, at a later date, you want to upgrade mirmon :" 2 .IX Item "If/when, at a later date, you want to upgrade mirmon :" .Vb 3 \& cd /usr/local/src/mirmon \& svn status -u \& svn up .Ve .SH "SEE ALSO" .IX Header "SEE ALSO" mirmon.pm(3) .SH "AUTHOR" .IX Header "AUTHOR" .Vb 4 \& (c) 2003-2016 Henk P. Penning \& Faculty of Science, Utrecht University \& http://www.staff.science.uu.nl/~penni101/ -- penning@uu.nl \& mirmon-2.11 - Sat Jul 23 09:12:31 2016 ; henkp .Ve