.\" Automatically generated by Pod::Man 4.07 (Pod::Simple 3.32)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is >0, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.if !\nF .nr F 0
.if \nF>0 \{\
.    de IX
.    tm Index:\\$1\t\\n%\t"\\$2"
..
.    if !\nF==2 \{\
.        nr % 0
.        nr F 2
.    \}
.\}
.\"
.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
.\" Fear.  Run.  Save yourself.  No user-serviceable parts.
.    \" fudge factors for nroff and troff
.if n \{\
.    ds #H 0
.    ds #V .8m
.    ds #F .3m
.    ds #[ \f1
.    ds #] \fP
.\}
.if t \{\
.    ds #H ((1u-(\\\\n(.fu%2u))*.13m)
.    ds #V .6m
.    ds #F 0
.    ds #[ \&
.    ds #] \&
.\}
.    \" simple accents for nroff and troff
.if n \{\
.    ds ' \&
.    ds ` \&
.    ds ^ \&
.    ds , \&
.    ds ~ ~
.    ds /
.\}
.if t \{\
.    ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
.    ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
.    ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
.    ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
.    ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
.    ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
.\}
.    \" troff and (daisy-wheel) nroff accents
.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
.ds ae a\h'-(\w'a'u*4/10)'e
.ds Ae A\h'-(\w'A'u*4/10)'E
.    \" corrections for vroff
.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
.    \" for low resolution devices (crt and lpr)
.if \n(.H>23 .if \n(.V>19 \
\{\
.    ds : e
.    ds 8 ss
.    ds o a
.    ds d- d\h'-1'\(ga
.    ds D- D\h'-1'\(hy
.    ds th \o'bp'
.    ds Th \o'LP'
.    ds ae ae
.    ds Ae AE
.\}
.rm #[ #] #H #V #F C
.\" ========================================================================
.\"
.IX Title "paexec 1"
.TH paexec 1 "2015-07-26" "" ""
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
paexec \- parallel executor, distribute tasks over network or CPUs
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
\&\fBpaexec\fR \fI[options]\fR
.PP
\&\fBpaexec\fR \-C \fI[options]\fR \fIcommand\fR \fI[args...]\fR
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
Suppose you have a long list of \s-1AUTONOMOUS\s0 tasks that need to be
done, for example, you want to convert thousands of .wav audio files
to .ogg format.  Also suppose that multiple CPUs are
available, e.g. multi-CPU \s-1SMP\s0 system (or modern multikernel \s-1CPU\s0) or a
cluster consisting of individual computers connected to the network or
internet. \fBpaexec\fR can efficiently do this work, that
is \fBpaexec\fR efficiently distributes different tasks to different
processors (or computers), receives the results of processing from them
and sends these results to stdout.
.PP
There are several notions that should be defined: \fItask\fR, \fIcommand\fR,
\&\fItransport\fR, \fInode\fR.
.PP
\&\fITasks\fR are read by \fBpaexec\fR from stdin and are represented as one
line of text, i.e. one input line \- one task.
.PP
\&\fInode\fR identifier \- remote computer or \s-1CPU\s0 identifier, for
example \s-1CPU\s0 ordinal number or computer's \s-1DNS\s0 name like
node12.cluster.company.com.
.PP
\&\fICommand\fR \- user's program that reads one-line task from stdin and sends
multiline result to stdout where an empty line means
JOB_IS_DONE_\|_I_AM_READY_FOR_THE_NEXT_ONE. After sending the empty line
to stdout, stdout \s-1MUST BE FLUSHED.\s0 Remember that empty line
\&\s-1MUST NOT\s0 appears in general result lines. Otherwise \fBpaexec\fR may hang
because of deadlock.
.PP
\&\fITransport\fR \- special program that helps to run \fIcommand\fR on the
\&\fInode\fR. It takes the \fInode\fR identifier as its first
argument and \fIcommand\fR with its arguments as the rest.  For example,
is '/usr/bin/ssh'. Both \fItransport\fR and \fIcommand\fR may 
be specified with their arguments, i.e. '/usr/bin/ssh \-x' is allowed
as a \fItransport\fR program.
.PP
Algorithm. \fICommands\fR are run on each \fInode\fR with a help of
\&\fItransport\fR program. Then, \fItasks\fR are read from stdin line-by-line
(one task per line)
and are sent to free \fInode\fR (exactly one task per node at
a time). At the same time result lines are read from \fIcommand's\fR
stdout and are output to \fBpaexec's\fR stdout. When an empty line is
obtained from the \fInode\fR (this means that \fInode\fR finished
its job) it is marked as free and becomes ready for the next
job. These steps repeat until the end of stdin is reached and all
\&\fInodes\fR finish their job.
.PP
More formally (to better understand how paexec works):
.PP
.Vb 10
\&   run_command_on_each_node
\&   mark_all_nodes_as_free
\&   while not(end_of_stdin) or not(all_nodes_are_free)
\&      while there_is_free_node/i and not(end_of_stdin)
\&         task = read_task_from_stdin
\&         send_task_to_node(task, i)
\&         mark_node_as_busy(i)
\&      end
\&      while result_line_from_node_is_available/i
\&         result = read_result_line_from_node(i)
\&         send_line_to_stdout(result)
\&         if is_empty_line(result)
\&            # end of job
\&            mark_node_as_free(i)
\&         end
\&      end
\&   end
\&   close_command_on_each_node
.Ve
.PP
Note that \fIcommand\fR that does your actual task is run once (per node),
it is not restarted for every task.
.PP
Also note that output contains result lines (obtained from different
\&\fInodes\fR) in the mixed order. That is, the first line of the
output may contain a result line obtain from the first \fInode\fR,
the second line of output \- from the second \fInode\fR, but the
third output line may contain result line from the first \fInode\fR
again. It is also not guaranteed that the first line of output will be
from the first \fInode\fR or from the first \fItask\fR. All result
lines are output as soon as they are read by \fBpaexec\fR, i.e as soon as
they are ready to be output. \fBpaexec\fR works this way for the
efficiency reasons.  You can play with \fI\-l\fR, \fI\-r\fR and \fI\-p\fR options to see
what happens.
.SH "OPTIONS"
.IX Header "OPTIONS"
.IP "\fB\-h\fR" 6
.IX Item "-h"
Display help information.
.IP "\fB\-V\fR" 6
.IX Item "-V"
Display version information.
.IP "\fB\-c\fR \fIcommand\fR" 6
.IX Item "-c command"
Command with its arguments.
.IP "\fB\-C\fR" 6
.IX Item "-C"
Command with its arguments are specified following the options.
.IP "\fB\-t\fR \fItransport\fR" 6
.IX Item "-t transport"
Transport command
.IP "\fB\-n\fR \fI+number\fR" 6
.IX Item "-n +number"
A number of commands to run in parallel.
.IP "\fB\-n\fR \fInodes\fR" 6
.IX Item "-n nodes"
List of nodes separated by space character. The first character must be
alphanumeric, `_' or `/'. All other characters are reserved
for future extensions.
.IP "\fB\-n\fR \fI:filename\fR" 6
.IX Item "-n :filename"
Filename containing list of nodes, one per line.
.IP "\fB\-x\fR" 6
.IX Item "-x"
Run command specificed by \fI\-c\fR for each task. Its stdout is passed to \fBpaexec\fR.
If both \f(CW\*(C`\-x\*(C'\fR and \f(CW\*(C`\-g\*(C'\fR are specified, task is considered failed
if command's exit status is non-zero.
.IP "\fB\-X\fR" 6
.IX Item "-X"
Implies \fI\-x\fR and ignore calculator's stdout.
.IP "\fB\-r\fR" 6
.IX Item "-r"
Include node identifier or node number (0\-based) to the
output, i.e. id/number of node that produces this particular
output line.  This identifier or number appears before line number if
\&\fI\-l\fR is also applied. Space character is used as a separator.
.IP "\fB\-l\fR" 6
.IX Item "-l"
Include a 0\-based task number (input line number) to the output,
i.e. line number from
which this particular output line was produced.  It appears before pid
if \fI\-p\fR is also applied. Space character is used as a separator.
.IP "\fB\-p\fR" 6
.IX Item "-p"
Include pid of paexec's subprocess that communicates with
\&\fInode+command\fR to the output. Pid prepends the actual result
line. Space character is used as a separator.
.IP "\fB\-e\fR" 6
.IX Item "-e"
When end-of-task marker is obtained from node, an empty line is
printed to stdout. This option may be useful together with \fI\-l\fR
and/or \fI\-r\fR.
.IP "\fB\-E\fR" 6
.IX Item "-E"
Imply \fB\-e\fR and flushes stdout.
.IP "\fB\-d\fR" 6
.IX Item "-d"
Turn on a debugging mode (for debugging purposes only)
.IP "\fB\-i\fR" 6
.IX Item "-i"
Copy input lines (i.e. tasks) to stdout.
.IP "\fB\-I\fR" 6
.IX Item "-I"
Imply \fB\-i\fR and flushes stdout.
.IP "\fB\-s\fR|\fB\-g\fR" 6
.IX Item "-s|-g"
Orgraph of tasks (partially ordered set) is read from stdin.
.Sp
Instead of autonomous tasks, graph of the tasks is read from stdin.
In this mode every task can either \s-1FAIL\s0 or \s-1SUCCEED.\s0
As always an empty line output by \fIcommand\fR means \fIend of task\fR.
The line before it shows an \s-1EXIT STATUS\s0 of the task.
The word \*(L"failure\*(R" means failure, \*(L"success\*(R" \- success and
\&\*(L"fatal\*(R" means that the current task is reassigned to another node
(and restarted, of course) (see option \-z).
See examples/1_div_x/cmd for the sample.
An input line (paexec's stdin) should contain
either single task without spaces inside
or two tasks separated by single space character,
e.g. task1<\s-1SPC\s0>task2. task1<\s-1SPC\s0>task2 line means that task1 must be done
before task2 and it is mandatory, that is if task1 \fIfail\fR all dependent
tasks (including task2) are also failed recursively.
Tasks having dependencies are started only after all dependencies
are succeeded. When a task succeeds paexec outputs \*(L"success\*(R" word
just before end_of_task marker (see \-e or \-E), otherwise \*(L"failure\*(R"
word is output followed by a list of tasks failed because of it.
.Sp
.Vb 1
\& Samples:
\&
\&   tasks (examples/make_package/tasks file)
\&
\&     textproc/dictem
\&     devel/autoconf wip/libmaa
\&     devel/gmake wip/libmaa
\&     wip/libmaa wip/dict\-server
\&     wip/libmaa wip/dict\-client
\&     devel/m4 wip/dict\-server
\&     devel/byacc wip/dict\-server
\&     devel/byacc wip/dict\-client
\&     devel/flex wip/dict\-server
\&     devel/flex wip/dict\-client
\&     devel/glib2
\&     devel/libjudy
\&
\&   command (examples/make_package/cmd_\|_flex)
\&
\&     #!/usr/bin/awk \-f
\&     {
\&        print $0
\&        if ($0 == "devel/flex")
\&           print "failure"
\&        else
\&           print "success"
\&
\&        print ""       # end of task marker
\&        fflush()
\&     }
\&
\&   output of "paexec \-s \-l \-c cmd_\|_flex \-n +10 \e
\&              < tasks"
\&
\&     3 devel/autoconf
\&     3 success
\&     4 devel/gmake
\&     4 success
\&     7 devel/m4
\&     7 success
\&     8 devel/byacc
\&     8 success
\&     9 devel/flex
\&     9 failure
\&     9 devel/flex wip/dict\-server wip/dict\-client 
\&     10 devel/glib2
\&     10 success
\&     11 devel/libjudy
\&     11 success
\&     1 textproc/dictem
\&     1 success
\&     2 wip/libmaa
\&     2 success
.Ve
.IP "\fB\-z\fR" 6
.IX Item "-z"
If applied, read/\fIwrite\fR\|(2) operations from/to nodes becomes not
critical. In case paexec has lost connection to the node, it will
reassign failed task to another node and, if \-s applied, will output
\&\*(L"fatal\*(R" string to stdout (\*(L"success\*(R" + \*(L"failure\*(R" + \*(L"fatal\*(R").  This
makes paexec resistant to the I/O errors, as a result you can create
paexec clusters even over network consisting of unreliable hosts
(Internet?). Failed hosts are marked as such and will not be used
during the current run of paexec.
.IP "\fB\-Z\fR \fItimeout\fR" 6
.IX Item "-Z timeout"
When \fI\-z\fR applied, if a \fIcommand\fR fails, appropriate node is marked
as broken and is excluded from the following task distribution. But if
\&\fB\-Z\fR applied, every \fItimeout\fR seconds an attempt to rerun a comand
on a failed node is made. \fI\-Z\fR implies \fI\-z\fR. This option makes
possible to organize clusters over unreliable networks/hardware.
.IP "\fB\-w\fR" 6
.IX Item "-w"
If \fI\-Z\fR option were applied, \fBpaexec\fR exits with error
if \fB\s-1ALL\s0\fR nodes failed. With \fB\-w\fR it will not exit
and will wait for restoring nodes.
.IP "\fB\-m\fR s=\fIsuccess\fR" 6
.IX Item "-m s=success"
.PD 0
.IP "\fB\-m\fR f=\fIfailure\fR" 6
.IX Item "-m f=failure"
.IP "\fB\-m\fR F=\fIfatal\fR" 6
.IX Item "-m F=fatal"
.IP "\fB\-m\fR t=\fIeot\fR" 6
.IX Item "-m t=eot"
.IP "\fB\-m\fR d=\fIdelimiter\fR" 6
.IX Item "-m d=delimiter"
.PD
Set alternative string for 'success', 'failure', 'fatal', '' (end of
task) and ' ' (task delimiter character). An empty string for 'fatal'
means it will not be output to stdout in case of fatal error.
.IP "\fB\-W\fR \fInum\fR" 6
.IX Item "-W num"
When multiple machines or CPUs are used for tasks processing, it makes sense
to start \*(L"heavier\*(R" tasks as soon as possible in order to minimize total 
calculatiion time.
If \fB\-W\fR is specified, special weight is assigned to each tasks
which is used for reordering tasks.
If \fInum\fR is 0, weights themselves are used for reordering tasks.
The bigger weight is, the more priority of the task is.
If \fInum\fR is 1, the total weight of task is a sum of its own weight
(specified on input)
and weights of all tasks depending on it directly or indirectly.
If \fInum\fR is 2, the total weight of task is a maximum value of task's own
weight
and weights of all tasks depending on it directly or indirectly.
Weights are specified with a help of \*(L"weight:\*(R" keyword. If
weight is not specified, it defaults to 1.  The following is the example
for input graph of tasks with weights.
.Sp
.Vb 10
\&  weight: gtk2 30
\&  weight: glib2 20
\&  gtk2 firefox
\&  weight: firefox 200
\&  glib2 gtk2
\&  weight: qt4 200
\&  weight: kcachegrind 2
\&  qt4 kcachegrind
\&  qt4 djview4
\&  tiff djview4
\&  png djview4
\&  weight: twm 1
\&  weight: gqview 4
.Ve
.IP "\fB\-y\fR" 6
.IX Item "-y"
If applied, the magic string is used as an end-of-task marker instead of
empty line.  It is unlikely that this line appears on calculator's
output.  This option has higher priority than \s-1PAEXEC_EOT\s0 environment
variable.
.SH "EXAMPLES"
.IX Header "EXAMPLES"
.IP "1." 6
.Vb 3
\& paexec \-t \*(Aq/usr/bin/ssh \-x\*(Aq \-n \*(Aqhost1 host2 host3\*(Aq \e
\&        \-le \-g \-c calculate\-me < tasks.txt |
\& paexec_reorder \-Mf \-Sl
.Ve
.IP "2." 6
.Vb 1
\& ls \-1 *.wav | paexec \-x \-n +4 \-c \*(Aqoggenc \-Q\*(Aq
.Ve
.IP "3." 6
.Vb 1
\& ls \-1 *.wav | paexec \-xCil \-n+4 flac \-f \-\-silent
.Ve
.IP "4." 6
.Vb 3
\& { uname \-s; uname \-r; uname \-m; } |
\& paexec \-x \-lp \-n+2 \-c banner |
\& paexec_reorder \-l
.Ve
.PP
For more examples see paexec.pdf and examples/ subdirectory in the distribution.
.SH "NOTES"
.IX Header "NOTES"
\&\fIselect\fR\|(2) system call and non-blocking \fIread\fR\|(2) are used to read result
lines from \fInodes\fR.
.PP
At the moment blocking \fIwrite\fR\|(2) is used to send \fItask\fR to the
\&\fInode\fR. This may slow down an entire processing if \fItasks\fR are
too big. So, it is recommended to use shorter \fItasks\fR, for example,
filename or \s-1URI \s0(several tens of bytes in size) instead of
multi-megabyte content. Though this may be fixed in the future.
.PP
Original paexec tarball contains a number of sample of use in
presentation/paexec.pdf file. After installation you can find this
file under share/doc/paexec/paexec.pdf or nearby.
.SH "ENVIRONMENT"
.IX Header "ENVIRONMENT"
.IP "\fI\s-1PAEXEC_BUFSIZE\s0\fR" 6
.IX Item "PAEXEC_BUFSIZE"
Overrides the compile time \fIinitial\fR size for internal buffers used
to store tasks and the result lines. Versions of \fBpaexec\fR prior to
0.9.0 used this value as a \fImaximum\fR buffer size.
Now internal buffers are resized automatically.
If unsure, do not set \s-1PAEXEC_BUFSIZE\s0 variable.
See the default value in Makefile.
.IP "\fI\s-1PAEXEC_ENV\s0\fR" 6
.IX Item "PAEXEC_ENV"
A list of variables passed to calculator.
.IP "\fI\s-1PAEXEC_EOT\s0\fR" 6
.IX Item "PAEXEC_EOT"
This variable sets the end-of-task marker which is an empty line by default.
Also, through this variable an end-of-task marker is passed to all calculators.
.IP "\fI\s-1PAEXEC_NODES\s0\fR" 6
.IX Item "PAEXEC_NODES"
Unless option \fB\-n\fR was applied, this variables specifies the nodes.
.IP "\fI\s-1PAEXEC_TRANSPORT\s0\fR" 6
.IX Item "PAEXEC_TRANSPORT"
Unless option \fB\-t\fR was applied, this variables specifies the transport.
.SH "BUGS/FEEDBACK"
.IX Header "BUGS/FEEDBACK"
Please send any comments, questions, bug reports etc. to me by e\-mail
or (even better) register them at sourceforge project home.  Feature
requests are also welcomed.
.SH "HOME"
.IX Header "HOME"
<http://sourceforge.net/projects/paexec>
.SH "SEE ALSO \fIssh\fP\|(1) \fIrsh\fP\|(1) \fIselect\fP\|(2) \fIread\fP\|(2) \fIwrite\fP\|(2)"
.IX Header "SEE ALSO ssh rsh select read write"