.TH "weaver" 1 "" "CCTools 7.0.9 FINAL" "Cooperative Computing Tools"

.SH NAME
.LP
\fBweaver\fP - workflow engine for executing distributed workflows

.SH SYNOPSIS
.LP
\FC\fBweaver [options] <weaverfile>\fP\FT

.SH DESCRIPTION
.LP

\fBWeaver\fP is a high level interface to \fBmakeflow\fP. A
\fBweaver\fP input file is written in \fBpython\fP, with the
definition of functions to be applied on sets of files. \fBweaver\fP
interprets this input file and generates a workflow specification that
can be executed by \fBmakeflow\fP. This allows an straightforward
implementation of different workflow execution patterns, such as
MapReduce, and AllPairs.

.fam C
.nf
.nh
.IP "" 8


				      /--------\
				    +-+ Python |
				    | \---+----/
+---------------------------------+ |     | Generate DAG
|	      Weaver		  +-+     v
+---------------------------------+   /-------\
|	     Makeflow		  +---+  DAG  |
+--------+-----------+-----+------+   \-------/
| Condor | WorkQueue | SGE | Unix +-+     | Dispatch Jobs
+--------+-----------+-----+------+ |     v
				    | /-------\
				    +-+ Jobs  |
				      \-------/

.fi
.hy
.fam
.P

.SH OPTIONS
.LP

By default, running \FCweaver\FT on a <weaverfile> generates an
input file for \FCmakeflow\FT, <Makeflow>, and a directory,
<_Stash>, in which intermediate files are stored.

General options:
.LP
.TP
.B \ -h
.
Give help information.
.TP
.B \ -W
.
Stop on warnings.
.TP
.B \ -g
.
Include debugging symbols in DAG.
.TP
.B \ -I
.
Do not automatically import built-ins.
.TP
.B \ -N
.
Do not normalize paths.
.TP
.BI \ -b \ <options>
.
Set batch job options (cpu, memory, disk, batch, local, collect).
.TP
.BI \ -d \ <subsystem>
.
Enable debugging for subsystem.
.TP
.BI \ -o \ <log_path>
.
Set log path (default: stderr).
.TP
.BI \ -O \ <directory>
.
Set stash output directory (default <_Stash>).


Optimization Options:
.LP

.TP
.B \ -a
.
Automatically nest abstractions.
.TP
.BI \ -t \ <group_size>
.
Inline tasks based on group size.

Engine Options:

.LP
.TP
.B \ -x
.
Execute DAG using workflow engine after compiling.
.TP
.BI \ -e \ <arguments>
.
Set arguments to workflow engine when executing.
.TP
.BI \ -w wrapper \ <>
.
Set workflow engine wrapper.


.SH EXIT STATUS
.LP

On success, returns zero.  On failure, returns non-zero.

.SH EXAMPLES
.LP

\fBWeaver\fP expresses common workflow patterns succinctly. For
example, with only the following three lines of code we can express a
\fBmap\fP pattern, in which we convert some images to the jpeg format:

.fam C
.nf
.nh
.IP "" 8

convert = ParseFunction('convert {IN} {OUT}')
dataset = Glob('/usr/share/pixmaps/*.xpm')
jpgs    = Map(convert, dataset, '{basename_woext}.jpg')

.fi
.hy
.fam
.P

Please refer to \FCcctools/doc/weaver_examples\FT for further information.

The Cooperative Computing Tools are Copyright (C) 2003-2004 Douglas Thain and Copyright (C) 2005-2015 The University of Notre Dame.  This software is distributed under the GNU General Public License.  See the file COPYING for details.