PEGASUS-KICKSTART(1) | PEGASUS-KICKSTART(1) |
NAME¶
pegasus-kickstart - run an executable in a more universal environment.SYNOPSIS¶
pegasus-kickstart [-n tr] [-N dv] [-H] [-R site] [-W | -w dir] [ -L lbl -T iso] [-s p | @fn] [-S p | @fn] [-i fn] [ -o fn] [-e fn] [-X] [-l fn sz] [-F] (-I fn | app [appflags]) pegasus-kickstart -V
DESCRIPTION¶
The pegasus-kickstart executable is a light wrapper program which connects the stdin, stdout and stderr file handles for grid jobs on the remote site, and reports back the remote application termination condition.OPTIONS¶
-n trIn order to associate the minimal performance
information of the job with the invocation records, the jobs needs to carry
which transformation was responsible for producing it. The format is
the textual notation for fully-qualified definition names, like
namespace::name:version, with only the name portion being mandatory.
There is no default. If no value is given, "null" will be
reported.
-N dv
The jobs may carry which instantiation of a
transformation was responsible for producing it. The format is the textual
notation for fully-qualified definition names, like namespace::name:version,
with only the name portion being mandatory.
There is no default. If no value is given, "null" will be
reported.
-H
This option avoids pegasus-kickstart writing
the XML preamble (entity), if you need to combine multiple pegasus-kickstart
records into one document.
Additionally, if specified, the environment and the resource usage segments will
not be written, assuming that a in a concatenated record version, the initial
run will have captured those settings.
-R site
In order to provide the greater picture,
pegasus-kickstart can reflect the site handle (resource identifier) into its
output.
There is no default. If no value is given, the attribute will not be
generated.
-L lbl, -T iso
These optional arguments denote the workflow
label (from DAX) and the workflow’s last modification time (from DAX).
The label lbl can be any sensible string of up to 32 characters, but
should use C identifier characters. The timestamp iso must be an ISO
8601 compliant time-stamp.
-S l=p
If stat information on any file is required
before any jobs were started, logical to physical file mappings to stat
can be passed using the -S option. The LFN and PFN are concatenated by
an equals (=) sign. The LFN is optional: If no equals sign is found, the
argument is taken as sole PFN specification without LFN.
This option may be specified multiple times. To reduce and overcome command line
length limits, if the argument is prefixed with an at (@) sign, the argument
is taken to be a textual file of LFN to PFN mappings. The optionality
mentioned above applies. Each line inside the file argument is the name of a
file to stat. Comments (#) and empty lines are permitted.
Each PFN will incur a statcall record (element) with attribute id
set to value initial. The optional lfn attribute is set to the
LFN stat’ed. The filename is part of the statinfo record inside.
There is no default.
-s fn
If stat information on any file is required
after all jobs have finished, logical to physical file mappings to stat
can be passed using the -s option. The LFN and PFN are concatenated by
an equals (=) sign. The LFN is optional: If no equals sign is found, the
argument is taken as sole PFN specification without LFN.
This option may be specified multiple times. To reduce and overcome commandline
length limits, if the argument is prefixed with an at (@) sign, the argument
is taken to be a textual file of LFN to PFN mappings. The optionality
mentioned above applies. Each line inside the file argument is the name of a
file to stat. Comments (#) and empty lines are permitted.
Each PFN will incur a statcall record (element) with attribute id
set to value final. The optional lfn attribute is set to the LFN
stat’ed. The filename is part of the statinfo record inside.
There is no default.
-i fn
This option allows pegasus-kickstart to
re-connect the stdin of the application that it starts. Use a single hyphen to
share stdin with the one provided to pegasus-kickstart.
The default is to connect stdin to /dev/null.
-o fn
This option allows pegasus-kickstart to
re-connect the stdout of the application that it starts. The mode is
used whenever an application produces meaningful results on its stdout
that need to be tracked by Pegasus. The real stdout of Globus jobs is
staged via GASS (GT2) or RFT (GT4). The real stdout is used to
propagate the invocation record back to the submit site. Use the single hyphen
to share the application’s stdout with the one that is provided
to pegasus-kickstart. In that case, the output from
pegasus-kickstart will interleave with application output. For this
reason, such a mode is not recommended.
In order to provide an un-captured stdout as part of the results, it is
the default to connect the stdout of the application to a temporary
file. The content of this temporary file will be transferred as payload data
in the pegasus-kickstart results. The content size is subject to
payload limits, see the -B option. If the content grows large, only an
initial portion will become part of the payload. If the temporary file grows
too large, it may flood the worker node’s temporary space. The temporary
file will be deleted after pegasus-kickstart finishes.
If the filename is prefixed with an exclamation point, the file will be opened
in append mode instead of overwrite mode. Note that you may need to escape the
exclamation point from the shell.
The default is to connect stdout to a temporary file.
-e fn
This option allows pegasus-kickstart to
re-connect the stderr of the application that it starts. This option is
used whenever an application produces meaningful results on stderr that
needs tracking by Pegasus. The real stderr of Globus jobs is staged via
GASS (GT2) or RFT (GT4). It is used to propagate abnormal behavior from both,
pegasus-kickstart and the application that it starts, though its main
use is to propagate application dependent data and heartbeats. Use a single
hyphen to share stderr with the stderr that is provided to
pegasus-kickstart. This is the backward compatible behavior.
In order to provide an un-captured stderr as part of the results, by
default the stderr of the application will be connected to a temporary
file. Its content is transferred as payload data in the
pegasus-kickstart results. If too large, only the an initial portion
will become part of the payload. If the temporary file grows too large, it may
flood the worker node’s temporary space. The temporary file will be
deleted after pegasus-kickstart finishes.
If the filename is prefixed with an exclamation point, the file will be opened
in append mode instead of overwrite mode. Note that you may need to escape the
exclamation point from the shell.
The default is to connect stderr to a temporary file.
-l logfn
allows to append the performance data to the
specified file. Thus, multiple XML documents may end up in the same file,
including their XML preamble. stdout is normally used to stream back
the results. Usually, this is a GASS-staged stream. Use a single hyphen to
generate the output on the stdout that was provided to
pegasus-kickstart, the default behavior.
Default is to append the invocation record onto the provided
stdout.
-w dir
permits the explicit setting of a new working
directory once pegasus-kickstart is started. This is useful in a remote
scheduling environment, when the chosen working directory is not visible on
the job submitting host. If the directory does not exist,
pegasus-kickstart will fail. This option is mutually exclusive with the
-W dir option.
Default is to use the working directory that the application was started in.
This is usually set up by a remote scheduling environment.
-W dir
permits the explicit creation and setting of a
new working directory once pegasus-kickstart is started. This is useful in a
remote scheduling environment, when the chosen working directory is not
visible on the job submitting host. If the directory does not exist,
pegasus-kickstart will attempt to create it, and then change into it.
Both, creation and directory change may still fail. This option is mutually
exclusive with the -w dir option.
Default is to use the working directory that the application was started in.
This is usually set up by a remote scheduling environment.
-X
make an application executable, no matter
what. It is a work-around code for a weakness of globus-url-copy which
does not copy the permissions of the source to the destination. Thus, if an
executable is staged-in using GridFTP, it will have the wrong permissions.
Specifying the -X flag will attempt to change the mode to include the
necessary x (and r) bits to make the application executable.
Default is not to change the mode of the application. Note that this feature can
be misused by hackers, as it is attempted to call chmod on whatever path is
specified.
-B sz
varies the size of the debug output data
section. If the file descriptors stdout and stderr remain
untracked, pegasus-kickstart tracks that output in temporary files. The
first few pages from this output is copied into a data section in the output.
In order to resize the length of the output within reasonable boundaries, this
option permits a changes. Data beyond the size will not be copied, i.e. is
truncated.
Warning: This is not a cheap way to obtain the stdio file handle data. Please
use tracked files for that. Due to output buffer pre-allocation, using
arbitrary large arguments may result in failures of pegasus-kickstart
itself to allocate the necessary memory.
The default maximum size of the data section is 262144 byte.
-F
This flag will issue an explicit
fsync() call on kickstart’s own stdout file. Typically you
won’t need this flag. Albeit, certain shared file system situations may
improve when adding a flush after the written invocation record.
The default is to just use kickstart’s NFS alleviation strategy by locking
and unlocking stdout.
-I fn
In this mode, the application name and any
arguments to the application are specified inside of file fn. The file
contains one argument per line. Escaping from Globus, Condor and shell meta
characters is not required. This mode permits to use the maximum possible
command line length of the underlying operating system, e.g. 128k for Linux.
Using the -I mode stops any further command line processing of
pegasus-kickstart command lines.
Default is to use the app flags mode, where the application is specified
explicitly on the command-line.
app
The path to the application has to be
completely specified. The application is a mandatory option.
appflags
Application may or may not have additional
flags.
RETURN VALUE¶
pegasus-kickstart will return the return value of the main job. In addition, the error code 127 signals that the call to exec failed, and 126 that reconnecting the stdio failed. A job failing with the same exit codes is indistinguishable from pegasus-kickstart failures.SEE ALSO¶
pegasus-plan(1), condor_submit_dag(1), condor_submit(1), getrusage(3c).SUBJOBS¶
Subjobs are a new feature and may have a few wrinkles left.VARIABLE REWRITING¶
Variable substitution is a new feature and may have a few wrinkles left.FEEDBACK CHANNEL¶
A long-running application may consider to stream back heart beats and other application-specific monitoring and progress data. For this reason, pegasus-kickstart provides a feedback channel. At start-up, a transient named pipe, also known as FIFO, is created. While waiting for started jobs to finish, pegasus-kickstart will attempt to read from the FIFO. By default, any information read will be encapsulated in XML tags, and written to stderr. Please note that in a Pegasus, Globus, Condor-G environment, stderr will be GASS streamed or staged to the submit host. At the submit host, an application specific monitor may unpack the data chunks and could for instance visually display them, or aggregate them with other data. Please note that pegasus-kickstart only provides a feedback channel. The content and interpretation is up to, and specific for the application.EXAMPLE¶
You can run the pegasus-kickstart executable locally to verify that it is functioning well. In the initial phase, the format of the performance data may be slightly adjusted.$ env GRIDSTART_PREJOB='/bin/usleep 250000' \\ GRIDSTART_POSTJOB='/bin/date -u' \\ pegasus-kickstart -l xx \\$PEGASUS_HOME/bin/keg -T1 -o- $ cat xx <?xml version="1.0" encoding="ISO-8859-1"?> ... </statcall> </invocation>
OUTPUT FORMAT¶
Refer to http://pegasus.isi.edu/wms/docs/schemas/iv-2.1/iv-2.1.html for an up-to-date description of elements and their attributes. Check with http://pegasus.isi.edu/documentation for invocation schemas with a higher version number.RESTRICTIONS¶
There is no version for the Condor standard universe. It is simply not possible within the constraints of Condor.FILES¶
/usr/share/pegasus/schema/iv-2.1.xsdis the suggested location of the latest XML
schema describing the data on the submit host.
ENVIRONMENT VARIABLES¶
GRIDSTART_TMPis the hightest priority to look for a
temporary directory, if specified. This rather special variable was introduced
to overcome some peculiarities with the FNAL cluster.
TMP
is the next hightest priority to look for a
temporary directory, if specified.
TEMP
is the next priority for an environment
variable denoting a temporary files directory.
TMPDIR
is next in the checklist. If none of these are
found, either the stdio definition P_tmpdir is taken, or the
fixed string /tmp.
GRIDSTART_SETUP
contains a string that starts a job to be
executed unconditionally before any other jobs, see above for a detailed
description.
GRIDSTART_PREJOB
contains a string that starts a job to be
executed before the main job, see above for a detailed description.
GRIDSTART_POSTJOB
contains a string that starts a job to be
executed conditionally after the main job, see above for a detailed
description.
GRIDSTART_CLEANUP
contains a string that starts a job to be
executed unconditionally after any of the previous jobs, see above for a
detailed description.
GRIDSTART_CHANNEL
is the name of a FIFO for an
application-specific feedback-channel, see above for a detailed
description.
HISTORY¶
As you may have noticed, pegasus-kickstart had the name kickstart in previous incantations. We are slowly moving to the new name to avoid clashes in a larger OS installation setting. However, there is no pertinent need to change the internal name, too, as no name clashes are expected.AUTHORS¶
Michael Milligan <mbmillig at uchicago dot edu>05/24/2012 |