Name¶
condor_submit_dag Manage - and queue jobs within a specified DAG for execution
on remote machines
Synopsis¶
condor_submit_dag [-help | -version]
condor_submit_dag[-no_submit] [-verbose] [-force] [-maxidle NumberOfJobs]
[-maxjobs NumberOfJobs] [-dagman DagmanExecutable] [-maxpre
NumberOfPREscripts] [-maxpost NumberOfPOSTscripts] [-notification value]
[-noeventchecks] [-allowlogerror] [-r schedd_name] [-debug level] [-usedagdir]
[-outfile_dir directory] [-config ConfigFileName] [-insert_sub_file FileName]
[-append Command] [-autorescue 0|1] [-dorescuefrom number]
[-allowversionmismatch] [-no_recurse] [-do_recurse] [-update_submit]
[-import_env] [-DumpRescue] [-valgrind] [-DontAlwaysRunPost]
DAGInputFile1[DAGInputFile2 ... DAGInputFileN ] [-schedd-daemon-ad-file
FileName] [-schedd-address-file FileName]
Description¶
condor_submit_dagis the program for submitting a DAG (directed acyclic graph) of
jobs for execution under Condor. The program enforces the job dependencies
defined in one or more DAGInputFiles. Each DAGInputFilecontains commands to
direct the submission of jobs implied by the nodes of a DAG to Condor. See the
Condor User Manual, section 2.10for a complete description.
Options¶
-help
-
- Display usage information.
-
-version
-
- Display version information.
-
-no_submit
-
- Produce the Condor submit description file for DAGMan, but
do not submit DAGMan as a Condor job.
-
-verbose
-
- Cause condor_submit_dag to give verbose error
messages.
-
-force
-
- Require condor_submit_dag to overwrite the files that it
produces, if the files already exist. Note that dagman.out will be
appended to, not overwritten. If new-style rescue DAG mode is in effect,
and any new-style rescue DAGs exist, the -forceflag will cause them to be
renamed, and the original DAG will be run. If old-style rescue DAG mode is
in effect, any existing old-style rescue DAGs will be deleted, and the
original DAG will be run. Section 2.10.8details rescue DAGs.
-
-maxidle NumberOfJobs
-
- Sets the maximum number of idle jobs allowed before
condor_dagmanstops submitting more jobs. Once idle jobs start to run,
condor_dagmanwill resume submitting jobs. NumberOfJobsis a positive
integer. If the option is omitted, the number of idle jobs is unlimited.
Note that for this argument, each individual process within a cluster
counts as a job, which is inconsistent with -maxjobs .
-
-maxjobs NumberOfJobs
-
- Sets the maximum number of jobs within the DAG that will be
submitted to Condor at one time. NumberOfJobsis a positive integer. If the
option is omitted, the default number of jobs is unlimited. Note that for
this argument, each cluster counts as one job, no matter how many
individual processes are in the cluster.
-
-dagman DagmanExecutable
-
- Allows the specification of an alternate
condor_dagmanexecutable to be used instead of the one found in the user's
path. This must be a fully qualified path.
-
-maxpre NumberOfPREscripts
-
- Sets the maximum number of PRE scripts within the DAG that
may be running at one time. NumberOfPREScriptsis a positive integer. If
this option is omitted, the default number of PRE scripts is
unlimited.
-
-maxpost NumberOfPOSTscripts
-
- Sets the maximum number of POST scripts within the DAG that
may be running at one time. NumberOfPOSTScriptsis a positive integer. If
this option is omitted, the default number of POST scripts is
unlimited.
-
-notification value
-
- Sets the e-mail notification for DAGMan itself. This
information will be used within the Condor submit description file for
DAGMan. This file is produced by condor_submit_dag . See
notificationwithin the section of submit description file commands in the
condor_submitmanual page on page for specification of value.
-
-noeventchecks
-
- This argument is no longer used; it is now ignored. Its
functionality is now implemented by the DAGMAN_ALLOW_EVENTS configuration
macro (see section 3.3.25).
-
-allowlogerror
-
- This optional argument has condor_dagmantry to run the
specified DAG, even in the case of detected errors in the user log
specification. As of version 7.3.2, this argument has an effect only on
DAGs containing Stork job nodes.
-
-r schedd_name
-
- Submit condor_dagmanto a remote machine, specifically the
condor_schedddaemon on that machine. The condor_dagmanjob will not run on
the local condor_schedd(the submit machine), but on the specified one.
This is implemented using the -remoteoption to condor_submit. Note that
this option does not currently specify input files for condor_dagman, nor
the individual nodes to be taken along! It is assumed that any necessary
files will be present on the remote computer, possibly via a shared file
system between the local computer and the remote computer. It is also
necessary that the user has appropriate permissions to submit a job to the
remote machine; the permissions are the same as those required to use
condor_submit's -remoteoption. If other options are desired, including
transfer of other input files, consider using the -no_submitoption,
modifying the resulting submit file for specific needs, and then using
condor_submiton that.
-
-debug level
-
- Passes the the levelof debugging output desired to
condor_dagman. levelis an integer, with values of 0-7 inclusive, where 7
is the most verbose output. See the condor_dagmanmanual page on page for
detailed descriptions of these values. If not specified, no -debug value
is passed to condor_dagman.
-
-usedagdir
-
- This optional argument causes condor_dagmanto run each
specified DAG as if condor_submit_dag had been run in the directory
containing that DAG file. This option is most useful when running multiple
DAGs in a single condor_dagman. Note that the -usedagdirflag must not be
used when running an old-style rescue DAG (see section 2.10.8).
-
-outfile_dir directory
-
- Specifies the directory in which the .dagman.out file will
be written. The directorymay be specified relative to the current working
directory as condor_submit_dag is executed, or specified with an absolute
path. Without this option, the .dagman.out file is placed in the same
directory as the first DAG input file listed on the command line.
-
-config ConfigFileName
-
- Specifies a configuration file to be used for this DAGMan
run. Note that the options specified in the configuration file apply to
all DAGs if multiple DAGs are specified. Further note that it is a fatal
error if the configuration file specified by this option conflicts with a
configuration file specified in any of the DAG files, if they specify one.
For more information about how condor_dagmanconfiguration files work, see
section 2.10.7.
-
-insert_sub_file FileName
-
- Specifies a file to insert into the .condor.sub file
created by condor_submit_dag . The specified file must contain only legal
submit file commands. Only one file can be inserted. (If both the
DAGMAN_INSERT_SUB_FILE configuration variable and -insert_sub_fileare
specified, -insert_sub_fileoverrides DAGMAN_INSERT_SUB_FILE.) The
specified file is inserted into the .condor.sub file before the Queue
command and before any commands specified with the -appendoption.
-
-append Command
-
- Specifies a command to append to the .condor.sub file
created by condor_submit_dag . The specified command is appended to the
.condor.sub file immediately before the Queue command. Multiple commands
are specified by using the -appendoption multiple times. Each new command
is given in a separate -appendoption. Commands with spaces in them must be
enclosed in double quotes. Commands specified with the -appendoption are
appended to the .condor.sub file aftercommands inserted from a file
specified by the -insert_sub_fileoption or the DAGMAN_INSERT_SUB_FILE
configuration variable, so the -appendcommand(s) will override commands
from the inserted file if the commands conflict.
-
-autorescue 0|1
-
- Whether to automatically run the newest rescue DAG for the
given DAG file, if one exists (0 = false , 1 = true ).
-
-dorescuefrom number
-
- Forces condor_dagmanto run the specified rescue DAG number
for the given DAG. A value of 0 is the same as not specifying this option.
Specifying a non-existent rescue DAG is a fatal error.
-
-allowversionmismatch
-
- This optional argument causes condor_dagmanto allow a
version mismatch between condor_dagmanitself and the .condor.sub file
produced by condor_submit_dag (or, in other words, between
condor_submit_dag and condor_dagman). WARNING! This option should be used
only if absolutely necessary. Allowing version mismatches can cause subtle
problems when running DAGs. (Note that, starting with version 7.4.0,
condor_dagmanno longer requires an exact version match between itself and
the .condor.sub file. Instead, a "minimum compatible version" is
defined, and any .condor.sub file of that version or newer is
accepted.)
-
-no_recurse
-
- This optional argument causes condor_submit_dag to notrun
itself recursively on nested DAGs (this is now the default; this flag has
been kept mainly for backwards compatibility).
-
-do_recurse
-
- This optional argument causes condor_submit_dag to run
itself recursively on nested DAGs (the default is now that it does notrun
itself recursively; instead the .condor.sub files for nested DAGs are
generated "lazily" by condor_dagmanitself). (DAG nodes specified
with the SUBDAG EXTERNAL keyword or with submit file names ending in
.condor.sub are considered nested DAGs.) (See also the
DAGMAN_GENERATE_SUBDAG_SUBMITS configuration variable in section 3.3.25for
more information.)
-
-update_submit
-
- This optional argument causes an existing .condor.sub file
to not be treated as an error; rather, the .condor.sub file will be
overwritten, but the existing values of -maxjobs, -maxidle, -maxpre, and
-maxpostwill be preserved.
-
-import_env
-
- This optional argument causes condor_submit_dag to import
the current environment into the environmentcommand of the .condor.sub
file it generates.
-
-DumpRescue
-
- This optional argument tells condor_dagmanto immediately
dump a rescue DAG and then exit, as opposed to actually running the DAG.
This feature is mainly intended for testing. The Rescue DAG file is
produced whether or not there are parse errors reading the original DAG
input file. The name of the file differs if there was a parse error.
-
-valgrind
-
- This optional argument causes the submit description file
generated for the submission of condor_dagmanto be modified. The
executable becomes valgrindrun on condor_dagman, with a specific set of
arguments intended for testing condor_dagman. Note that this argument is
intended for testing purposes only. Using the -valgrindoption without the
necessary valgrindsoftware installed will cause the DAG to fail. If the
DAG does run, it will run much more slowly than usual.
-
-DontAlwaysRunPost
-
- This option causes the submit description file generated
for the submission of condor_dagmanto be modified. It causes the
-DontAlwaysRunPostoption to be in the arguments to condor_dagmanin the
submit description file, which causes condor_dagmanto use the return value
from a PRE script to determine whether or not a POST script will run. By
default, condor_dagmanruns the POST script regardless of the return value
of the PRE script. Versions of condor_dagmanprior to 7.7.2 did not ignore
the return value and would not run the POST script if the PRE script
failed.
-
-schedd-daemon-ad-file FileName
-
- Specifies a full path to a daemon ad file dropped by a
condor_schedd. Therefore this allows submission to a specific scheduler if
several are available without repeatedly querying the condor_collector.
The value for this argument defaults to the configuration attribute
SCHEDD_DAEMON_AD_FILE .
-
-schedd-address-file FileName
-
- Specifies a full path to an address file dropped by a
condor_schedd. Therefore this allows submission to a specific scheduler if
several are available without repeatedly querying the condor_collector.
The value for this argument defaults to the configuration attribute
SCHEDD_ADDRESS_FILE .
-
See Also¶
Condor User Manual
Exit Status¶
condor_submit_dagwill exit with a status value of 0 (zero) upon success, and it
will exit with the value 1 (one) upon failure.
Examples¶
To run a single DAG:
% condor_submit_dag diamond.dag
To run a DAG when it has already been run and the output files exist:
% condor_submit_dag -force diamond.dag
To run a DAG, limiting the number of idle node jobs in the DAG to a maximum of
five:
% condor_submit_dag -maxidle 5 diamond.dag
To run a DAG, limiting the number of concurrent PRE scripts to 10 and the number
of concurrent POST scripts to five:
% condor_submit_dag -maxpre 10 -maxpost 5 diamond.dag
To run two DAGs, each of which is set up to run in its own directory:
% condor_submit_dag -usedagdir dag1/diamond1.dag dag2/diamond2.dag
Author¶
Condor Team, University of Wisconsin-Madison
Copyright¶
Copyright (C) 1990-2012 Condor Team, Computer Sciences Department, University of
Wisconsin-Madison, Madison, WI. All Rights Reserved. Licensed under the Apache
License, Version 2.0.
See the Condor Version 7.8.2 Manualor
http://www.condorproject.org/licensefor
additional notices. condor-admin@cs.wisc.edu