.\" Automatically generated by Pod::Man 2.25 (Pod::Simple 3.16)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is turned on, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.ie \nF \{\
.    de IX
.    tm Index:\\$1\t\\n%\t"\\$2"
..
.    nr % 0
.    rr F
.\}
.el \{\
.    de IX
..
.\}
.\"
.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
.\" Fear.  Run.  Save yourself.  No user-serviceable parts.
.    \" fudge factors for nroff and troff
.if n \{\
.    ds #H 0
.    ds #V .8m
.    ds #F .3m
.    ds #[ \f1
.    ds #] \fP
.\}
.if t \{\
.    ds #H ((1u-(\\\\n(.fu%2u))*.13m)
.    ds #V .6m
.    ds #F 0
.    ds #[ \&
.    ds #] \&
.\}
.    \" simple accents for nroff and troff
.if n \{\
.    ds ' \&
.    ds ` \&
.    ds ^ \&
.    ds , \&
.    ds ~ ~
.    ds /
.\}
.if t \{\
.    ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
.    ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
.    ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
.    ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
.    ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
.    ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
.\}
.    \" troff and (daisy-wheel) nroff accents
.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
.ds ae a\h'-(\w'a'u*4/10)'e
.ds Ae A\h'-(\w'A'u*4/10)'E
.    \" corrections for vroff
.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
.    \" for low resolution devices (crt and lpr)
.if \n(.H>23 .if \n(.V>19 \
\{\
.    ds : e
.    ds 8 ss
.    ds o a
.    ds d- d\h'-1'\(ga
.    ds D- D\h'-1'\(hy
.    ds th \o'bp'
.    ds Th \o'LP'
.    ds ae ae
.    ds Ae AE
.\}
.rm #[ #] #H #V #F C
.\" ========================================================================
.\"
.IX Title "oarsub 1"
.TH oarsub 1 "2012-05-23" "oarsub" "OAR commands"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
oarsub \- OAR batch scheduler job submission command.
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
\&\fBoarsub\fR [\s-1OPTIONS\s0] <job executable>
.PP
\&\fBoarsub\fR [\s-1OPTIONS\s0] \-I
.PP
\&\fBoarsub\fR [\s-1OPTIONS\s0] \-C <\s-1JOB\s0 \s-1ID\s0>
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
One uses \fBoarsub\fR to submit a job to the \fB\s-1OAR\s0 batch scheduler\fR managing the resources of a \fB\s-1HPC\s0 Cluster\fR.
A job is defined by the description of a set of resources needed to execute a task and a script or executable to run.
A job may also be run interactively, and one may also use oarsub to connect to a previously submitted job.
.PP
The scheduler is in charge of providing a set of resources matching the oarsub command request. Once scheduled and then launched, a job consists of one process executed on the first node of the resources it was attibuted, with a set of environment variables that define the resources which are at the job disposal. That means that the job's executable is responsible for connecting those resources and dispatching the tasks.
.SH "OPTIONS"
.IX Header "OPTIONS"
.IP "\fB\-I, \-\-interactive\fR" 4
.IX Item "-I, --interactive"
Request an interactive job. Open a login shell on the first node of the reservation instead of running a script.
.IP "\fB\-C, \-\-connect\fR <\s-1JOB\s0 \s-1ID\s0>" 4
.IX Item "-C, --connect <JOB ID>"
Connect to a running job.
.IP "\fB\-l, \-\-resource\fR <\s-1LIST\s0>" 4
.IX Item "-l, --resource <LIST>"
Set the requested resources for the job. The different parameters are resource properties registered in \s-1OAR\s0 database, and `walltime' which specifies the duration before the job must be automatically terminated if still running. 
Walltime format is [hour:mn:sec|hour:mn|hour]. Ex: nodes=4/cpu=1,walltime=2:00:00
.Sp
You can specify multiple \-l options at the same line. This tells \s-1OAR\s0 that this is a moldable job so it can take different shapes. For example if you have an application that is very scalable:
.Sp
oarsub \-l cpu=2,walltime=20:00:00 \-l cpu=4,walltime=10:00:00 \-l cpu=8,walltime=5:00:00 ./script.sh
.Sp
\&\s-1OAR\s0 will schedule one of these three resource definitions (depending of the current load of the cluster).
.IP "\fB\-\-array\fR <\s-1NUMBER\s0>" 4
.IX Item "--array <NUMBER>"
Submit an array job containing \*(L"\s-1NUMBER\s0\*(R" subjobs. All the subjobs share the same array_id but each subjob is independent and has its own job_id. All the subjobs have the same characteristics (script, requirements) and can be identified by an environment variable \f(CW$OAR_ARRAY_INDEX\fR.
.Sp
Array jobs can neither be Interactive (\-I) nor a reservation (\-r).
.IP "\fB\-\-array\-param\-file\fR <\s-1FILE\s0>" 4
.IX Item "--array-param-file <FILE>"
Submit a parametric array job. Each non-empty line of \*(L"\s-1FILE\s0\*(R" defines the parameters for the submition of new subjob. All the subjobs have the same characteristics (script, requirements) and can be identified by an environment variable \f(CW$OAR_ARRAY_INDEX\fR.   '#' is the comment sign.
.Sp
Parametric array jobs can neither be Interactive (\-I) nor a reservation (\-r).
.IP "\fB\-S, \-\-scanscript\fR" 4
.IX Item "-S, --scanscript"
Batch mode only: ask oarsub to scan the given script for \s-1OAR\s0 directives (#OAR \-l ...)
.IP "\fB\-q, \-\-queue\fR <\s-1QUEUE\s0>" 4
.IX Item "-q, --queue <QUEUE>"
Set the the queue to submit the job to.
.ie n .IP "\fB\-p, \-\-property\fR ""<\s-1LIST\s0>""" 4
.el .IP "\fB\-p, \-\-property\fR ``<\s-1LIST\s0>''" 4
.IX Item "-p, --property <LIST>"
Add a list of constraints to properties for the job.
The format of a contraint is the one of a \s-1WHERE\s0 clause using the \s-1SQL\s0 syntax.
.IP "\fB\-r, \-\-reservation\fR  <\s-1DATE\s0>" 4
.IX Item "-r, --reservation  <DATE>"
Request that the job starts at a specified time. A job creation using this option is called a \fIreservation\fR and instead of a \fIsubmission\fR.
.IP "\fB\-\-checkpoint\fR <\s-1DELAY\s0>" 4
.IX Item "--checkpoint <DELAY>"
Enable the checkpointing mechanism for the job.
A signal will be sent \s-1DELAY\s0 seconds before the walltime to the first processus of the job (on the first node of the resources).
.IP "\fB\-\-signal\fR <#SIG>" 4
.IX Item "--signal <#SIG>"
Specify the signal to use when checkpointing. Use signal numbers (see kill \-l), default is \fI12\fR (\s-1SIGUSR2\s0).
.IP "\fB\-t, \-\-type\fR <\s-1TYPE\s0>" 4
.IX Item "-t, --type <TYPE>"
Specify a specific type (\fIbesteffort\fR, \fItimesharing\fR, \fIidempotent\fR, \fIcosystem\fR, \fIdeploy\fR, \fIcontainer\fR, \fIinner\fR, \fItoken:xxx=yy\fR)
.Sp
Note: a job with a type of \fBidempotent\fR will be automatically resubmitted if its exit code is 99.
.IP "\fB\-d, \-\-directory\fR <\s-1DIR\s0>" 4
.IX Item "-d, --directory <DIR>"
Specify the directory where to launch the command (default is current directory)
.IP "\fB\-\-project\fR <\s-1TXT\s0>" 4
.IX Item "--project <TXT>"
Specify a name of a project the job belongs to.
.IP "\fB\-n, \-\-name\fR <\s-1TXT\s0>" 4
.IX Item "-n, --name <TXT>"
Specify an arbitrary name for the job.
.IP "\fB\-a, \-\-anterior\fR <\s-1OAR\s0 \s-1JOB\s0 \s-1ID\s0>" 4
.IX Item "-a, --anterior <OAR JOB ID>"
Previously submitted job that this new job execution must depend on. The new job will only start upon the end of the previous one.
.IP "\fB\-\-notify\fR <\s-1TXT\s0>" 4
.IX Item "--notify <TXT>"
Specify a notification method (mail or command to execute).
Ex: 
 \-\-notify \*(L"mail:name@domain.com\*(R"
 \-\-notify \*(L"exec:/path/to/script args\*(R"
.Sp
args are job_id,job_name,TAG,comment
.Sp
\&\s-1TAG\s0 can be:
  \- \s-1RUNNING\s0 : when the job is launched
  \- \s-1END\s0 : when the job is finished normally
  \- \s-1ERROR\s0 : when the job is finished abnormally
  \- \s-1INFO\s0 : used when oardel is called on the job
  \- \s-1SUSPENDED\s0 : when the job is suspended
  \- \s-1RESUMING\s0 : when the job is resumed
.IP "\fB\-\-resubmit\fR <\s-1OAR\s0 \s-1JOB\s0 \s-1ID\s0>" 4
.IX Item "--resubmit <OAR JOB ID>"
Resubmit the given job as a new one.
.IP "\fB\-k, \-\-use\-job\-key\fR" 4
.IX Item "-k, --use-job-key"
Activate the job-key mechanism. A job-key will be generated allowing to connect the job from outside the set of resources managed by \s-1OAR\s0. The job-key mechanism may be activated by default in your \s-1OAR\s0 environment. In this case this option is useless.
.IP "\fB\-i, \-\-import\-job\-key\-from\-file\fR" 4
.IX Item "-i, --import-job-key-from-file"
Import the job-key to use from existing files (public and private key files) instead of generating a new one. One may also use the \s-1OAR_JOB_KEY_FILE\s0 environment variable.
.IP "\fB\-\-import\-job\-key\-inline\fR" 4
.IX Item "--import-job-key-inline"
Import the job-key to use inline instead of generating a new one.
.IP "\fB\-e, \-\-export\-job\-key\-to\-file\fR" 4
.IX Item "-e, --export-job-key-to-file"
Export the the job key to a file.
\&\fIWarning\fR: the file will be overwritten if it already exists.
(the \f(CW%jobid\fR% pattern is automatically replaced)
.IP "\fB\-O\fR <\s-1FILE\s0>" 4
.IX Item "-O <FILE>"
Specify the files that will store the standard output stream of the job. The \f(CW%jobid\fR% pattern is automatically replaced.
.IP "\fB\-E\fR <\s-1FILE\s0>" 4
.IX Item "-E <FILE>"
Specify the files that will store the standard error stream of the job. The \f(CW%jobid\fR% pattern is automatically replaced.
.IP "\fB\-\-hold\fR" 4
.IX Item "--hold"
Set the job state into Hold instead of Waiting, so that it is not scheduled (you must run \fIoarresume\fR to turn it into the Waiting state).
.IP "\fB\-D, \-\-dumper\fR" 4
.IX Item "-D, --dumper"
Print result in Perl Data::Dumper format.
.IP "\fB\-X, \-\-xml\fR" 4
.IX Item "-X, --xml"
Print result in \s-1XML\s0 format.
.IP "\fB\-Y, \-\-yaml\fR" 4
.IX Item "-Y, --yaml"
Print result in \s-1YAML\s0 format.
.IP "\fB\-J, \-\-json\fR" 4
.IX Item "-J, --json"
Print result in \s-1JSON\s0 format.
.IP "\fB\-h, \-\-help\fR" 4
.IX Item "-h, --help"
Print this help message.
.IP "\fB\-V, \-\-version\fR" 4
.IX Item "-V, --version"
Print the version of \s-1OAR\s0.
.SH "ENVIRONMENT"
.IX Header "ENVIRONMENT"
.IP "\fB\s-1OAR_FILE_NODES\s0\fR aka \fB\s-1OAR_NODE_FILE\s0\fR aka \fB\s-1OAR_NODEFILE\s0\fR" 4
.IX Item "OAR_FILE_NODES aka OAR_NODE_FILE aka OAR_NODEFILE"
Pathname to the file containing the list of the nodes that are allocated to the job.
.IP "\fB\s-1OAR_JOB_NAME\s0\fR" 4
.IX Item "OAR_JOB_NAME"
Name of the job as given using the \fB\-n\fR option.
.IP "\fB\s-1OAR_JOB_ID\s0\fR aka \fB\s-1OAR_JOBID\s0\fR" 4
.IX Item "OAR_JOB_ID aka OAR_JOBID"
Id of the job. Each job get a unique job identifier. This identifier can be use to retrieve information about the job using \fBoarstat\fR, or to connect to a running job using \fBoarsub \-C\fR or \fBoarsh\fR for instance.
.IP "\fB\s-1OAR_ARRAY_ID\s0\fR aka \fB\s-1OAR_ARRAYID\s0\fR" 4
.IX Item "OAR_ARRAY_ID aka OAR_ARRAYID"
Array Id of the job. Each array job get an unique array identifier that is shared by all the subjobs of the array job. This identifier can be used to identify the different subjobs pertaining to a same array job. It can also be used to deal with all the subjobs of a given array at once (by means of the option \-\-array in the case of oarstat, oarhold, oarresume and oardel). By definition, single jobs are considered array jobs with only one subjob.
.IP "\fB\s-1OAR_JOB_INDEX\s0\fR aka \fB\s-1OAR_JOBINDEX\s0\fR" 4
.IX Item "OAR_JOB_INDEX aka OAR_JOBINDEX"
Array Index of the job. On an array job, each job get an unique an unique (on a given array) job index. This identifier can be used to differ jobs on the context of a given array, for instance to give a different behaviour to each of the subjobs. By definition, single jobs are considered array jobs with only one subjob, having \s-1OAR_JOB_INDEX\s0 = 0.
.IP "\fB\s-1OAR_JOB_WALLTIME\s0\fR resp. \fB\s-1OAR_JOB_WALLTIME_SECONDS\s0\fR" 4
.IX Item "OAR_JOB_WALLTIME resp. OAR_JOB_WALLTIME_SECONDS"
Walltime of the job in the hh:mm:ss format resp. in seconds.
.IP "\fB\s-1OAR_RESOURCE_PROPERTIES_FILE\s0\fR" 4
.IX Item "OAR_RESOURCE_PROPERTIES_FILE"
Pathname to the file containing the list of all resources attributes for the job, and their value
.IP "\fB\s-1OAR_PROJECT_NAME\s0\fR" 4
.IX Item "OAR_PROJECT_NAME"
Name of the project the job is part of, as given using the \fB\-\-project\fR option.
.IP "\fB\s-1OAR_STDOUT\s0\fR and \fB\s-1OAR_STDERR\s0\fR" 4
.IX Item "OAR_STDOUT and OAR_STDERR"
Pathname to the files storing the standard output and error of the job's executable when not running in interactive mode.
.IP "\fB\s-1OAR_WORKING_DIRECTORY\s0\fR aka \fB\s-1OAR_WORKDIR\s0\fR aka \fB\s-1OAR_O_WORKDIR\s0\fR" 4
.IX Item "OAR_WORKING_DIRECTORY aka OAR_WORKDIR aka OAR_O_WORKDIR"
Working directory for the job. The job executable will be executed in this directory on the first node allocated to the job.
.IP "\fB\s-1OAR_JOB_KEY_FILE\s0\fR" 4
.IX Item "OAR_JOB_KEY_FILE"
Key file to use for the submission (or for oarsh) if using a job key (\-k or \-\-use\-job\-key option). One may provide the job key to import using the \-i or \-\-import\-job\-key\-from\-file option as well.
.SH "SCRIPT"
.IX Header "SCRIPT"
Script can contain the description of the job. Lines with options must begin by the key \fB#OAR\fR. There are the same options as previous.
.SH "EXAMPLES"
.IX Header "EXAMPLES"
Job submission with arguments :
.IP "oarsub \-l /nodes=4 \-I" 4
.IX Item "oarsub -l /nodes=4 -I"
.PD 0
.ie n .IP "oarsub \-q default \-l /nodes=10/cpu=3,walltime=50:30:00 \-p ""switch = 'sw1'"" /home/users/toto/prog" 4
.el .IP "oarsub \-q default \-l /nodes=10/cpu=3,walltime=50:30:00 \-p ``switch = 'sw1''' /home/users/toto/prog" 4
.IX Item "oarsub -q default -l /nodes=10/cpu=3,walltime=50:30:00 -p switch = 'sw1' /home/users/toto/prog"
.ie n .IP "oarsub \-r ""2009\-04\-27 11:00:00"" \-l /nodes=12/cpu=2" 4
.el .IP "oarsub \-r ``2009\-04\-27 11:00:00'' \-l /nodes=12/cpu=2" 4
.IX Item "oarsub -r 2009-04-27 11:00:00 -l /nodes=12/cpu=2"
.IP "oarsub \-C 154" 4
.IX Item "oarsub -C 154"
.PD
.PP
Submit an array job with 10 identic subjobs:
.IP "oarsub \-l /nodes=4 /home/users/toto/prog \-\-array 10" 4
.IX Item "oarsub -l /nodes=4 /home/users/toto/prog --array 10"
.PP
Submit a parametric array job (file params.txt):
.IP "oarsub /home/users/toto/prog \-\-array\-param\-file /home/users/toto/params.txt" 4
.IX Item "oarsub /home/users/toto/prog --array-param-file /home/users/toto/params.txt"
.PD 0
.IP "Parameter File example (params.txt):" 4
.IX Item "Parameter File example (params.txt):"
.ie n .IP "# my param file =item #single param =item 100 =item #a subjob without parameters =item """" =item #a subjob with a string containing spaces as parameter =item ""arg1a arg1b arg1c"" ""arg2a arg2b""" 4
.el .IP "# my param file =item #single param =item 100 =item #a subjob without parameters =item ``'' =item #a subjob with a string containing spaces as parameter =item ``arg1a arg1b arg1c'' ``arg2a arg2b''" 4
.IX Item "# my param file =item #single param =item 100 =item #a subjob without parameters =item "" =item #a subjob with a string containing spaces as parameter =item arg1a arg1b arg1c arg2a arg2b"""
.PD
.PP
Script example (file /home/users/toto/script.sh):
.IP "#!/bin/bash" 4
.IX Item "#!/bin/bash"
.PD 0
.IP "#OAR \-l /nodes=4/cpu=1,walltime=3:15:00" 4
.IX Item "#OAR -l /nodes=4/cpu=1,walltime=3:15:00"
.IP "#OAR \-p switch = 'sw3' or switch = 'sw5'" 4
.IX Item "#OAR -p switch = 'sw3' or switch = 'sw5'"
.IP "#OAR \-t besteffort" 4
.IX Item "#OAR -t besteffort"
.IP "#OAR \-t type2" 4
.IX Item "#OAR -t type2"
.IP "#OAR \-k" 4
.IX Item "#OAR -k"
.IP "#OAR \-e /path/to/job/key" 4
.IX Item "#OAR -e /path/to/job/key"
.IP "#OAR \-\-stdout stdoutfile.log" 4
.IX Item "#OAR --stdout stdoutfile.log"
.IP "/home/users/toto/prog" 4
.IX Item "/home/users/toto/prog"
.PD
.PP
Submit the script :
.IP "oarsub \-S /home/users/toto/script.sh" 4
.IX Item "oarsub -S /home/users/toto/script.sh"
.SH "SEE ALSO"
.IX Header "SEE ALSO"
\&\fIoarsh\fR\|(1), \fIoardel\fR\|(1), \fIoarstat\fR\|(1), \fIoarnodes\fR\|(1), \fIoarhold\fR\|(1), \fIoarresume\fR\|(1)
.SH "COPYRIGHTS"
.IX Header "COPYRIGHTS"
.Vb 1
\& Copyright X 2008 Laboratoire d\*(AqInformatique de Grenoble (http://www.liglab.fr). This software is licensed under the GNU Library General Public License. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
.Ve