NAME¶
sge_submit.1 - Job submission commands for SGE
- qsub -
- submit a batch job to Sun Grid Engine.
- qsh -
- submit an interactive X-windows session to Sun Grid
Engine.
- qlogin -
- submit an interactive login session to Sun Grid
Engine.
- qrsh -
- submit an interactive rsh session to Sun Grid Engine.
- qalter -
- modify a pending or running batch job of Sun Grid
Engine.
- qresub -
- submit a copy of an existing Sun Grid Engine job.
SYNTAX¶
qsub [
options ] [
command | -- [
command_args ]]
qsh [
options ] [
-- xterm_args ]
qlogin [
options ]
qrsh [
options ] [
command [
command_args ]]
qalter [
options ]
wc_job_range_list [
-- [
command_args ]]
qalter [
options ]
-u user_list | -uall [
-- [
command_args ]]
qresub [
options ]
job_id_list
DESCRIPTION¶
Qsub submits batch jobs to the Sun Grid Engine queuing system. Sun Grid
Engine supports single- and multiple-node jobs.
Command can be a path
to a binary or a script (see
-b below) which contains the commands to
be run by the job using a shell (for example, or Arguments to the command are
given as
command_args to
qsub . If
command is handled as
a script then it is possible to embed flags in the script. If the first two
characters of a script line either match '#$' or are equal to the prefix
string defined with the
-C option described below, the line is parsed
for embedded command flags.
Qsh submits an interactive X-windows session to Sun Grid Engine. An is
brought up from the executing machine with the display directed either to the
X-server indicated by the DISPLAY environment variable or as specified with
the
-display qsh option. Interactive jobs are not spooled if no
resource is available to execute them. They are either dispatched to a
suitable machine for execution immediately or the user submitting the job is
notified by
qsh that appropriate resources to execute the job are not
available.
xterm_args are passed to the executable. Note, however, that
the
-e and
-ls xterm options do not work with
qsh .
Qlogin is similar to
qsh in that it submits an interactive job to
the queuing system. It does not open an window on the X display, but uses the
current terminal for user I/O. Usually,
qlogin establishes a connection
with the remote host, using standard client- and server-side commands. These
commands can be configured with the
qlogin_daemon (server-side, Sun
Grid Engine
telnetd if not set, otherwise something like
/usr/sbin/in.telnetd) and
qlogin_command (client-side, Sun Grid Engine
telnet if not set, otherwise something like /usr/bin/telnet) parameters
in the global and local configuration settings of The client side command is
automatically parameterized with the remote host name and port number to which
to connect, resulting in an invocation like
- /usr/bin/telnet my_exec_host 2442
for example.
Qlogin is invoked exactly like
qsh and its jobs can
only run on INTERACTIVE queues.
Qlogin jobs can only be used if the is
running under the root account.
Qrsh is similar to
qlogin in that it submits an interactive job to
the queuing system. It uses the current terminal for user I/O. Usually,
qrsh establishes a connection with the remote host. If no command is
given to
qrsh, an session is established. The server-side commands used
can be configured with the
rsh_daemon and
rlogin_daemon
parameters in the global and local configuration settings of An Sun Grid
Engine
rshd or
rlogind is used if the parameters are not set. If
the parameters are set, they should be set to something like /usr/sbin/in.rshd
or /usr/sbin/in.rlogind. On the client-side, the
rsh_command and
rlogin_command parameters can be set in the global and local
configuration settings of If they are not set, special Sun Grid Engine and
binaries delivered with Sun Grid Engine are used. Use the cluster
configuration parameters to integrate mechanisms like
ssh or the and
facilities supplied with the operating system.
Qrsh jobs can only run in INTERACTIVE queues unless the option
-now
no is used (see below). They can also only be run, if the is running under
the root account.
Qrsh provides an additional useful feature for integrating with
interactive tools providing a specific command shell. If the environment
variable
QRSH_WRAPPER is set when
qrsh is invoked, the command
interpreter pointed to by
QRSH_WRAPPER will be executed to run
qrsh commands instead of the users login shell or any shell specified
in the
qrsh command-line. The options
-cwd,
-v,
-V, and
-display only apply to batch jobs.
Qalter can be used to change the attributes of pending jobs. For array
jobs with a mix of running and pending tasks (see the
-t option below),
modification with
qalter only affects the pending tasks.
Qalter
can change most of the characteristics of a job (see the corresponding
statements in the OPTIONS section below), including those which were defined
as embedded flags in the script file (see above). Some submit options, such as
the job script, cannot be changed with I. qalter.
Qresub allows the user to create jobs as copies of existing pending or
running jobs. The copied jobs will have exactly the same attributes as the
ones from which they were copied, except with a new job ID and with a cleared
hold state. The only modification to the copied jobs supported by
qresub is assignment of a new hold state with the
-h option.
This option can be used to first copy a job and then change its attributes via
qalter.
Only a manager can use
qresub on jobs submitted by another user. Regular
users can only use
qresub on their own jobs.
For
qsub,
qsh,
qrsh, and
qlogin the administrator
and the user may define default request files (see which can contain any of
the options described below. If an option in a default request file is
understood by
qsub and
qlogin but not by
qsh the option
is silently ignored if
qsh is invoked. Thus you can maintain shared
default request files for both
qsub and
qsh.
A cluster wide default request file may be placed under
$SGE_ROOT/$SGE_CELL/common/sge_request. User private default request files are
processed under the locations $HOME/.sge_request and $cwd/.sge_request. The
working directory local default request file has the highest precedence, then
the home directory located file and then the cluster global file. The option
arguments, the embedded script flags and the options in the default request
files are processed in the following order:
left to right in the script line,
left to right in the default request files,
from top to bottom of the script file ( qsub only),
from top to bottom of default request files,
from left to right of the command line.
In other words, the command line can be used to override the embedded flags and
the default request settings. The embedded flags, however, will override the
default settings.
Note, that the
-clear option can be used to discard any previous
settings at any time in a default request file, in the embedded script flags,
or in a command-line option. It is, however, not available with
qalter.
The options described below can be requested either hard or soft. By default,
all requests are considered hard until the
-soft option (see below) is
encountered. The hard/soft status remains in effect until its counterpart is
encountered again. If all the hard requests for a job cannot be met, the job
will not be scheduled. Jobs which cannot be run at the present time remain
spooled.
OPTIONS¶
- -@ optionfile
- Forces qsub, qrsh, qsh, or
qlogin to use the options contained in optionfile. The
indicated file may contain all valid options. Comment lines must start
with a "#" sign.
- -a date_time
- Available for qsub and qalter only.
Defines or redefines the time and date at which a job is eligible for
execution. Date_time conforms to [[CC]]YY]MMDDhhmm[.SS], for the
details, please see Date_time in:
If this option is used with qsub or if a corresponding value is
specified in qmon then a parameter named a and the value in
the format CCYYMMDDhhmm.SS will be passed to the defined JSV instances
(see -jsv option below or find more information concerning JSV
in
- -ac variable[=value],...
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Adds the given name/value pair(s) to the job's context. Value may be
omitted. Sun Grid Engine appends the given argument to the list of context
variables for the job. Multiple -ac, -dc, and -sc
options may be given. The order is important here.
The outcome of the evaluation of all -ac, -dc, and -sc
options or corresponding values in qmon is passed to defined JSV
instances as parameter with the name aac. (see -jsv option
below or find more information concerning JSV in QALTER allows
changing this option even while the job executes.
- -ar ar_id
- Available for qsub, qalter, qrsh,
qsh, or qlogin only.
Assigns the submitted job to be a part of an existing Advance Reservation.
The complete list of existing Advance Reservations can be obtained using
the command.
Note that the -ar option adds implicitly the -w e option if
not otherwise requested.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job however.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name aar. (see -jsv option below or find more information
concerning JSV in
- -A account_string
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Identifies the account to which the resource consumption of the job should
be charged. The account_string should conform to the name
definition in M sge_types 1 . In the absence of this parameter Sun Grid
Engine will place the default account string "sge" in the
accounting record of the job.
Qalter allows changing this option even while the job executes.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name A. (see -jsv option below or find more information
concerning JSV in
- -binding [ binding_instance ] binding_strategy
- A job can request a specific processor core binding
(processor affinity) with this parameter. This request is neither a hard
nor a soft request, it is a hint for the execution host to do this if
possible. Please note that the requested binding strategy is not used for
resource selection within Sun Grid Engine. As a result an execution host
might be selected where Sun Grid Engine does not even know the hardware
topology and therefore is not able to apply the requested binding.
To enforce Sun Grid Engine to select hardware on which the binding can be
applied please use the -l switch in combination with the complex
attribute m_topology.
binding_instance is an optional parameter. It might either be
env, pe or set depending on which instance should
accomplish the job to core binding. If the value for
binding_instance is not specified then set will be used.
env means that the environment variable SGE_BINDING will be
exported to the job environment of the job. This variable contains the
selected operating system internal processor numbers. They might be more
than selected cores in presence of SMT or CMT because each core could be
represented by multiple processor identifiers. The processor numbers are
space separated.
pe means that the information about the selected cores appears in
the fourth column of the pe_hostfile. Here the logical core and
socket numbers are printed (they start at 0 and have no holes) in colon
separated pairs (i.e. 0,0:1,0 which means core 0 on socket 0 and core 0 on
socket 1). For more information about the $pe_hostfile check
set (default if nothing else is specified). The binding strategy is
applied by Sun Grid Engine. How this is achieved depends on the underlying
hardware architecture of the execution host where the submitted job will
be started.
On Solaris 10 hosts a processor set will be created where the job can
exclusively run in. Because of operating system limitations at least one
core must remain unbound. This resource could of course used by an unbound
job.
On Linux hosts a processor affinity mask will be set to restrict the job to
run exclusively on the selected cores. The operating system allows other
unbound processes to use these cores. Please note that on Linux the
binding requires a Linux kernel version of 2.6.16 or greater. It might be
even possible to use a kernel with lower version number but in that case
additional kernel patches have to be applied. The loadcheck tool in
the utilbin directory can be used to check if the hosts capabilities. You
can also use the -sep in combination with -cb of command to
identify if Sun Grid Engine is able to recognize the hardware topology.
Possible values for binding_strategy are as follows:
linear:<amount>[:<socket>,<core>]
striding:<amount>:<n>[:<socket>,<core>]
explicit:[<socket>,<core>;...]<socket>,<core>
For the binding strategy linear and striding there is an optional socket and
core pair attached. These denotes the mandatory starting point for the
first core to bind on.
linear means that Sun Grid Engine tries to bind the job on
amount successive cores. If socket and core is
omitted then Sun Grid Engine first allocates successive cores on the first
empty socket found. Empty means that there are no jobs bound to the socket
by Sun Grid Engine. If this is not possible or is not sufficient Sun Grid
Engine tries to find (further) cores on the socket with the most unbound
cores and so on. If the amount of allocated cores is lower than requested
cores, no binding is done for the job. If socket and core is
specified then Sun Grid Engine tries to find amount of empty cores
beginning with this starting point. If this is not possible then binding
is not done.
striding means that Sun Grid Engine tries to find cores with a
certain offset. It will select amount of empty cores with a offset
of n -1 cores in between. Start point for the search algorithm is
socket 0 core 0. As soon as amount cores are found they will be
used to do the job binding. If there are not enough empty cores or if
correct offset cannot be achieved then there will be no binding done.
explicit binds the specified sockets and cores that are mentioned in
the provided socket/core list. Each socket/core pair has to be specified
only once. If a socket/core pair is already in use by a different job the
whole binding request will be ignored.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified then these
values will be passed to defined JSV instances as parameters with the
names binding_strategy, binding_type, binding_amount,
binding_step, binding_socket, binding_core,
binding_exp_n, binding_exp_socket<id>,
binding_exp_core<id>.
Please note that the length of the socket/core value list of the explicit
binding is reported as binding_exp_n. <id> will be
replaced by the position of the socket/core pair within the explicit list
(0 <= id < binding_exp_n). The first socket/core pair
of the explicit binding will be reported with the parameter names
binding_exp_socket0 and binding_exp_core0.
Values that do not apply for the specified binding will not be reported to
JSV. E.g. binding_step will only be reported for the striding
binding and all binding_exp_* values will passed to JSV if explicit
binding was specified. (see -jsv option below or find more information
concerning JSV in jsv(1))
- -b y[es]|n[o]
- Available for qsub, qrsh only. Qalter
does not allow changing this option. This option cannot be embedded in the
script file itself.
Gives the user the possibility to indicate explicitly whether command
should be treated as binary or script. If the value of -b is 'y',
then command may be a binary or script. The command might
not be accessible from the submission host. Nothing except the path of the
command will be transferred from the submission host to the
execution host. Path aliasing will be applied to the path of
command before command will be executed.
If the value of -b is 'n' then command needs to be a script
and it will be handled as script. The script file has to be accessible by
the submission host. It will be transferred to the execution host.
qsub/ qrsh will search directive prefixes within script.
qsub will implicitly use -b n whereas qrsh will apply
the -b y option if nothing else is specified.
The value specified with this option or the corresponding value specified in
qmon will only be passed to defined JSV instances if the value is
yes. The name of the parameter will be b. The value will be
y also when then long form yes was specified during
submission. (see -jsv option below or find more information
concerning JSV in
Please note that submission of command as script (-b n) can
have a significant performance impact, especially for short running jobs
and big job scripts. Script submission adds a number of operations to the
submission process: The job script needs to be
- parsed at client side (for special comments)
- transferred from submit client to qmaster
- spooled in qmaster
- transferred to execd at job execution
- spooled in execd
- removed from spooling both in execd and qmaster once the job is done
If job scripts are available on the execution nodes, e.g. via NFS, binary
submission can be the better choice.
- -c occasion_specifier
- Available for qsub and qalter only.
Defines or redefines whether the job should be checkpointed, and if so,
under what circumstances. The specification of the checkpointing occasions
with this option overwrites the definitions of the when parameter
in the checkpointing environment (see referenced by the qsub -ckpt
switch. Possible values for occasion_specifier are
n no checkpoint is performed.
s checkpoint when batch server is shut down.
m checkpoint at minimum CPU interval.
x checkpoint when job gets suspended.
<interval> checkpoint in the specified time interval.
The minimum CPU interval is defined in the queue configuration (see for
details). <interval> has to be specified in the format hh:mm:ss. The
maximum of <interval> and the queue's minimum CPU interval is used
if <interval> is specified. This is done to ensure that a machine is
not overloaded by checkpoints being generated too frequently.
The value specified with this option or the corresponding value specified in
qmon will be passed to defined JSV instances. The <interval>
will be available as parameter with the name c_interval. The
character sequence specified will be available as parameter with the name
c_occasion. Please note that if you change c_occasion via
JSV then the last setting of c_interval will be overwritten and
vice versa. (see -jsv option below or find more information
concerning JSV in
- -ckpt ckpt_name
- Available for qsub and qalter only.
Selects the checkpointing environment (see to be used for checkpointing the
job. Also declares the job to be a checkpointing job.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name ckpt. (see -jsv option below or find more information
concerning JSV in
- -clear
- Available for qsub, qsh, qrsh, and
qlogin only.
Causes all elements of the job to be reset to the initial default status
prior to applying any modifications (if any) appearing in this specific
command.
- -cwd
- Available for qsub, qsh, qrsh and
qalter only.
Execute the job from the current working directory. This switch will
activate Sun Grid Engine's path aliasing facility, if the corresponding
configuration files are present (see
In the case of qalter, the previous definition of the current working
directory will be overwritten if qalter is executed from a
different directory than the preceding qsub or qalter.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name cwd. The value of this parameter will be the absolute path to
the current working directory. JSV scripts can remove the path from jobs
during the verification process by setting the value of this parameter to
an empty string. As a result the job behaves as if -cwd was not
specified during job submission. (see -jsv option below or find
more information concerning JSV in
- -C prefix_string
- Available for qsub and qrsh with script
submission ( -b n).
Prefix_string defines the prefix that declares a directive in the
job's command. The prefix is not a job attribute, but affects the behavior
of qsub and qrsh. If prefix is a null string, the
command will not be scanned for embedded directives.
The directive prefix consists of two ASCII characters which, when appearing
in the first two bytes of a script line, indicate that what follows is an
Sun Grid Engine command. The default is "#$".
The user should be aware that changing the first delimiting character can
produce unforeseen side effects. If the script file contains anything
other than a "#" character in the first byte position of the
line, the shell processor for the job will reject the line and may exit
the job prematurely.
If the -C option is present in the script file, it is ignored.
- -dc variable,...
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Removes the given variable(s) from the job's context. Multiple -ac,
-dc, and -sc options may be given. The order is important.
Qalter allows changing this option even while the job executes.
The outcome of the evaluation of all -ac, -dc, and -sc
options or corresponding values in qmon is passed to defined JSV
instances as parameter with the name aac. (see -jsv option
below or find more information concerning JSV in
- -display display_specifier
- Available for qsh and qrsh.
Directs to use display_specifier in order to contact the X server.
The display_specifier has to contain the hostname part of the
display name (e.g. myhost:1). Local display names (e.g. :0) cannot be used
in grid environments. Values set with the -display option overwrite
settings from the submission environment and from -v command line
options.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name display. This value will also be available in the job
environment which might optionally be passed to JSV scripts. The variable
name will be DISPLAY. (see -jsv option below or find more
information concerning JSV in
- -dl date_time
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Specifies the deadline initiation time in [[CC]YY]MMDDhhmm[.SS] format (see
-a option above). The deadline initiation time is the time at which
a deadline job has to reach top priority to be able to complete within a
given deadline. Before the deadline initiation time the priority of a
deadline job will be raised steadily until it reaches the maximum as
configured by the Sun Grid Engine administrator.
This option is applicable only for users allowed to submit deadline jobs.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name dl. The format for the date_time value is CCYYMMDDhhmm.SS (see
-jsv option below or find more information concerning JSV in
- -e [[hostname]:]path,...
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Defines or redefines the path used for the standard error stream of the job.
For qsh, qrsh and qlogin only the standard error
stream of prolog and epilog is redirected. If the path constitutes
an absolute path name, the error-path attribute of the job is set to
path, including the hostname. If the path name is relative,
Sun Grid Engine expands path either with the current working
directory path (if the -cwd switch (see above) is also specified)
or with the home directory path. If hostname is present, the
standard error stream will be placed in the corresponding location only if
the job runs on the specified host. If the path contains a ":"
without a hostname, a leading ":" has to be specified.
By default the file name for interactive jobs is /dev/null. For batch
jobs the default file name has the form job_name.ejob_id and
job_name.ejob_id.task_id for array job tasks (see
-t option below).
If path is a directory, the standard error stream of the job will be
put in this directory under the default file name. If the pathname
contains certain pseudo environment variables, their value will be
expanded at runtime of the job and will be used to constitute the standard
error stream path name. The following pseudo environment variables are
supported currently:
$HOME home directory on execution machine
$USER user ID of job owner
$JOB_ID current job ID
$JOB_NAME current job name (see -N option)
$HOSTNAME name of the execution host
$TASK_ID array job task index number
Alternatively to $HOME the tilde sign "~" can be used as common in
or Note, that the "~" sign also works in combination with user
names, so that "~<user>" expands to the home directory of
<user>. Using another user ID than that of the job owner requires
corresponding permissions, of course.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name e. (see -jsv option below or find more information
concerning JSV in
- -hard
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Signifies that all -q and -l resource requirements following
in the command line will be hard requirements and must be satisfied in
full before a job can be scheduled.
As Sun Grid Engine scans the command line and script file for Sun Grid
Engine options and parameters it builds a list of resources required by a
job. All such resource requests are considered as absolutely essential for
the job to commence. If the -soft option (see below) is encountered
during the scan then all following resources are designated as "soft
requirements" for execution, or "nice-to-have, but not
essential". If the -hard flag is encountered at a later stage
of the scan, all resource requests following it once again become
"essential". The -hard and -soft options in effect
act as "toggles" during the scan.
If this option or a corresponding value in qmon is specified then the
corresponding -q and -l resource requirements will be passed
to defined JSV instances as parameter with the names q_hard and
l_hard. Find for information in the sections describing -q
and -l. (see -jsv option below or find more information
concerning JSV in
- -h | -h {u|s|o|n|U|O|S}...
- Available for qsub (only -h), qrsh,
qalter and qresub (hold state is removed when not set
explicitly).
List of holds to place on a job, a task or some tasks of a job.
`u' denotes a user hold.
`s' denotes a system hold.
`o' denotes a operator hold.
`n' denotes no hold (requires manager privileges).
As long as any hold other than `n' is assigned to the job the job is not
eligible for execution. Holds can be released via qalter and In
case of qalter this is supported by the following additional option
specifiers for the -h switch:
`U' removes a user hold.
`S' removes a system hold.
`O' removes a operator hold.
Sun Grid Engine managers can assign and remove all hold types, Sun Grid
Engine operators can assign and remove user and operator holds, and users
can only assign or remove user holds.
In the case of qsub only user holds can be placed on a job and thus
only the first form of the option with the -h switch alone is
allowed. As opposed to this, qalter requires the second form
described above.
An alternate means to assign hold is provided by the facility.
If the job is a array job (see the -t option below), all tasks
specified via -t are affected by the -h operation
simultaneously.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option is specified with qsub or during the submission of a
job in qmon then the parameter h with the value u
will be passed to the defined JSV instances indicating that the job will
be in user hold after the submission finishes. (see -jsv option
below or find more information concerning JSV in
- -help
- Prints a listing of all options.
- -hold_jid wc_job_list
- Available for qsub, qrsh, and qalter
only. See for wc_job_list definition.
Defines or redefines the job dependency list of the submitted job. A
reference by job name or pattern is only accepted if the referenced job is
owned by the same user as the referring job. The submitted job is not
eligible for execution unless all jobs referenced in the comma-separated
job id and/or job name list have completed. If any of the referenced jobs
exits with exit code 100, the submitted job will remain ineligible for
execution.
With the help of job names or regular pattern one can specify a job
dependency on multiple jobs satisfying the regular pattern or on all jobs
with the requested name. The name dependencies are resolved at submit time
and can only be changed via qalter. New jobs or name changes of other jobs
will not be taken into account.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name hold_jid. (see -jsv option below or find more
information concerning JSV in
- -hold_jid_ad wc_job_list
- Available for qsub, qrsh, and qalter
only. See for wc_job_list definition.
Defines or redefines the job array dependency list of the submitted job. A
reference by job name or pattern is only accepted if the referenced job is
owned by the same user as the referring job. Each sub-task of the
submitted job is not eligible for execution unless the corresponding
sub-tasks of all jobs referenced in the comma-separated job id and/or job
name list have completed. If any array task of the referenced jobs exits
with exit code 100, the dependent tasks of the submitted job will remain
ineligible for execution.
With the help of job names or regular pattern one can specify a job
dependency on multiple jobs satisfying the regular pattern or on all jobs
with the requested name. The name dependencies are resolved at submit time
and can only be changed via qalter. New jobs or name changes of other jobs
will not be taken into account.
If either the submitted job or any job in wc_job_list are not array jobs
with the same range of sub-tasks (see -t option below), the request
list will be rejected and the job create or modify operation will error.
qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name hold_jid_ad. (see -jsv option below or find more
information concerning JSV in
- -i [[hostname]:]file,...
- Available for qsub, and qalter only.
Defines or redefines the file used for the standard input stream of the job.
If the file constitutes an absolute filename, the input-path
attribute of the job is set to path, including the hostname.
If the path name is relative, Sun Grid Engine expands path either
with the current working directory path (if the -cwd switch (see
above) is also specified) or with the home directory path. If
hostname is present, the standard input stream will be placed in
the corresponding location only if the job runs on the specified host. If
the path contains a ":" without a hostname, a leading
":" has to be specified.
By default /dev/null is the input stream for the job.
It is possible to use certain pseudo variables, whose values will be
expanded at runtime of the job and will be used to express the standard
input stream as described in the -e option for the standard error
stream.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name i. (see -jsv option below or find more information
concerning JSV in
- -inherit
- Available only for qrsh and
qrsh allows the user to start a task in an already scheduled parallel
job. The option -inherit tells qrsh to read a job id from
the environment variable JOB_ID and start the specified command as a task
in this job. Please note that in this case, the hostname of the host where
the command will be executed must precede the command to execute; the
syntax changes to
qrsh -inherit [ other options ] hostname command [
command_args ]
Note also, that in combination with -inherit, most other command line
options will be ignored. Only the options -verbose, -v and
-V will be interpreted. As a replacement to option -cwd
please use -v PWD.
Usually a task should have the same environment (including the current
working directory) as the corresponding job, so specifying the option
-V should be suitable for most applications.
Note: If in your system the qmaster tcp port is not configured as a
service, but rather via the environment variable SGE_QMASTER_PORT, make
sure that this variable is set in the environment when calling qrsh
or qmake with the -inherit option. If you call qrsh
or qmake with the -inherit option from within a job script,
export SGE_QMASTER_PORT with the option "-v SGE_QMASTER_PORT"
either as a command argument or an embedded directive.
This parameter is not available in the JSV context. (see -jsv option
below or find more information concerning JSV in
- -j y[es]|n[o]
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Specifies whether or not the standard error stream of the job is merged into
the standard output stream.
If both the -j y and the -e options are present, Sun Grid
Engine sets but ignores the error-path attribute.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
The value specified with this option or the corresponding value specified in
qmon will only be passed to defined JSV instances if the value is
yes. The name of the parameter will be j. The value will be
y also when then long form yes was specified during
submission. (see -jsv option below or find more information
concerning JSV in
- -js job_share
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Defines or redefines the job share of the job relative to other jobs. Job
share is an unsigned integer value. The default job share value for jobs
is 0.
The job share influences the Share Tree Policy and the Functional Policy. It
has no effect on the Urgency and Override Policies (see and the Sun
Grid Engine Installation and Administration Guide for further
information on the resource management policies supported by Sun Grid
Engine).
In case of the Share Tree Policy, users can distribute the tickets to which
they are currently entitled among their jobs using different shares
assigned via -js. If all jobs have the same job share value, the
tickets are distributed evenly. Otherwise, jobs receive tickets relative
to the different job shares. Job shares are treated like an additional
level in the share tree in the latter case.
In connection with the Functional Policy, the job share can be used to
weight jobs within the functional job category. Tickets are distributed
relative to any uneven job share distribution treated as a virtual share
distribution level underneath the functional job category.
If both the Share Tree and the Functional Policy are active, the job shares
will have an effect in both policies, and the tickets independently
derived in each of them are added to the total number of tickets for each
job.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name js. (see -jsv option below or find more information
concerning JSV in
- -jsv jsv_url
- Available for qsub, qsh, qrsh and
qlogin only.
Defines a client JSV instance which will be executed to verify the job
specification before the job is sent to qmaster.
In contrast to other options this switch will not be overwritten if it is
also used in sge_request files. Instead all specified JSV instances will
be executed to verify the job to be submitted.
The JSV instance which is directly passed with the commandline of a client
is executed as first to verify the job specification. After that the JSV
instance which might have been defined in various sge_request files will
be triggered to check the job. Find more details in man page and
The syntax of the jsv_url is specified in
- -l resource=value,...
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Launch the job in a Sun Grid Engine queue meeting the given resource request
list. In case of qalter the previous definition is replaced by the
specified one.
describes how a list of available resources and their associated valid value
specifiers can be obtained.
There may be multiple -l switches in a single command. You may
request multiple -l options to be soft or hard both in the same
command line. In case of a serial job multiple -l switches refine
the definition for the sought queue.
Qalter allows changing the value of this option even while the job is
running, but only if the initial list of resources does not contain a
resource that is marked as consumable. However the modification will only
be effective after a restart or migration of the job.
If this option or a corresponding value in qmon is specified the
these hard and soft resource requirements will be passed to defined JSV
instances as parameter with the names l_hard and l_soft. If
regular expressions will be used for resource requests, then these
expressions will be passed as they are. Also shortcut names will not be
expanded. (see -jsv option above or find more information
concerning JSV in
- -m b|e|a|s|n,...
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Defines or redefines under which circumstances mail is to be sent to the job
owner or to the users defined with the -M option described below.
The option arguments have the following meaning:
`b' Mail is sent at the beginning of the job.
`e' Mail is sent at the end of the job.
`a' Mail is sent when the job is aborted or
rescheduled.
`s' Mail is sent when the job is suspended.
`n' No mail is sent.
Currently no mail is sent when a job is suspended.
Qalter allows changing the b, e, and a option arguments even while
the job executes. The modification of the b option argument will only be
in effect after a restart or migration of the job, however.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name m. (see -jsv option above or find more information
concerning JSV in
- -M user[@host],...
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Defines or redefines the list of users to which the server that executes the
job has to send mail, if the server sends mail about the job. Default is
the job owner at the originating host.
Qalter allows changing this option even while the job executes.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name M. (see -jsv option above or find more information
concerning JSV in
- -masterq wc_queue_list
- Available for qsub, qrsh, qsh,
qlogin and qalter. Only meaningful for parallel jobs, i.e.
together with the -pe option.
Defines or redefines a list of cluster queues, queue domains and queue
instances which may be used to become the so called master queue of
this parallel job. A more detailed description of wc_queue_list can
be found in The master queue is defined as the queue where the
parallel job is started. The other queues to which the parallel job spawns
tasks are called slave queues. A parallel job only has one
master queue.
This parameter has all the properties of a resource request and will be
merged with requirements derived from the -l option described
above.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified the this
hard resource requirement will be passed to defined JSV instances as
parameter with the name masterq. (see -jsv option above or
find more information concerning JSV in
- -notify
- Available for qsub, qrsh (with command) and
qalter only.
This flag, when set causes Sun Grid Engine to send "warning"
signals to a running job prior to sending the signals themselves. If a
SIGSTOP is pending, the job will receive a SIGUSR1 several seconds before
the SIGSTOP. If a SIGKILL is pending, the job will receive a SIGUSR2
several seconds before the SIGKILL. This option provides the running job,
before receiving the SIGSTOP or SIGKILL, a configured time interval to do
e.g. cleanup operations. The amount of time delay is controlled by the
notify parameter in each queue configuration (see
Note that the Linux operating system "misused" the user signals
SIGUSR1 and SIGUSR2 in some early Posix thread implementations. You might
not want to use the -notify option if you are running
multi-threaded applications in your jobs under Linux, particularly on 2.0
or earlier kernels.
Qalter allows changing this option even while the job executes.
Only if this option is used the parameter named notify with the value
y will be passed to defined JSV instances. (see -jsv option
above or find more information concerning JSV in
- -now y[es]|n[o]
- Available for qsub, qsh, qlogin and
qrsh.
-now y tries to start the job immediately or not at all. The command
returns 0 on success, or 1 on failure (also if the job could not be
scheduled immediately). For array jobs submitted with the -now
option, if all tasks cannot be immediately scheduled, no tasks are
scheduled. -now y is default for qsh, qlogin and
qrsh
With the -now n option, the job will be put into the pending queue if
it cannot be executed immediately. -now n is default for
qsub.
The value specified with this option or the corresponding value specified in
qmon will only be passed to defined JSV instances if the value is
yes. The name of the parameter will be now. The value will
be y also when then long form yes was specified during
submission. (see -jsv option above or find more information
concerning JSV in
- -N name
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
The name of the job. The name should follow the " name"
definition in Invalid job names will be denied at submit time.
If the -N option is not present, Sun Grid Engine assigns the name of
the job script to the job after any directory pathname has been removed
from the script-name. If the script is read from standard input, the job
name defaults to STDIN.
In the case of qsh or qlogin with the -N option is
absent, the string `INTERACT' is assigned to the job.
In the case of qrsh if the -N option is absent, the resulting
job name is determined from the qrsh command line by using the argument
string up to the first occurrence of a semicolon or whitespace and
removing the directory pathname.
Qalter allows changing this option even while the job executes.
The value specified with this option or the corresponding value specified in
qmon will be passed to defined JSV instances as parameter with the
name N. (see -jsv option above or find more information
concerning JSV in
- -noshell
- Available only for qrsh with a command line.
Do not start the command line given to qrsh in a user's login shell,
i.e. execute it without the wrapping shell.
This option can be used to speed up execution as some overhead, like the
shell startup and sourcing the shell resource files, is avoided.
This option can only be used if no shell-specific command line parsing is
required. If the command line contains shell syntax like environment
variable substitution or (back) quoting, a shell must be started. In this
case, either do not use the -noshell option or include the shell
call in the command line.
Example:
qrsh echo '$HOSTNAME'
Alternative call with the -noshell option
qrsh -noshell /bin/tcsh -f -c 'echo $HOSTNAME'
- -nostdin
- Available only for qrsh.
Suppress the input stream STDIN - qrsh will pass the option -n to the
command. This is especially useful, if multiple tasks are executed in
parallel using qrsh, e.g. in a process - it would be undefined,
which process would get the input.
- -o [[hostname]:]path,...
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
The path used for the standard output stream of the job. The path is
handled as described in the -e option for the standard error
stream.
By default the file name for standard output has the form
job_name.ojob_id and
job_name.ojob_id.task_id for array job tasks (see
-t option below).
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name o. (see -jsv option above or find more information
concerning JSV in
- -ot override_tickets
- Available for qalter only.
Changes the number of override tickets for the specified job. Requires
manager/operator privileges.
- -P project_name
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Specifies the project to which this job is assigned. The administrator needs
to give permission to individual users to submit jobs to a specific
project. (see -aprj option to
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name ot. (see -jsv option above or find more information
concerning JSV in
- -p priority
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Defines or redefines the priority of the job relative to other jobs.
Priority is an integer in the range -1023 to 1024. The default priority
value for jobs is 0.
Users may only decrease the priority of their jobs. Sun Grid Engine managers
and administrators may also increase the priority associated with jobs. If
a pending job has higher priority, it is earlier eligible for being
dispatched by the Sun Grid Engine scheduler.
If this option or a corresponding value in qmon is specified and the
priority is not 0 then this value will be passed to defined JSV instances
as parameter with the name p. (see -jsv option above or find
more information concerning JSV in
- -pe parallel_environment
n[-[
m]]|[-]m ,...
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Parallel programming environment (PE) to instantiate. For more detail about
PEs, please see the
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified then the
parameters pe_name, pe_min and pe_max will be passed
to configured JSV instances where pe_name will be the name of the
parallel environment and the values pe_min and pe_max
represent the values n and m which have been provided with the -pe
option. A missing specification of m will be expanded as value 9999999 in
JSV scripts and it represents the value infinity. (see -jsv option
above or find more information concerning JSV in
- -pty y[es]|n[o]
- Available for qrsh and qlogin only.
-pty yes enforces the job to be started in a pseudo terminal (pty). If no
pty is available, the job start fails. -pty no enforces the job to be
started without a pty. By default, qrsh without a command and
qlogin start the job in a pty, qrsh with a command starts
the job without a pty.
This parameter is not available in the JSV context. (see -jsv option
above or find more information concerning JSV in
- -q wc_queue_list
- Available for qsub, qrsh, qsh,
qlogin and qalter.
Defines or redefines a list of cluster queues, queue domains or queue
instances which may be used to execute this job. Please find a description
of wc_queue_list in This parameter has all the properties of a
resource request and will be merged with requirements derived from the
-l option described above.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified the
these hard and soft resource requirements will be passed to defined JSV
instances as parameters with the names q_hard and q_soft. If
regular expressions will be used for resource requests, then these
expressions will be passed as they are. Also shortcut names will not be
expanded. (see -jsv option above or find more information
concerning JSV in
- -R y[es]|n[o]
- Available for qsub, qrsh, qsh,
qlogin and qalter.
Indicates whether a reservation for this job should be done. Reservation is
never done for immediate jobs, i.e. jobs submitted using the -now
yes option. Please note that regardless of the reservation request,
job reservation might be disabled using max_reservation in and might be
limited only to a certain number of high priority jobs.
By default jobs are submitted with the -R n option.
The value specified with this option or the corresponding value specified in
qmon will only be passed to defined JSV instances if the value is
yes. The name of the parameter will be R. The value will be
y also when then long form yes was specified during
submission. (see -jsv option above or find more information
concerning JSV in
- -r y[es]|n[o]
- Available for qsub and qalter only.
Identifies the ability of a job to be rerun or not. If the value of
-r is 'yes', the job will be rerun if the job was aborted without
leaving a consistent exit state. (This is typically the case if the node
on which the job is running crashes). If -r is 'no', the job will
not be rerun under any circumstances.
Interactive jobs submitted with qsh, qrsh or qlogin are
not rerunnable.
Qalter allows changing this option even while the job executes.
The value specified with this option or the corresponding value specified in
qmon will only be passed to defined JSV instances if the value is
yes. The name of the parameter will be r. The value will be
y also when then long form yes was specified during
submission. (see -jsv option above or find more information
concerning JSV in
- -sc variable[=value],...
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Sets the given name/value pairs as the job's context. Value may be
omitted. Sun Grid Engine replaces the job's previously defined context
with the one given as the argument. Multiple -ac, -dc, and
-sc options may be given. The order is important.
Contexts provide a way to dynamically attach and remove meta-information to
and from a job. The context variables are not passed to the job's
execution context in its environment.
Qalter allows changing this option even while the job executes.
The outcome of the evaluation of all -ac, -dc, and -sc
options or corresponding values in qmon is passed to defined JSV
instances as parameter with the name aac. (see -jsv option
above or find more information concerning JSV in
- -shell y[es]|n[o]
- Available only for qsub.
-shell n causes qsub to execute the command line directly, as if by
No command shell will be executed for the job. This option only applies
when -b y is also used. Without -b y, -shell n has no
effect.
This option can be used to speed up execution as some overhead, like the
shell startup and sourcing the shell resource files is avoided.
This option can only be used if no shell-specific command line parsing is
required. If the command line contains shell syntax, like environment
variable substitution or (back) quoting, a shell must be started. In this
case either do not use the -shell n option or execute the shell as
the command line and pass the path to the executable as a parameter.
If a job executed with the -shell n option fails due to a user error,
such as an invalid path to the executable, the job will enter the error
state.
-shell y cancels the effect of a previous -shell n.
Otherwise, it has no effect.
See -b and -noshell for more information.
The value specified with this option or the corresponding value specified in
qmon will only be passed to defined JSV instances if the value is
yes. The name of the parameter will be shell. The value will
be y also when then long form yes was specified during
submission. (see -jsv option above or find more information
concerning JSV in
- -soft
- Available for qsub, qsh, qrsh,
qlogin and qalter only.
Signifies that all resource requirements following in the command line will
be soft requirements and are to be filled on an "as available"
basis.
As Sun Grid Engine scans the command line and script file for Sun Grid
Engine options and parameters, it builds a list of resources required by
the job. All such resource requests are considered as absolutely essential
for the job to commence. If the -soft option is encountered during
the scan then all following resources are designated as "soft
requirements" for execution, or "nice-to-have, but not
essential". If the -hard flag (see above) is encountered at a
later stage of the scan, all resource requests following it once again
become "essential". The -hard and -soft options in
effect act as "toggles" during the scan.
If this option or a corresponding value in qmon is specified then the
corresponding -q and -l resource requirements will be passed
to defined JSV instances as parameter with the names q_soft and
l_soft. Find for information in the sections describing -q
and -l. (see -jsv option above or find more information
concerning JSV in
- -sync y[es]|n[o]
- Available for qsub.
-sync y causes qsub to wait for the job to complete before
exiting. If the job completes successfully, qsub's exit code will
be that of the completed job. If the job fails to complete successfully,
qsub will print out a error message indicating why the job failed
and will have an exit code of 1. If qsub is interrupted, e.g. with
CTRL-C, before the job completes, the job will be canceled.
With the -sync n option, qsub will exit with an exit code of 0
as soon as the job is submitted successfully. -sync n is default
for qsub.
If -sync y is used in conjunction with -now y, qsub
will behave as though only -now y were given until the job has been
successfully scheduled, after which time qsub will behave as though
only -sync y were given.
If -sync y is used in conjunction with -t n[-m[:i]],
qsub will wait for all the job's tasks to complete before exiting.
If all the job's tasks complete successfully, qsub's exit code will
be that of the first completed job tasks with a non-zero exit code, or 0
if all job tasks exited with an exit code of 0. If any of the job's tasks
fail to complete successfully, qsub will print out an error message
indicating why the job task(s) failed and will have an exit code of 1. If
qsub is interrupted, e.g. with CTRL-C, before the job completes,
all of the job's tasks will be canceled.
Information that this switch was specified during submission is not
available in the JSV context. (see -jsv option above or find more
information concerning JSV in
- -S [[hostname]:]pathname,...
- Available for qsub, qsh and qalter.
Specifies the interpreting shell for the job. Only one pathname
component without a host specifier is valid and only one path name
for a given host is allowed. Shell paths with host assignments define the
interpreting shell for the job if the host is the execution host. The
shell path without host specification is used if the execution host
matches none of the hosts in the list.
Furthermore, the pathname can be constructed with pseudo environment
variables as described for the -e option above.
In the case of qsh the specified shell path is used to execute the
corresponding command interpreter in the (via its -e option)
started on behalf of the interactive job. Qalter allows changing
this option even while the job executes. The modified parameter will only
be in effect after a restart or migration of the job, however.
If this option or a corresponding value in qmon is specified then
this value will be passed to defined JSV instances as parameter with the
name S. (see -jsv option above or find more information
concerning JSV in
- -t n[-m[:s]]
- Available for qsub and qalter only.
Submits a so called Array Job, i.e. an array of identical tasks being
differentiated only by an index number and being treated by Sun Grid
Engine almost like a series of jobs. The option argument to -t
specifies the number of array job tasks and the index number which will be
associated with the tasks. The index numbers will be exported to the job
tasks via the environment variable SGE_TASK_ID. The option
arguments n, m and s will be available through the environment variables
SGE_TASK_FIRST, SGE_TASK_LAST and SGE_TASK_STEPSIZE.
Following restrictions apply to the values n and m:
1 <= n <= MIN(2^31-1, max_aj_tasks)
1 <= m <= MIN(2^31-1, max_aj_tasks)
n <= m
max_aj_tasks is defined in the cluster configuration (see
The task id range specified in the option argument may be a single number, a
simple range of the form n-m or a range with a step size. Hence, the task id
range specified by 2-10:2 would result in the task id indexes 2, 4, 6, 8, and
10, for a total of 5 identical tasks, each with the environment variable
SGE_TASK_ID containing one of the 5 index numbers.
All array job tasks inherit the same resource requests and attribute definitions
as specified in the
qsub or
qalter command line, except for the
-t option. The tasks are scheduled independently and, provided enough
resources exist, concurrently, very much like separate jobs. However, an array
job or a sub-array there of can be accessed as a single unit by commands like
or See the corresponding manual pages for further detail.
Array jobs are commonly used to execute the same type of operation on varying
input data sets correlated with the task index number. The number of tasks in
a array job is unlimited.
STDOUT and STDERR of array job tasks will be written into different files with
the default location
<jobname>.['e'|'o']<job_id>'.'<task_id>
In order to change this default, the
-e and
-o options (see above)
can be used together with the pseudo environment variables $HOME, $USER,
$JOB_ID, $JOB_NAME, $HOSTNAME, and $SGE_TASK_ID.
Note, that you can use the output redirection to divert the output of all tasks
into the same file, but the result of this is undefined.
If this option or a corresponding value in
qmon is specified then this
value will be passed to defined JSV instances as parameters with the name
t_min,
t_max and
t_step (see
-jsv option above or
find more information concerning JSV in
- -tc max_running_tasks
- -allow users to limit concurrent array job task execution.
Parameter max_running_tasks specifies maximum number of simultaneously
running tasks. For example we have running SGE with 10 free slots. We call
qsub -t 1-100 -tc 2 jobscript. Then only 2 tasks will be scheduled to run
even when 8 slots are free.
- -terse
- Available for qsub only.
-terse causes the qsub to display only the job-id of the job being
submitted rather than the regular "Your job ..." string. In case
of an error the error is reported on stderr as usual.
This can be helpful for scripts which need to parse qsub output to
get the job-id.
Information that this switch was specified during submission is not
available in the JSV context. (see -jsv option above or find more
information concerning JSV in
- -u username,...
- Available for qalter only. Changes are only made on
those jobs which were submitted by users specified in the list of
usernames. For managers it is possible to use the qalter -u '*'
command to modify all jobs of all users.
If you use the -u switch it is not permitted to specify an additional
wc_job_range_list.
- -v variable[=value],...
- Available for qsub, qrsh (with command
argument) and qalter.
Defines or redefines the environment variables to be exported to the
execution context of the job. If the -v option is present Sun Grid
Engine will add the environment variables defined as arguments to the
switch and, optionally, values of specified variables, to the execution
context of the job.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
All environment variables specified with -v, -V or the DISPLAY
variable provided with -display will be exported to the defined JSV
instances only optionally when this is requested explicitly during the job
submission verification. (see -jsv option above or find more
information concerning JSV in
- -verbose
- Available only for qrsh and
Unlike qsh and qlogin, qrsh does not output any
informational messages while establishing the session, compliant with the
standard and system calls. If the option -verbose is set,
qrsh behaves like the qsh and qlogin commands,
printing information about the process of establishing the or
session.
- -verify
- Available for qsub, qsh, qrsh,
qlogin and qalter.
Instead of submitting a job, prints detailed information about the would-be
job as though -j were used, including the effects of command-line
parameters and the external environment.
- -V
- Available for qsub, qsh, qrsh with
command and qalter.
Specifies that all environment variables active within the qsub
utility be exported to the context of the job.
All environment variables specified with -v, -V or the DISPLAY
variable provided with -display will be exported to the defined JSV
instances only optionally when this is requested explicitly during the job
submission verification. (see -jsv option above or find more
information concerning JSV in
- -w e|w|n|p|v
- Available for qsub, qsh, qrsh,
qlogin and qalter.
Specifies a validation level applied to the job to be submitted (
qsub, qlogin, and qsh) or the specified queued job
(qalter). The information displayed indicates whether the job can
possibly be scheduled assuming an empty system with no other jobs.
Resource requests exceeding the configured maximal thresholds or
requesting unavailable resource attributes are possible causes for jobs to
fail this validation.
The specifiers e, w, n and v define the following validation modes:
`e' error - jobs with invalid requests will be
rejected.
`w' warning - only a warning will be displayed
for invalid requests.
`n' none - switches off validation; the default for
qsub, qalter, qrsh, qsh
and qlogin.
`p' poke - does not submit the job but prints a
validation report based on a cluster as is with
all resource utilizations in place.
`v' verify - does not submit the job but prints a
validation report based on an empty cluster.
Note, that the necessary checks are performance consuming and hence the
checking is switched off by default. It should also be noted that load
values are not taken into account with the verification since they are
assumed to be too volatile. To cause -w e verification to be passed at
submission time, it is possible to specify non-volatile values
(non-consumables) or maximum values (consumables) in complex_values.
- -wd working_dir
- Available for qsub, qsh, qrsh and
qalter only.
Execute the job from the directory specified in working_dir. This switch
will activate Sun Grid Engine's path aliasing facility, if the
corresponding configuration files are present (see
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however. The parameter value will be available in defined JSV
instances as parameter with the name cwd (see -cwd switch
above or find more information concerning JSV in
- command
- Available for qsub and qrsh only.
The job's scriptfile or binary. If not present or if the operand is the
single-character string '-', qsub reads the script from standard
input.
The command will be available in defined JSV instances as parameter with the
name CMDNAME (see -jsv option above or find more information
concerning JSV in
- command_args
- Available for qsub, qrsh and qalter
only.
Arguments to the job. Not valid if the script is entered from standard
input.
Qalter allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or migration of
the job, however.
The number of command arguments is provided to configured JSV instances as
parameter with the name CMDARGS. Also the argument values can by
accessed. Argument names have the format CMDARG<number> where
<number> is a integer between 0 and CMDARGS - 1. (see
-jsv option above or find more information concerning JSV in
- xterm_args
- Available for qsh only.
Arguments to the executable, as defined in the configuration. For details,
refer to
Information concerning xterm_args will be available in JSV context as
parameters with the name CMDARGS and CMDARG<number>.
Find more information above in section command_args. (see
-jsv option above or find more information concerning JSV in
ENVIRONMENTAL VARIABLES¶
- SGE_ROOT
- Specifies the location of the Sun Grid Engine standard
configuration files.
- SGE_CELL
- If set, specifies the default Sun Grid Engine cell. To
address a Sun Grid Engine cell qsub, qsh, qlogin or
qalter use (in the order of precedence):
The name of the cell specified in the
environment variable SGE_CELL, if it is set.
The name of the default cell, i.e.
default.
- SGE_DEBUG_LEVEL
- If set, specifies that debug information should be written
to stderr. In addition the level of detail in which debug information is
generated is defined.
- SGE_QMASTER_PORT
- If set, specifies the tcp port on which is expected to
listen for communication requests. Most installations will use a services
map entry for the service "sge_qmaster" instead to define that
port.
- DISPLAY
- For qsh jobs the DISPLAY has to be specified at job
submission. If the DISPLAY is not set by using the -display or the
-v switch, the contents of the DISPLAY environment variable are
used as default.
In addition to those environment variables specified to be exported to the job
via the
-v or the
-V option (see above)
qsub,
qsh,
and
qlogin add the following variables with the indicated values to the
variable list:
- SGE_O_HOME
- the home directory of the submitting client.
- SGE_O_HOST
- the name of the host on which the submitting client is
running.
- SGE_O_LOGNAME
- the LOGNAME of the submitting client.
- SGE_O_MAIL
- the MAIL of the submitting client. This is the mail
directory of the submitting client.
- SGE_O_PATH
- the executable search path of the submitting client.
- SGE_O_SHELL
- the SHELL of the submitting client.
- SGE_O_TZ
- the time zone of the submitting client.
- SGE_O_WORKDIR
- the absolute path of the current working directory of the
submitting client.
Furthermore, Sun Grid Engine sets additional variables into the job's
environment, as listed below.
- ARC
- SGE_ARCH
- The Sun Grid Engine architecture name of the node on which
the job is running. The name is compiled-in into the binary.
- SGE_CKPT_ENV
- Specifies the checkpointing environment (as selected with
the -ckpt option) under which a checkpointing job executes. Only
set for checkpointing jobs.
- SGE_CKPT_DIR
- Only set for checkpointing jobs. Contains path
ckpt_dir (see ) of the checkpoint interface.
- SGE_STDERR_PATH
- the pathname of the file to which the standard error stream
of the job is diverted. Commonly used for enhancing the output with error
messages from prolog, epilog, parallel environment start/stop or
checkpointing scripts.
- SGE_STDOUT_PATH
- the pathname of the file to which the standard output
stream of the job is diverted. Commonly used for enhancing the output with
messages from prolog, epilog, parallel environment start/stop or
checkpointing scripts.
- SGE_STDIN_PATH
- the pathname of the file from which the standard input
stream of the job is taken. This variable might be used in combination
with SGE_O_HOST in prolog/epilog scripts to transfer the input file from
the submit to the execution host.
- SGE_JOB_SPOOL_DIR
- The directory used by to store job related data during job
execution. This directory is owned by root or by a Sun Grid Engine
administrative account and commonly is not open for read or write access
to regular users.
- SGE_TASK_ID
- The index number of the current array job task (see
-t option above). This is an unique number in each array job and
can be used to reference different input data records, for example. This
environment variable is set to "undefined" for non-array jobs.
It is possible to change the predefined value of this variable with
-v or -V (see options above).
- SGE_TASK_FIRST
- The index number of the first array job task (see -t
option above). It is possible to change the predefined value of this
variable with -v or -V (see options above).
- SGE_TASK_LAST
- The index number of the last array job task (see -t
option above). It is possible to change the predefined value of this
variable with -v or -V (see options above).
- SGE_TASK_STEPSIZE
- The step size of the array job specification (see -t
option above). It is possible to change the predefined value of this
variable with -v or -V (see options above).
- ENVIRONMENT
- The ENVIRONMENT variable is set to BATCH to identify that
the job is being executed under Sun Grid Engine control.
- HOME
- The user's home directory path from the file.
- HOSTNAME
- The hostname of the node on which the job is running.
- JOB_ID
- A unique identifier assigned by the when the job was
submitted. The job ID is a decimal integer in the range 1 to 99999.
- JOB_NAME
- The job name. For batch jobs or jobs submitted by
qrsh with a command, the job name is built as basename of the
qsub script filename resp. the qrsh command. For interactive
jobs it is set to `INTERACTIVE' for qsh jobs, `QLOGIN' for
qlogin jobs and `QRLOGIN' for qrsh jobs without a command.
This default may be overwritten by the -N. option.
- JOB_SCRIPT
- The path to the job script which is executed. The value can
not be overwritten by the -v or -V option.
- LOGNAME
- The user's login name from the file.
- NHOSTS
- The number of hosts in use by a parallel job.
- NQUEUES
- The number of queues allocated for the job (always 1 for
serial jobs).
- NSLOTS
- The number of queue slots in use by a parallel job.
- PATH
- A default shell search path of:
/usr/local/bin:/usr/ucb:/bin:/usr/bin
- SGE_BINARY_PATH
- The path where the Sun Grid Engine binaries are installed.
The value is the concatenation of the cluster configuration value
binary_path and the architecture name $SGE_ARCH environment
variable.
- PE
- The parallel environment under which the job executes (for
parallel jobs only).
- PE_HOSTFILE
- The path of a file containing the definition of the virtual
parallel machine assigned to a parallel job by Sun Grid Engine. See the
description of the $pe_hostfile parameter in for details on the
format of this file. The environment variable is only available for
parallel jobs.
- QUEUE
- The name of the cluster queue in which the job is
running.
- REQUEST
- Available for batch jobs only.
The request name of a job as specified with the -N switch (see above)
or taken as the name of the job script file.
- RESTARTED
- This variable is set to 1 if a job was restarted either
after a system crash or after a migration in case of a checkpointing job.
The variable has the value 0 otherwise.
- SHELL
- The user's login shell from the file. Note: This is
not necessarily the shell in use for the job.
- TMPDIR
- The absolute path to the job's temporary working
directory.
- TMP
- The same as TMPDIR; provided for compatibility with
NQS.
- TZ
- The time zone variable imported from if set.
- USER
- The user's login name from the file.
- SGE_JSV_TIMEOUT
- If the response time of the client JSV is greater than this
timeout value, then the JSV will attempt to be re-started. The default
value is 10 seconds, and this value must be greater than 0. If the timeout
has been reached, the JSV will only try to re-start once, if the timeout
is reached again an error will occur.
RESTRICTIONS¶
There is no controlling terminal for batch jobs under Sun Grid Engine, and any
tests or actions on a controlling terminal will fail. If these operations are
in your .login or .cshrc file, they may cause your job to abort.
Insert the following test before any commands that are not pertinent to batch
jobs in your .login:
if ( $?JOB_NAME) then
echo "Sun Grid Engine spooled job"
exit 0
endif
Don't forget to set your shell's search path in your shell start-up before this
code.
EXIT STATUS¶
The following exit values are returned:
- 0
- Operation was executed successfully.
- 25
- It was not possible to register a new job according to the
configured max_u_jobs or max_jobs limit. Additional
information may be found in
- >0
- Error occurred.
EXAMPLES¶
The following is the simplest form of a Sun Grid Engine script file.
=====================================================
#!/bin/csh
a.out
=====================================================
The next example is a more complex Sun Grid Engine script.
=====================================================
#!/bin/csh
# Which account to be charged cpu time
#$ -A santa_claus
# date-time to run, format [[CC]yy]MMDDhhmm[.SS]
#$ -a 12241200
# to run I want 6 or more parallel processes
# under the PE pvm. the processes require
# 128M of memory
#$ -pe pvm 6- -l mem=128
# If I run on dec_x put stderr in /tmp/foo, if I
# run on sun_y, put stderr in /usr/me/foo
#$ -e dec_x:/tmp/foo,sun_y:/usr/me/foo
# Send mail to these users
#$ -M santa@nothpole,claus@northpole
# Mail at beginning/end/on suspension
#$ -m bes
# Export these environmental variables
#$ -v PVM_ROOT,FOOBAR=BAR
# The job is located in the current
# working directory.
#$ -cwd
a.out
==========================================================
FILES¶
$REQUEST.oJID[.TASKID] STDOUT of job #JID
$REQUEST.eJID[.TASKID] STDERR of job
$REQUEST.poJID[.TASKID] STDOUT of par. env. of job
$REQUEST.peJID[.TASKID] STDERR of par. env. of job
$cwd/.sge_aliases cwd path aliases
$cwd/.sge_request cwd default request
$HOME/.sge_aliases user path aliases
$HOME/.sge_request user default request
<sge_root>/<cell>/common/sge_aliases
cluster path aliases
<sge_root>/<cell>/common/sge_request
cluster default request
<sge_root>/<cell>/common/act_qmaster
Sun Grid Engine master host file
SEE ALSO¶
COPYRIGHT¶
If configured correspondingly,
qrsh and
qlogin contain portions of
the
rsh,
rshd,
telnet and
telnetd code copyrighted
by The Regents of the University of California. Therefore, the following note
applies with respect to
qrsh and
qlogin: This product includes
software developed by the University of California, Berkeley and its
contributors.
See as well as the information provided in <sge_root>/3rd_party/qrsh and
<sge_root>/3rd_party/qlogin for a statement of further rights and
permissions.