'\" t
.\"___INFO__MARK_BEGIN__
.\"
.\" Copyright: 2004 by Sun Microsystems, Inc.
.\" Copyright (C), 2013  Dave Love, University of Liverpool
.\"
.\"___INFO__MARK_END__
.\"
.\" Some handy macro definitions [from Tom Christensen's man(1) manual page].
.\"
.de SB		\" small and bold
.if !"\\$1"" \\s-2\\fB\&\\$1\\s0\\fR\\$2 \\$3 \\$4 \\$5
..
.\"
.de T		\" switch to typewriter font
.ft CW		\" probably want CW if you don't have TA font
..
.\"
.de TY		\" put $1 in typewriter font
.if t .T
.if n ``\c
\\$1\c
.if t .ft P
.if n \&''\c
\\$2
..
.\" "
.de M		\" man page reference
\\fI\\$1\\fR\\|(\\$2)\\$3
..
.de MO		\" other man page reference
\\fI\\$1\\fR\\|(\\$2)\\$3
..
.TH CHECKPOINT 5 "2012-01-07" "SGE 8.1.3pre" "Grid Engine File Formats"
.\"
.SH NAME
checkpoint \- Grid Engine checkpointing environment configuration file format
.\"
.\"
.SH DESCRIPTION
Checkpointing is a facility to save the complete status of an executing
program or job and to restore and restart from this so-called checkpoint
at a later point of time if the original program or job was halted, e.g.
through a system crash.
.PP
Grid Engine provides various levels of checkpointing support (see
.M sge_ckpt 1 ).
The checkpointing environment described here is a means to configure
the different types of checkpointing in use for your Grid Engine cluster or
parts thereof. For that purpose you can define the operations which
have to be executed in initiating a checkpoint generation, a migration
of a checkpoint to another host, or a restart of a checkpointed
application.
.PP
Supporting different operating systems may easily force Grid Engine to 
introduce operating system dependencies for the configuration of the 
checkpointing configuration file and updates of the supported operating 
system versions may lead to frequently changing implementation details. 
Please refer to the <sge_root>/ckpt directory for more 
information.
.PP
Please use the \fI\-ackpt\fP, \fI\-dckpt\fP, \fI\-mckpt\fP or \fI\-sckpt\fP
options to the
.M qconf 1
command to manipulate checkpointing environments from the command-line or
use the corresponding
.M qmon 1
dialogue for X-Windows based interactive configuration.
.PP
Note, Grid Engine allows backslashes (\\) be used to escape newline
characters. The backslash and the newline are replaced with a space
character before any interpretation.
.\"
.\"
.SH FORMAT
The format of a
.I checkpoint
file is defined as follows:
.SS "\fBckpt_name\fP"
The name of the checkpointing environment in the format for \fIckpt_name\fP in
.M sge_types 5 .
To be used in the
.M qsub 1
\fB\-ckpt\fP switch or for the
.M qconf 1
options mentioned above.
.SS "\fBinterface\fP"
The type of checkpointing to be used. Currently, the following types are
valid:
.IP "\fIhibernator\fP"
The Hibernator kernel level checkpointing is interfaced.
.IP "\fIcpr\fP"
The SGI kernel level checkpointing is used.
.IP "\fItransparent\fP"
Grid Engine assumes that the jobs submitted with reference to this checkpointing
interface use a checkpointing library such as provided by 
the free package \fICondor\fP.
.IP "\fIuserdefined\fP"
Grid Engine assumes that the jobs submitted with reference to this checkpointing
interface perform their private checkpointing method.
.IP "\fIapplication-level\fP"
Uses all of the interface commands configured in the checkpointing object
like in the case of one of the kernel level checkpointing interfaces
(\fIcpr\fP, etc.) except for the
.B restart_command
(see below), which is not
used (even if it is configured) but the job script is invoked in case of a
restart instead.
.SS "\fBckpt_command\fP"
A command-line type command string to be executed by Grid Engine in order to
initiate a checkpoint.  The following pseudo-variables are available to be
substituted in the value:
.\" fixme: explain them
.IP \fB$host\fP
The name of the host on which the command is executed.
.IP \fB$ja_task_id\fP
The array job task index (0 if not an array job).
.IP "\fB$job_owner\fP"
The user name of the job owner.
.IP "\fB$job_id\fP"
Grid Engine's unique job identification number.
.IP "\fB$job_name\fP"
The name of the job.
.IP \fB$queue\fP
The cluster queue name of the master queue instance, on which the
command is started.
.IP \fB$job_pid\fP
The process id of the job/task to checkpoint.
.IP \fB$ckpt_dir\fP
See
.B ckpt_dir
below.
.IP \fB$ckpt_signal\fP
See
.B signal
below.
.IP \fB$sge_cell\fP
The SGE_CELL environment variable (useful for locating files).
.IP \fB$sge_root\fP
The SGE_ROOT environment variable (useful for locating files).
.SS "\fBmigr_command\fP"
A command-line type command string to be executed by Grid Engine during a
migration of a checkpointing job from one host to another.  The same
pseudo-variables are available as for \fBckpt_command\fP.  Note that
the command is expected to create a checkpoint itself \- the
checkpointing command isn't called automatically on migration.
.SS "\fBrestart_command\fP"
A command-line type command string to be executed by Grid Engine when restarting
a previously checkpointed application.  The same
pseudo-variables are available as for \fBckpt_command\fP.
.SS "\fBclean_command\fP"
A command-line type command string to be executed by Grid Engine in order
to cleanup after a checkpointed application has finished.  The same
pseudo-variables are available as for \fBckpt_command\fP.
.SS "\fBckpt_dir\fP"
A file system location to which checkpoints of potentially considerable
size should be stored.
.SS "\fBsignal\fP"
A Unix signal to be sent to a job by Grid Engine to initiate checkpoint
generation. The value for this field can either be a symbolic name from the
list produced by the \fI\-l\fP option of the
.MO kill 1
command or an integer number which must be a valid signal on the systems
used for checkpointing.
.SS "\fBwhen\fP"
The points of time when checkpoints are expected to be generated.
Valid values for this parameter are composed from the letters \fIs\fP,
\fIm\fP,
\fIx\fP,
\fIr\fP, and
any combinations thereof without any separating character in between. The
same letters are allowed for the \fI\-c\fP option of the
.M qsub 1
command which will overwrite the definitions in the checkpointing environment
used.
The meaning of the letters is as follows:
.IP "\fIs\fP"
A job is checkpointed, aborted and, if possible, migrated if the
corresponding
.M sge_execd 8
is shut down on the job's host.  This operation is
handled by the specified \fBmigr_command\fP.
.IP "\fIm\fP"
checkpoints are generated periodically at the \fImin_cpu_interval\fP
interval defined by the queue (see
.M queue_conf 5 )
in which a job executes.
.IP "\fIx\fP"
A job is checkpointed, aborted and, if possible, migrated as soon as the job
gets suspended (manually as well as automatically).  This operation is
handled by the specified \fBmigr_command\fP.
.IP "\fIr\fP"
A job will be rescheduled (not checkpointed) when the host on which the job
currently runs goes into the "unknown" state and the time interval
\fIreschedule_unknown\fP (see
.M sge_conf 5 )
defined in the global/local cluster configuration is exceeded.
.\"
.SH "ENVIRONMENT VARIABLES"
\fBSGE_BINDING\fP and \fBSGE_CKPT_DIR\fP may be
specified on job submission.  See
.M submit 1 .
.\"
.\"
.SH RESTRICTIONS
\fBNote\fP that the functionality of any checkpointing,
migration or restart procedures provided by default with
the Grid Engine distribution, as well as the way how they are invoked in
the \fIckpt_command\fP, \fImigr_command\fP or \fIrestart_command\fP
parameters of any default checkpointing environments, should not be
changed; otherwise the functionality remains the full responsibility
of the administrator configuring the checkpointing environment.
Grid Engine will just invoke these procedures and evaluate their
exit status. If the procedures do not perform their tasks
properly, or are not invoked in a proper fashion, the checkpointing
mechanism may behave unexpectedly; Grid Engine has no means to detect
this \- all exit codes are treated as successful operation except for
the case of kernel checkpointing.
.PP
See also the restrictions in
.M sge_ckpt 5 .
.\"
.\"
.SH "SEE ALSO"
.M sge_intro 1 ,
.M sge_ckpt 5 ,
.M sge_types 5 ,
.M qconf 1 ,
.M qmod 1 ,
.M qsub 1 ,
.M sge_execd 8 .
.\"
.SH "COPYRIGHT"
See
.M sge_intro 1
for a full statement of rights and permissions.