other versions
- jessie-backports 8.1.9+dfsg-1~bpo8+1
- stretch 8.1.9+dfsg-4+deb9u2
- testing 8.1.9+dfsg-9
- unstable 8.1.9+dfsg-9
remote_startup(5) | Grid Engine User Commands | remote_startup(5) |
NAME¶
remote_startup - the Grid Engine remote startup mechanismDESCRIPTION¶
Grid Engine supports several commands to facilitate interactive commands or remote startup of a tightly integrated parallel job. Each command can be set up with (option -sconf) to use different daemons and commands to start the final session. Different startup methods can therefore contain different daemons and commands, and are not related to other startup methods in any way, although it is often desirable to have the same communication method for all startup methods. Each method requires a separate instance of the communication daemon to be started by for each job, which thus must use a randomly-chosen port, to which the client will try to connect. This is necessary to support tight integration, ensuring that everything for a given job stays in the same process tree and can be properly controlled and accounted by Grid Engine.QLOGIN¶
An interactive qlogin session invoked by will show up in qstat with the default name QLOGIN unless changed by the -N name option. The two entries qlogin_daemon and qlogin_command are responsible for establishing the communication to start such a session. The default is the value builtin, which will trigger an SGE internal communication method:- qlogin_command
- builtin
- qlogin_daemon
- builtin
- qlogin_command
- /usr/bin/telnet
- qlogin_daemon
- /usr/sbin/in.telnetd
- qlogin_command
- /opt/sge/util/resources/wrappers/qlogin_wrapper
- qlogin_daemon
- /usr/sbin/sshd -i
QRLOGIN¶
An interactive qrlogin session invoked by without a command will show up in qstat having the default name QRLOGIN unless changed by the -N name option. The two entries rlogin_daemon and rlogin_command are responsible for establishing the command to start such a session. The default is the value builtin, which will trigger an SGE internal communication method:- rlogin_command
- builtin
- rlogin_daemon
- builtin
- rlogin_command
- $SGE_ROOT/utilbin/$ARC/rlogin
- rlogin_daemon
- /usr/sbin/in.rlogind
- rlogin_command
- /usr/bin/ssh
- rlogin_daemon
- /usr/sbin/sshd -i
QRSH¶
An interactive session for a remote command invoked by with a command will show up in qstat by default with name of the command issued, unless changed by the -N name option. The two entries rsh_daemon and rsh_command are responsible for establishing the communication to start such a session. This startup method will also be used by the master task of a tightly integrated parallel job to start slave processes on other granted exechosts. The default is the value builtin, which will trigger an SGE internal communication method:- rsh_command
- builtin
- rsh_daemon
- builtin
- rsh_command
- $SGE_ROOT/utilbin/$ARC/rsh
- rsh_daemon
- $SGE_ROOT/utilbin/$ARC/rshd -l
- rsh_command
- /usr/bin/ssh
- rsh_daemon
- /usr/sbin/sshd -i
LOCAL CONFIGURATIONS OF EXECHOSTS¶
It is important to note that the communication method set up for one particular startup method must match at each end. This can either be achieved by using only a global configuration, or carefully setting up local configurations for the exechosts involved. Whether or not local configurations exist, which must be taken care of, can be checked with qconf -sconfl. As a general rule, for setting up a communication method between a machine A (where the command is issued) and a machine B (where the daemon is started) it must be guaranteed that the:setup communication method for the command on
machine A
(either global configuration from or local configuration qconf -sconf
A of machine A)
matches
setup communication method for the daemon for
machine B
(either global configuration from or local configuration qconf -sconf
B of machine B)
This way it is also possible to use different communication methods, depending
whether a connection from A to B is invoked, or from B to A.
RESTRICTIONS¶
For all three communication methods, a direct connection between the target and the source machine where the particular command was issued must exist. This can also be implemented using TCP/IP forwarding, but will usually fail if one machine is behind NAT which will mangle the machines' addresses. The communication methods won't work with simple firewalling of the exec hosts since the methods use a random port. It may be possible to set up application-specific firewalling, if necessary, or to wrap the methods and start an SSH tunnel on the port specified for each communication instance. The builtin method does not support forwarding of X graphics from the compute nodes, or GSSAPI tokens to them. If you need that for any of the remote methods, you will want to set up SSH communication instead.SSH AUTHENTICATION¶
To allow the SSH setup explained above to work, the user must be authenticated without the use of a passphrase. While entering a passphrase would still work for interactive commands, it will fail in case of a tightly integrated parallel job, where the master process tries to start a slave process on another exechost. You can set up passphraseless SSH keys, although this is discouraged. A simpler and global working setup is to use host-based authentication ⟨URL: http://arc.liv.ac.uk/SGE/howto/hostbased-ssh.html ⟩ for the machines inside the cluster.SSH TIGHT INTEGRATION¶
To have a tight integration of SSH into SGE, the started sshd needs an additional group ID to be attached. With this additional group ID, SGE is able to record the resource consumption and computing time in a correct way. Also a qdel of such a job will be able to succeed. Such a tight SSH integration can be achieved by two means:- Use of PAM
- The easiest way on supported platforms (at least GNU/Linux): a module is available for use with the system ssh; it attaches the necessary additional group ID to the started process to provide tight integration. See also the workshop paper ⟨URL: http://gridengine.org/assets/static/ws2007/K5SGE.pdf ⟩.
- Recompile Grid Engine with ./aimk -tight-ssh ...
- The source of Grid Engine contains the necessary additions to compile a modified sshd, which will honor the additional group ID and attach it also to the started process. It's necessary to provide the source of OpenSSH in the directory 3rd_party inside $SGE_ROOT having a plain name ‘openssh’. Inside this directory the file sshd.c needs to be patched:
in main():
init_rng();
#ifdef SGESSH_INTEGRATION
sgessh_readconfig();
#endif
in privsep_postauth():
/* Drop privileges */
#ifdef SGESSH_INTEGRATION
sgessh_do_setusercontext(authctxt->pw);
#else
do_setusercontext(authctxt->pw);
#endif
See the original documentation ⟨URL:
http://gridengine.org/assets/static/ws2007/SGE-openSSHTightIntegration.RonChen.pdf
⟩.RESTRICTING ACCESS¶
With the builtin method in use, there is no need to allow direct access for normal users to compute nodes with ssh etc. However, you may want to allow users to access the nodes for debugging. If you don't want to over-subscribe the nodes, so that qrsh etc. can be used for access, you can use PAM to restrict access for a user only to the nodes on which they have a running job, so as to minimize interference with other others. There are two possible ways. The cleanest uses Otherwise you can use generic PAM modules, such as or with modifications to their configuration set up and taken down in the job prolog and epilog respectively. See, for instance, a user list message ⟨URL: http://gridengine.markmail.org/message/mu3i7haeshlevu6q?q=282211 ⟩, and other examples of similar prolog/epilog scripts provided with locking in the pam_authuser contribution in the Torque distribution.SECURITY¶
See the notes above concerning security of the communication channel.EXAMPLES¶
Using SSH with the PAM module, forcing tty allocation, and preventing the delegation of GSSAPI credentials to the compute nodes:rsh_daemon /opt/sge/util/rshdwrapper
rsh_command ssh -tt -o GSSAPIDelegateCredentials=no
qlogin_daemon /opt/sge/util/rshdwrapper
qlogin_command ssh -tt -o GSSAPIDelegateCredentials=no
rlogin_daemon /opt/sge/util/rshdwrapper
rlogin_command ssh -tt -o GSSAPIDelegateCredentials=no
Old-style method, using telnet and rlogin:
qlogin_command /usr/bin/telnet
qlogin_daemon /usr/sbin/in.telnetd
rlogin_command /opt/sge/utilbin/lx-x86/rlogin
rlogin_daemon /usr/sbin/in.rlogind
rsh_command /opt/sge/utilbin/lx-x86/rsh
rsh_daemon /opt/sge/utilbin/lx-x86/rshd -l
FILES¶
- $SGE_ROOT/util/resources/wrappers/qlogin_wrapper
- SSH-based wrapper for qlogin (see above)
SEE ALSO¶
Grid Engine-specific remote programs ⟨URL: http://arc.liv.ac.uk/repos/darcs/sge/source/3rdparty/remote/remote.html ⟩.AUTHOR¶
Man page written by Reuti, partly based on Sun material. Some additions by Dave Love.COPYRIGHT¶
See for a full statement of rights and permissions.2010/11/22 20:58:24 | SGE 8.1.3pre |