NAME¶
mpi - LAM/MPI implementation
DESCRIPTION¶
LAM features a full implementation of Message Passing Interface (MPI) 1 standard
with many features included from the MPI-2 standard.
Compliant applications are source code portable between LAM and any other
implementation of MPI. In addition to meeting the standard in a high quality
manner, LAM offers extensive monitoring capabilities to support debugging.
Monitoring happens on two levels. LAM has the hooks to allow a snapshot of
process and message status to be taken at any time during an application run.
The status includes all aspects of synchronization plus datatype map /
signature, communicator group membership and message contents. On the second
level, the MPI library is instrumented to produce a cumulative record of
communication, which can be visualized either at runtime or post-mortem.
LAM/MPI features the ability to change its underlying message transport
mechanism, as well as pass tunable parameters to different components in
LAM/MPI at run-time -- without the need to recompile or relink user MPI
applications. This provides great flexibility for both developers of MPI
software as well as researchers investigating MPI performance. This modual
framework is called the System Services Interface (SSI), and is responsible
for much of the back-end functionality in LAM/MPI.
Another strength of this MPI implementation is the movement of non-blocking
communication requests, including those that result from buffered sends. This
is the real challenge of implementing MPI; everything else is mostly a
straight forward wrapping of an underlying communication mechanism. LAM allows
messages to be buffered on the source end in any state of progression,
including partially transmitted packets. This capability leads to great
portability and robustness.
Users are strongly encouraged to read the
LAM/MPI User's Guide that is
included with the LAM/MPI distribution, and is provided on the main LAM/MPI
web site (
http://www.lam-mpi.org/).
The LAM home page can be found on the World Wide Web at:
http://www.lam-mpi.org/. It should be consulted for the most current
information about LAM, as well as updates, patches, etc.
MPI Communication¶
The sophisticated message advancing engine at the heart of the MPI library uses
only a handful of routines to drive the underlying communication system.
Runtime flags decide which message passing engine module is used, so
recompilation of user programs is not necessary.
The different message passing engines are commonly referred to as "Request
Progression Interface" (RPI) modules. The LAM/MPI distribution includes
multiple RPI modules; see the
lamssi_rpi(7) man page for more details.
One notable module uses LAM's network message-passing subsystem, including its
buffer daemon. In this "daemon" mode, LAM's extensive monitoring
features are fully available. Although the "daemon" mode typically
incurrs higher latency than the "native" RPI modules, applications
that can utilize latency-hiding techniques may experience greater performance
due to the daemon-mode's ability to exhibit true asynchronous message passing.
Guaranteed Envelope Resources¶
Applications may fail, legitimately, on some implementations but not others due
to an escape hatch in the MPI Standard called "resource
limitations". Most resources are managed locally and it is easy for an
implementation to provide a helpful error code and/or message when a resource
is exhausted. Buffer space for message envelopes is often a remote resource
(as in LAM) which is difficult to manage. An overflow may not be reported (as
in some other implementations) to the process that caused the overflow.
Moreover, interpretation of the MPI guarantee on message progress may confuse
the debugging of an application that actually died on envelope overflow.
LAM has a property called "Guaranteed Envelope Resources" (GER) which
serves two purposes. It is a promise from the implementation to the
application that a minimum amount of envelope buffering will be available to
each process pair. Secondly, it ensures that the producer of messages that
overflows this resource will be throttled or cited with an error as necessary.
A minimum GER is configured when LAM is built. The MPI library uses a protocol
to ensure GER when running in daemon mode. The default C2C mode (TCP/IP) does
not use a protocol, because process-pair protection is provided by TCP/IP
itself. Errors are only reported to the receiving process in C2C mode. An
option to mpirun(1) disables GER.
The MPI standard does not specify standard I/O functionality. LAM does not
interfere with the I/O capabilities of the underlying system but it does make
special provisions for remote terminal I/O using the ANSI/POSIX routines. See
mpirun(1) and tstdio(3).
LAM now includes the ROMIO distribution for MPI-2 file input and output. If
ROMIO support is compiled into LAM, the functionality from Chapter 9 of the
MPI-2 standard is provided.
ROMIO has some important limitations under LAM; the User's Guide in the LAM
distribution should be consulted before writing MPI programs that use MPI I/O.
Dynamic Processes¶
LAM includes an implementation of MPI-2 dynamic process creation.
LAM Extensions to MPI¶
Debugging Aids¶
LAM includes the MPI-2 functionality for naming opaque types. Support for the
Etnus TotalView parallel debugger is also provided; see the User's Guide for
more details.
Additionally, LAM provides the capability to launch non-MPI programs on remote
nodes. This includes shell scripts, debuggers, etc. As long as an MPI program
is eventually launched (as a child, grandchild, etc.), LAM can handle
executing as many intermediate programs as necessary. This can greatly help
debugging and logging of user programs.
Trace Generation¶
To avoid being swamped with trace data from a long running application, LAM
supplies collective operations to turn the tap on and off. See
MPIL_Trace_on(2) and MPIL_Trace_off(2).
Asynchronous Signals¶
LAM has an signal handling package which mirrors but does not interfere with
POSIX signal handling. An MPI extension routine delivers a signal to a
process. See MPIL_Signal(2).
SEE ALSO¶
Overview of Commands and Libraries¶
introu(1), introc(2), INTROF(2)
System Services Interface (SSI)¶
lamssi(7),
lamssi_boot(7),
lamssi_coll(7),
lamssi_rpi(7)
Starting / Stopping LAM¶
recon(1),
lamboot(1),
lamhalt(1),
lamnodes(1),
lamwipe(1),
tping(1),
lamgrow(1),
lamshrink(1)
Compiling MPI Applications¶
mpicc(1), mpiCC(1), mpif77(1)
Running MPI Applications¶
mpirun(1),
lamclean(1)
Running Non-MPI Applications¶
lamexec(1)
Monitoring MPI Applications¶
mpitask(1)
Unloading MPI Trace Data¶
lamtrace(1)
Reference Documents¶
- "LAM/MPI Installation Guide"
-
included in the LAM/MPI distribution and available on
http://www.lam-mpi.org/
- "LAM/MPI User's Guide"
- included in the LAM/MPI distribution and available on
http://www.lam-mpi.org/
- "LAM Frequently Asked Questions"
- at http://www.lam-mpi.org/faq/
- "MPI Primer / Developing with LAM", Ohio Supercomputer
Center
- "MPI: A Message-Passing Interface Standard", Message-Passing
Interface Forum, version 1.1
- at http://www.mpi-forum.org/
- "MPI-2: Extensions to the Message Passing Interface", Message
Passing Interface Forum, version 2.0
- at http://www.mpi-forum.org/
MPI Quick Tutorials¶
- "LAM/MPI ND User Guide / Introduction"
- "MPI: It's Easy to Get Started"
- "MPI: Everyday Datatypes"
- "MPI: Everyday Collective Communication"
- at http://www.lam-mpi.org/mpi/tutorials/lam/
Guaranteed Envelope Resources¶
- "Robust MPI Message Delivery Through Guaranteed Resources",
- MPI Developer's Conference, 1995