MPI consists of a library of routines with Fortran, C, and C++ bindings to facilitate the coding of parallel codes in a distributed memory MIMD environment. In addition it provides a run-time environment to build and execute the parallel codes. It comes with implementations on most popular platforms, including the beowulf type systems.
MPICH is a full implementation of MPI in a platform independent way. The MPICH implementation of MPI is what we will use in this course.
MPI maintains internal data-structures related to the administrations of the parallel tasks and to allow internal communications between the tasks and the environment. The latter are referred to as handles.
The C MPI routines return an int and the Fortran
routines return an integer ierror in the calls which
contain the error status of the calls. Error detection and debugging
of parallel codes are difficult in general, and MPI can only attempt
to detect errors.
All MPI routines are labeled MPI_Name, with the first letter of the name capitalized.
The C calls syntax is
int MPI_Name( <argument list> )
with an error code returned as an int upon execution.
The header file is mpi.h. The MPI provided constants are
all capitalized, i.e., MPI_COMM_WORLD.
Fortran calls follow the syntax
call MPI_Name ( <argument list>, ierror )
with the error code being an integer type. The header file is
mpif.h. The MPI provided constants are all captialized,
i.e., MPI_COMM_WORLD. However, note that Fortran is case
insensitive, and that consequently the use of capital letters is for
clarity only.
There are also a number of Python wrappers for MPI, such as mpi4py, as well as wrappers for many other languages. For more related Python packages, see Python's parallel processing wiki page.
int MPI_Init( int *argc, char ***argv)
Initializes the executables on the nodes to the MPI handling
daemons. It creates a communicator MPI_COMM_WORLD which
encompasses all the initialized processes on the various nodes and
give a unique identification to each process, the rank number. The
rank and size of the communicator can be obtained via MPI query
routines.
int MPI_Comm_size( MPI_Comm comm, int *size) int MPI_Comm_rank( MPI_Comm comm, int *rank)
Returns respectively the size of the specified communicator and the
rank (process number) of the current process within the specified
communicator. The rank number goes between 0
and size-1.
int MPI_Finalize()
Must be the last call to MPI in any MPI code. No other call to MPI
routines can be executed after it. Note that a new communicator can
not be invoked after MPI_finalize().
mpicc <name.c> -o <name>
Compiles a C code name.c, produces the object under
name.o, and links it with the MPI
library. The -o option is to specify the executable name,
the default being a.out. This command calls the default C
compiler, usualy the GNU gcc compiler on linux based
computers.
mpif77 <name.f> -o <name>
Compiles a Fortran code name.f, produces the object
under name.o, and links it with the MPI
library. The -o option is to specify the executable name,
the default being a.out. This command calls the default
Fortran compiler, usualy the GNU f77 compiler on linux based
computers.
MPI2 provides the mpd subset of commands to build a
virtual parallel machine, a communication ring. These were described
in the previous section on Communication
Ring under MPI2.
Once a communication ring has been established, the command
mpiexec launches an executable on a specified number of
nodes.
mpiexec -np <number-of-processes> <executable>
will start the executable code executable on
<number-of-processes> nodes.
MPI2 will choose the nodes automatically, starting from the current
one (launching node) and selecting from the list of available
processors in the communication
ring. If <number-of-processes> is larger than the
number of nodes included in the communication ring, the executable
will be launched more than once in some of the nodes chosen in a
cyclic way.