Contents
This page decribes the features of the last version of SCI-MPICH. The feature-list is split into "usual stuff"
(features that any serious MPI implementation has) and "goodies", which are features not commonly
found in MPI implementations.
Please note that some features are not special to SCI-MPICH, but are a features of MP-MPICH, the superset of SCI-MPICH.
Goodies
Available now:
- Low latency: The SCI interconnect allows inter-node latencies of 5us or less, although
the software architecture of SCI-MPICH is still flexible and not reduced-to-the-limit.
- High Bandwidth: The transport protocols of SCI-MPICH deliver more than 95% of the
peak-bandwidth of the SCI interconnect .
- Polling or blocking message receiption: SCI-MPICH can poll for new messages or
can block (using no CPU cycles), waiting for a remote interrupt.
- Asynchronous progress: Messages can be transfered "in the background" with MPI's
asynchronous communication calls, allowing for true overlap of computation and communication
- DMA and PIO transfer: SCI-MPICH uses both, PIO data transfers (performed by the CPU) and
DMA data transfers (performed by the PCI-SCI adapter), whatever delivers more performance.
- Optimized collective operations: MP-MPICH has an interface to let a communication device
provide its own collective operations. SCI-MPICH has implemented some of them for best efficiency with
the SCI interconnect.
- Fully configurable: All relevant aspects of SCI-MPICH's setup and protocol configuration can
be configured dynamically on startup, allowing the user to optimize the configuration to the
communication behaviour of the application.
- Adaptive resource usage: SCI-MPICH tries to cope with any (temporary) resource
shortage of the SCI interconnect to ensure the proper continuation of the application.
- Combined shared-memory/message-passing programming possible: Sacrifying MPI portability,
SCI-MPICH allows mixing message-passing and shared-memory programming on an SCI-Cluster, using the
Shared Memory Interface (SMI) library.
- Advanced design: SCI-MPICH is based on the well-known MPICH-distribution, but has undergone
many internal changes to improve the functionality and extensability (pluggable devices as shared
libraries, enhanced device interface, ready for multi-device usage, ...)
- Full MPICH compatibility: Although SCI-MPICH has many internal changes, it remains fully
compatible with the widespread MPICH distribution, concerning include files (object file compatibility),
"look and feel" (handling) and third-party tools like Vampir.
- One-sided operations: The complete one-sided operations as defined by the MPI-2 standard are
supported, for any type of memory. Very efficient for SCI shared memory!
- Zero-copy transfers: Efficient zero-copy transfers via DMA (requires special SCI driver)
- Optimized persistent communication: Together with zero-copy transfers, using persistent
communication will improve performance even more.
- Efficient communication with custom datatypes: The efficiency of communication with
non-contiguous datatype is (relatively) higher than for most other MPI implemenations, due to
a unique pack-on-the-fly technique.
Usual Stuff
- Full MPI-1: SCI-MPICH implements the full MPI-1 (1.2) standard, incl. MPI_Cancel.
- Parts of MPI-2: MPI-IO is available via ROMIO, with an ADIO device for NTFS.
- Open Source: SCI-MPICH is open-source software, freely available for private, academic
and commercial use. Only the use of SCI-MPICH in commercial products requires a license.
- Portability: SCI-MPICH runs on Solaris, Linux and Win32 (NT/2000/XP) on Intel/AMD x86/x64 and Sun Sparc CPUs.
- SMP support: Efficient intra-node communication on SMPs is supported as well (the stuff that
others describe as "automatically chooses the fastest communication path").
Limitations
SCI-MPICH is developed as part of a research project. This does not mean that it cannot be used on production systems. But it currently has some limitations which we describe below. If you can't already see from the list below that SCI-MPICH is not for you, you might try it out and see if it fits your needs!
- Number of processes: Because of limitations of the underlying SMI library, which was originally developed on small-scale clusters, SCI-MPICH doesn't run stable with more than 16 processes. This is of course a serious limitation and as our clusters now allow us to test configurations with more processes, we strive to remove this barrier in the future.
- MPI-2: The basis for SCI-MPICH is MPICH, which implements the MPI-1.2 standard. Additionally, support for One-Sided Communications was built into SCI-MPICH. Dynamic Process creation and management is currently unsupported.
- Thread safety: Also part of its MPICH-heritage is the lack of Thread-Safety in SCI-MPICH. Unfortunately, this prevents the increasingly popular "Hybrid Parallelisation" (OpenMP + MPI) from being used with SCI-MPICH.
|
