FG-MPI : Fine-Grain MPI
The Message Passing Interface (MPI) is widely used to write sophisticated parallel applications ranging from cognitive computing to weather predictions and is almost universally adopted for High Performance Computing (HPC). Many popular MPI implementations bind MPI processes to OS-processes. This ru...
Main Author: | |
---|---|
Language: | English |
Published: |
University of British Columbia
2013
|
Online Access: | http://hdl.handle.net/2429/44668 |
id |
ndltd-LACETR-oai-collectionscanada.gc.ca-BVAU.-44668 |
---|---|
record_format |
oai_dc |
spelling |
ndltd-LACETR-oai-collectionscanada.gc.ca-BVAU.-446682013-07-25T03:15:17ZFG-MPI : Fine-Grain MPIKamal, HumairaThe Message Passing Interface (MPI) is widely used to write sophisticated parallel applications ranging from cognitive computing to weather predictions and is almost universally adopted for High Performance Computing (HPC). Many popular MPI implementations bind MPI processes to OS-processes. This runtime model has closely matched single or multi-processor compute clusters. Since 2008, however, clusters of multicore nodes have been the predominant architecture for HPC, with the opportunity for parallelism inside one compute node. There are a number of popular parallel programming languages for multicore that use message passing. One notable difference between MPI and these languages is the granularity of the MPI processes. Processes written using MPI tend to be coarse-grained and designed to match the number of processes to the available hardware, rather than the program structure. Binding MPI processes to OS-processes fails to take full advantage of the finer-grain parallelism available on today's multicore systems. Our goal was to take advantage of the type of runtime systems used by fine-grain languages and integrate that into MPI to obtain the best of these programming models; the ability to have fine-grain parallelism, while maintaining MPI's rich support for communication inside clusters. Fine-Grain MPI (FG-MPI) is a system that extends the execution model of MPI to include interleaved concurrency through integration into the MPI middleware. FG-MPI is integrated into the MPICH2 middleware, which is an open source, production-quality implementation of MPI. The FG-MPI runtime uses coroutines to implement light-weight MPI processes that are non-preemptively scheduled by its MPI-aware scheduler. The use of coroutines enables fast context-switching time and low communication and synchronization overhead. FG-MPI enables expression of finer-grain function-level parallelism, which allows for flexible process mapping, scalability, and can lead to better program performance. We have demonstrated FG-MPI's ability to scale to over a 100 million MPI processes on a large cluster of 6,480 cores. This is the first time any system has executed such a large number of MPI processes, and this capability will be useful in exploring scalability issues of the MPI middleware as systems move towards compute clusters with millions of processor cores.University of British Columbia2013-07-18T13:36:59Z2013-07-19T09:15:25Z20132013-07-182013-11Electronic Thesis or Dissertationhttp://hdl.handle.net/2429/44668eng |
collection |
NDLTD |
language |
English |
sources |
NDLTD |
description |
The Message Passing Interface (MPI) is widely used to write sophisticated parallel applications
ranging from cognitive computing to weather predictions and is almost universally
adopted for High Performance Computing (HPC).
Many popular MPI implementations bind MPI processes to OS-processes. This runtime
model has closely matched single or multi-processor compute clusters. Since 2008, however,
clusters of multicore nodes have been the predominant architecture for HPC, with the
opportunity for parallelism inside one compute node. There are a number of popular parallel
programming languages for multicore that use message passing. One notable difference
between MPI and these languages is the granularity of the MPI processes. Processes written
using MPI tend to be coarse-grained and designed to match the number of processes to the
available hardware, rather than the program structure. Binding MPI processes to OS-processes
fails to take full advantage of the finer-grain parallelism available on today's
multicore systems. Our goal was to take advantage of the type of runtime systems used by
fine-grain languages and integrate that into MPI to obtain the best of these programming
models; the ability to have fine-grain parallelism, while maintaining MPI's rich support for
communication inside clusters.
Fine-Grain MPI (FG-MPI) is a system that extends the execution model of MPI to
include interleaved concurrency through integration into the MPI middleware. FG-MPI
is integrated into the MPICH2 middleware, which is an open source, production-quality
implementation of MPI. The FG-MPI runtime uses coroutines to implement light-weight MPI processes that are non-preemptively scheduled by its MPI-aware scheduler. The use of
coroutines enables fast context-switching time and low communication and synchronization
overhead. FG-MPI enables expression of finer-grain function-level parallelism, which allows
for flexible process mapping, scalability, and can lead to better program performance.
We have demonstrated FG-MPI's ability to scale to over a 100 million MPI processes
on a large cluster of 6,480 cores. This is the first time any system has executed such a large
number of MPI processes, and this capability will be useful in exploring scalability issues of
the MPI middleware as systems move towards compute clusters with millions of processor
cores. |
author |
Kamal, Humaira |
spellingShingle |
Kamal, Humaira FG-MPI : Fine-Grain MPI |
author_facet |
Kamal, Humaira |
author_sort |
Kamal, Humaira |
title |
FG-MPI : Fine-Grain MPI |
title_short |
FG-MPI : Fine-Grain MPI |
title_full |
FG-MPI : Fine-Grain MPI |
title_fullStr |
FG-MPI : Fine-Grain MPI |
title_full_unstemmed |
FG-MPI : Fine-Grain MPI |
title_sort |
fg-mpi : fine-grain mpi |
publisher |
University of British Columbia |
publishDate |
2013 |
url |
http://hdl.handle.net/2429/44668 |
work_keys_str_mv |
AT kamalhumaira fgmpifinegrainmpi |
_version_ |
1716594440118730752 |