1201 lines
138 KiB
HTML
1201 lines
138 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
|
||
<html>
|
||
<head>
|
||
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
|
||
<title>Tutorial</title>
|
||
<link rel="stylesheet" href="../../../doc/src/boostbook.css" type="text/css">
|
||
<meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
|
||
<link rel="home" href="../index.html" title="The Boost C++ Libraries BoostBook Documentation Subset">
|
||
<link rel="up" href="../mpi.html" title="Chapter 26. Boost.MPI">
|
||
<link rel="prev" href="getting_started.html" title="Getting started">
|
||
<link rel="next" href="c_mapping.html" title="Mapping from C MPI to Boost.MPI">
|
||
</head>
|
||
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
|
||
<table cellpadding="2" width="100%"><tr>
|
||
<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../boost.png"></td>
|
||
<td align="center"><a href="../../../index.html">Home</a></td>
|
||
<td align="center"><a href="../../../libs/libraries.htm">Libraries</a></td>
|
||
<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
|
||
<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
|
||
<td align="center"><a href="../../../more/index.htm">More</a></td>
|
||
</tr></table>
|
||
<hr>
|
||
<div class="spirit-nav">
|
||
<a accesskey="p" href="getting_started.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../mpi.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="c_mapping.html"><img src="../../../doc/src/images/next.png" alt="Next"></a>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
|
||
<a name="mpi.tutorial"></a><a class="link" href="tutorial.html" title="Tutorial">Tutorial</a>
|
||
</h2></div></div></div>
|
||
<div class="toc"><dl class="toc">
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.point_to_point">Point-to-Point communication</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives">Collective operations</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.user_data_types">User-defined data types</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.communicators">Communicators</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.threading">Threads</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.skeleton_and_content">Separating structure
|
||
from content</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.performance_optimizations">Performance optimizations</a></span></dt>
|
||
</dl></div>
|
||
<p>
|
||
A Boost.MPI program consists of many cooperating processes (possibly running
|
||
on different computers) that communicate among themselves by passing messages.
|
||
Boost.MPI is a library (as is the lower-level MPI), not a language, so the
|
||
first step in a Boost.MPI is to create an <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code>
|
||
object that initializes the MPI environment and enables communication among
|
||
the processes. The <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code>
|
||
object is initialized with the program arguments (which it may modify) in your
|
||
main program. The creation of this object initializes MPI, and its destruction
|
||
will finalize MPI. In the vast majority of Boost.MPI programs, an instance
|
||
of <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code> will
|
||
be declared in <code class="computeroutput"><span class="identifier">main</span></code> at the
|
||
very beginning of the program.
|
||
</p>
|
||
<div class="warning"><table border="0" summary="Warning">
|
||
<tr>
|
||
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Warning]" src="../../../doc/src/images/warning.png"></td>
|
||
<th align="left">Warning</th>
|
||
</tr>
|
||
<tr><td align="left" valign="top"><p>
|
||
Declaring an <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code>
|
||
at global scope is undefined behavior. <a href="#ftn.mpi.tutorial.f0" class="footnote" name="mpi.tutorial.f0"><sup class="footnote">[11]</sup></a>
|
||
</p></td></tr>
|
||
</table></div>
|
||
<p>
|
||
Communication with MPI always occurs over a <span class="bold"><strong>communicator</strong></span>,
|
||
which can be created by simply default-constructing an object of type <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">mpi::communicator</a></code>. This communicator
|
||
can then be queried to determine how many processes are running (the "size"
|
||
of the communicator) and to give a unique number to each process, from zero
|
||
to the size of the communicator (i.e., the "rank" of the process):
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"I am process "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><<</span> <span class="string">" of "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span>
|
||
<span class="special"><<</span> <span class="string">"."</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
If you run this program with 7 processes, for instance, you will receive output
|
||
such as:
|
||
</p>
|
||
<pre class="programlisting">I am process 5 of 7.
|
||
I am process 0 of 7.
|
||
I am process 1 of 7.
|
||
I am process 6 of 7.
|
||
I am process 2 of 7.
|
||
I am process 4 of 7.
|
||
I am process 3 of 7.
|
||
</pre>
|
||
<p>
|
||
Of course, the processes can execute in a different order each time, so the
|
||
ranks might not be strictly increasing. More interestingly, the text could
|
||
come out completely garbled, because one process can start writing "I
|
||
am a process" before another process has finished writing "of 7.".
|
||
</p>
|
||
<p>
|
||
If you should still have an MPI library supporting only MPI 1.1 you will need
|
||
to pass the command line arguments to the environment constructor as shown
|
||
in this example:
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"I am process "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><<</span> <span class="string">" of "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span>
|
||
<span class="special"><<</span> <span class="string">"."</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h3 class="title">
|
||
<a name="mpi.tutorial.point_to_point"></a><a class="link" href="tutorial.html#mpi.tutorial.point_to_point" title="Point-to-Point communication">Point-to-Point communication</a>
|
||
</h3></div></div></div>
|
||
<div class="toc"><dl class="toc">
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.point_to_point.blocking">Blocking communication</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.point_to_point.nonblocking">Non-blocking
|
||
communication</a></span></dt>
|
||
</dl></div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h4 class="title">
|
||
<a name="mpi.tutorial.point_to_point.blocking"></a><a class="link" href="tutorial.html#mpi.tutorial.point_to_point.blocking" title="Blocking communication">Blocking communication</a>
|
||
</h4></div></div></div>
|
||
<p>
|
||
As a message passing library, MPI's primary purpose is to routine messages
|
||
from one process to another, i.e., point-to-point. MPI contains routines
|
||
that can send messages, receive messages, and query whether messages are
|
||
available. Each message has a source process, a target process, a tag,
|
||
and a payload containing arbitrary data. The source and target processes
|
||
are the ranks of the sender and receiver of the message, respectively.
|
||
Tags are integers that allow the receiver to distinguish between different
|
||
messages coming from the same sender.
|
||
</p>
|
||
<p>
|
||
The following program uses two MPI processes to write "Hello, world!"
|
||
to the screen (<code class="computeroutput"><span class="identifier">hello_world</span><span class="special">.</span><span class="identifier">cpp</span></code>):
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">string</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"Hello"</span><span class="special">));</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">"!"</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">", "</span><span class="special">;</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span><span class="special">.</span><span class="identifier">flush</span><span class="special">();</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"world"</span><span class="special">));</span>
|
||
<span class="special">}</span>
|
||
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
The first processor (rank 0) passes the message "Hello" to the
|
||
second processor (rank 1) using tag 0. The second processor prints the
|
||
string it receives, along with a comma, then passes the message "world"
|
||
back to processor 0 with a different tag. The first processor then writes
|
||
this message with the "!" and exits. All sends are accomplished
|
||
with the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_4-bb">communicator::send</a></code>
|
||
method and all receives use a corresponding <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_9-bb">communicator::recv</a></code>
|
||
call.
|
||
</p>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h4 class="title">
|
||
<a name="mpi.tutorial.point_to_point.nonblocking"></a><a class="link" href="tutorial.html#mpi.tutorial.point_to_point.nonblocking" title="Non-blocking communication">Non-blocking
|
||
communication</a>
|
||
</h4></div></div></div>
|
||
<p>
|
||
The default MPI communication operations--<code class="computeroutput"><span class="identifier">send</span></code>
|
||
and <code class="computeroutput"><span class="identifier">recv</span></code>--may have to wait
|
||
until the entire transmission is completed before they can return. Sometimes
|
||
this <span class="bold"><strong>blocking</strong></span> behavior has a negative
|
||
impact on performance, because the sender could be performing useful computation
|
||
while it is waiting for the transmission to occur. More important, however,
|
||
are the cases where several communication operations must occur simultaneously,
|
||
e.g., a process will both send and receive at the same time.
|
||
</p>
|
||
<p>
|
||
Let's revisit our "Hello, world!" program from the previous
|
||
<a class="link" href="tutorial.html#mpi.tutorial.point_to_point.blocking" title="Blocking communication">section</a>. The
|
||
core of this program transmits two messages:
|
||
</p>
|
||
<pre class="programlisting"><span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"Hello"</span><span class="special">));</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">"!"</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">", "</span><span class="special">;</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span><span class="special">.</span><span class="identifier">flush</span><span class="special">();</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"world"</span><span class="special">));</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
The first process passes a message to the second process, then prepares
|
||
to receive a message. The second process does the send and receive in the
|
||
opposite order. However, this sequence of events is just that--a <span class="bold"><strong>sequence</strong></span>--meaning that there is essentially no parallelism.
|
||
We can use non-blocking communication to ensure that the two messages are
|
||
transmitted simultaneously (<code class="computeroutput"><span class="identifier">hello_world_nonblocking</span><span class="special">.</span><span class="identifier">cpp</span></code>):
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">string</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">request</span> <span class="identifier">reqs</span><span class="special">[</span><span class="number">2</span><span class="special">];</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">,</span> <span class="identifier">out_msg</span> <span class="special">=</span> <span class="string">"Hello"</span><span class="special">;</span>
|
||
<span class="identifier">reqs</span><span class="special">[</span><span class="number">0</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">isend</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">out_msg</span><span class="special">);</span>
|
||
<span class="identifier">reqs</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">irecv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">wait_all</span><span class="special">(</span><span class="identifier">reqs</span><span class="special">,</span> <span class="identifier">reqs</span> <span class="special">+</span> <span class="number">2</span><span class="special">);</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">"!"</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">request</span> <span class="identifier">reqs</span><span class="special">[</span><span class="number">2</span><span class="special">];</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">,</span> <span class="identifier">out_msg</span> <span class="special">=</span> <span class="string">"world"</span><span class="special">;</span>
|
||
<span class="identifier">reqs</span><span class="special">[</span><span class="number">0</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">isend</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">out_msg</span><span class="special">);</span>
|
||
<span class="identifier">reqs</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">irecv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">wait_all</span><span class="special">(</span><span class="identifier">reqs</span><span class="special">,</span> <span class="identifier">reqs</span> <span class="special">+</span> <span class="number">2</span><span class="special">);</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">", "</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
We have replaced calls to the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_4-bb">communicator::send</a></code>
|
||
and <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_9-bb">communicator::recv</a></code>
|
||
members with similar calls to their non-blocking counterparts, <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_16-bb">communicator::isend</a></code>
|
||
and <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_21-bb">communicator::irecv</a></code>.
|
||
The prefix <span class="bold"><strong>i</strong></span> indicates that the operations
|
||
return immediately with a <code class="computeroutput"><a class="link" href="../boost/mpi/request.html" title="Class request">mpi::request</a></code>
|
||
object, which allows one to query the status of a communication request
|
||
(see the <code class="computeroutput"><a class="link" href="../boost/mpi/request.html#id-1_3_27_7_21_2_1_1_7_2-bb">test</a></code>
|
||
method) or wait until it has completed (see the <code class="computeroutput"><a class="link" href="../boost/mpi/request.html#id-1_3_27_7_21_2_1_1_7_1-bb">wait</a></code>
|
||
method). Multiple requests can be completed at the same time with the
|
||
<code class="computeroutput"><a class="link" href="../boost/mpi/wait_all.html" title="Function wait_all">wait_all</a></code> operation.
|
||
</p>
|
||
<div class="important"><table border="0" summary="Important">
|
||
<tr>
|
||
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Important]" src="../../../doc/src/images/important.png"></td>
|
||
<th align="left">Important</th>
|
||
</tr>
|
||
<tr><td align="left" valign="top"><p>
|
||
Regarding communication completion/progress: The MPI standard requires
|
||
users to keep the request handle for a non-blocking communication, and
|
||
to call the "wait" operation (or successfully test for completion)
|
||
to complete the send or receive. Unlike most C MPI implementations, which
|
||
allow the user to discard the request for a non-blocking send, Boost.MPI
|
||
requires the user to call "wait" or "test", since
|
||
the request object might contain temporary buffers that have to be kept
|
||
until the send is completed. Moreover, the MPI standard does not guarantee
|
||
that the receive makes any progress before a call to "wait"
|
||
or "test", although most implementations of the C MPI do allow
|
||
receives to progress before the call to "wait" or "test".
|
||
Boost.MPI, on the other hand, generally requires "test" or
|
||
"wait" calls to make progress. More specifically, Boost.MPI
|
||
guarantee that calling "test" multiple time will eventually
|
||
complete the communication (this is due to the fact that serialized communication
|
||
are potentially a multi step operation.).
|
||
</p></td></tr>
|
||
</table></div>
|
||
<p>
|
||
If you run this program multiple times, you may see some strange results:
|
||
namely, some runs will produce:
|
||
</p>
|
||
<pre class="programlisting"><span class="identifier">Hello</span><span class="special">,</span> <span class="identifier">world</span><span class="special">!</span>
|
||
</pre>
|
||
<p>
|
||
while others will produce:
|
||
</p>
|
||
<pre class="programlisting"><span class="identifier">world</span><span class="special">!</span>
|
||
<span class="identifier">Hello</span><span class="special">,</span>
|
||
</pre>
|
||
<p>
|
||
or even some garbled version of the letters in "Hello" and "world".
|
||
This indicates that there is some parallelism in the program, because after
|
||
both messages are (simultaneously) transmitted, both processes will concurrent
|
||
execute their print statements. For both performance and correctness, non-blocking
|
||
communication operations are critical to many parallel applications using
|
||
MPI.
|
||
</p>
|
||
</div>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h3 class="title">
|
||
<a name="mpi.tutorial.collectives"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives" title="Collective operations">Collective operations</a>
|
||
</h3></div></div></div>
|
||
<div class="toc"><dl class="toc">
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.broadcast">Broadcast</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.gather">Gather</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.scatter">Scatter</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.reduce">Reduce</a></span></dt>
|
||
</dl></div>
|
||
<p>
|
||
<a class="link" href="tutorial.html#mpi.tutorial.point_to_point" title="Point-to-Point communication">Point-to-point operations</a>
|
||
are the core message passing primitives in Boost.MPI. However, many message-passing
|
||
applications also require higher-level communication algorithms that combine
|
||
or summarize the data stored on many different processes. These algorithms
|
||
support many common tasks such as "broadcast this value to all processes",
|
||
"compute the sum of the values on all processors" or "find
|
||
the global minimum."
|
||
</p>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h4 class="title">
|
||
<a name="mpi.tutorial.collectives.broadcast"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.broadcast" title="Broadcast">Broadcast</a>
|
||
</h4></div></div></div>
|
||
<p>
|
||
The <code class="computeroutput"><a class="link" href="../boost/mpi/broadcast.html" title="Function broadcast">broadcast</a></code>
|
||
algorithm is by far the simplest collective operation. It broadcasts a
|
||
value from a single process to all other processes within a <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">communicator</a></code>. For instance,
|
||
the following program broadcasts "Hello, World!" from process
|
||
0 to every other process. (<code class="computeroutput"><span class="identifier">hello_world_broadcast</span><span class="special">.</span><span class="identifier">cpp</span></code>)
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">string</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">value</span><span class="special">;</span>
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">value</span> <span class="special">=</span> <span class="string">"Hello, World!"</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
|
||
<span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">value</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
|
||
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"Process #"</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><<</span> <span class="string">" says "</span> <span class="special"><<</span> <span class="identifier">value</span>
|
||
<span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
Running this program with seven processes will produce a result such as:
|
||
</p>
|
||
<pre class="programlisting">Process #0 says Hello, World!
|
||
Process #2 says Hello, World!
|
||
Process #1 says Hello, World!
|
||
Process #4 says Hello, World!
|
||
Process #3 says Hello, World!
|
||
Process #5 says Hello, World!
|
||
Process #6 says Hello, World!
|
||
</pre>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h4 class="title">
|
||
<a name="mpi.tutorial.collectives.gather"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.gather" title="Gather">Gather</a>
|
||
</h4></div></div></div>
|
||
<p>
|
||
The <code class="computeroutput"><a class="link" href="../boost/mpi/gather.html" title="Function gather">gather</a></code>
|
||
collective gathers the values produced by every process in a communicator
|
||
into a vector of values on the "root" process (specified by an
|
||
argument to <code class="computeroutput"><span class="identifier">gather</span></code>). The
|
||
/i/th element in the vector will correspond to the value gathered from
|
||
the /i/th process. For instance, in the following program each process
|
||
computes its own random number. All of these random numbers are gathered
|
||
at process 0 (the "root" in this case), which prints out the
|
||
values that correspond to each processor. (<code class="computeroutput"><span class="identifier">random_gather</span><span class="special">.</span><span class="identifier">cpp</span></code>)
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">vector</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">time</span><span class="special">(</span><span class="number">0</span><span class="special">)</span> <span class="special">+</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span>
|
||
<span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span>
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">all_numbers</span><span class="special">;</span>
|
||
<span class="identifier">gather</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">all_numbers</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
|
||
<span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">proc</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">proc</span> <span class="special"><</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">proc</span><span class="special">)</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"Process #"</span> <span class="special"><<</span> <span class="identifier">proc</span> <span class="special"><<</span> <span class="string">" thought of "</span>
|
||
<span class="special"><<</span> <span class="identifier">all_numbers</span><span class="special">[</span><span class="identifier">proc</span><span class="special">]</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span>
|
||
<span class="identifier">gather</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
|
||
<span class="special">}</span>
|
||
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
Executing this program with seven processes will result in output such
|
||
as the following. Although the random values will change from one run to
|
||
the next, the order of the processes in the output will remain the same
|
||
because only process 0 writes to <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span></code>.
|
||
</p>
|
||
<pre class="programlisting">Process #0 thought of 332199874
|
||
Process #1 thought of 20145617
|
||
Process #2 thought of 1862420122
|
||
Process #3 thought of 480422940
|
||
Process #4 thought of 1253380219
|
||
Process #5 thought of 949458815
|
||
Process #6 thought of 650073868
|
||
</pre>
|
||
<p>
|
||
The <code class="computeroutput"><span class="identifier">gather</span></code> operation collects
|
||
values from every process into a vector at one process. If instead the
|
||
values from every process need to be collected into identical vectors on
|
||
every process, use the <code class="computeroutput"><a class="link" href="../boost/mpi/all_gather.html" title="Function all_gather">all_gather</a></code> algorithm,
|
||
which is semantically equivalent to calling <code class="computeroutput"><span class="identifier">gather</span></code>
|
||
followed by a <code class="computeroutput"><span class="identifier">broadcast</span></code>
|
||
of the resulting vector.
|
||
</p>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h4 class="title">
|
||
<a name="mpi.tutorial.collectives.scatter"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.scatter" title="Scatter">Scatter</a>
|
||
</h4></div></div></div>
|
||
<p>
|
||
The <code class="computeroutput"><a class="link" href="../boost/mpi/scatter.html" title="Function scatter">scatter</a></code>
|
||
collective scatters the values from a vector in the "root" process
|
||
in a communicator into values in all the processes of the communicator.
|
||
The /i/th element in the vector will correspond to the value received by
|
||
the /i/th process. For instance, in the following program, the root process
|
||
produces a vector of random nomber and send one value to each process that
|
||
will print it. (<code class="computeroutput"><span class="identifier">random_scatter</span><span class="special">.</span><span class="identifier">cpp</span></code>)
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">collectives</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">vector</span><span class="special">></span>
|
||
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">time</span><span class="special">(</span><span class="number">0</span><span class="special">)</span> <span class="special">+</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">all</span><span class="special">;</span>
|
||
<span class="keyword">int</span> <span class="identifier">mine</span> <span class="special">=</span> <span class="special">-</span><span class="number">1</span><span class="special">;</span>
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">all</span><span class="special">.</span><span class="identifier">resize</span><span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">());</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">generate</span><span class="special">(</span><span class="identifier">all</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">all</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">);</span>
|
||
<span class="special">}</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">scatter</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">all</span><span class="special">,</span> <span class="identifier">mine</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
|
||
<span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">r</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">r</span> <span class="special"><</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">r</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">barrier</span><span class="special">();</span>
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">r</span> <span class="special">==</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">())</span> <span class="special">{</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"Rank "</span> <span class="special"><<</span> <span class="identifier">r</span> <span class="special"><<</span> <span class="string">" got "</span> <span class="special"><<</span> <span class="identifier">mine</span> <span class="special"><<</span> <span class="char">'\n'</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
<span class="special">}</span>
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
Executing this program with seven processes will result in output such
|
||
as the following. Although the random values will change from one run to
|
||
the next, the order of the processes in the output will remain the same
|
||
because of the barrier.
|
||
</p>
|
||
<pre class="programlisting">Rank 0 got 1409381269
|
||
Rank 1 got 17045268
|
||
Rank 2 got 440120016
|
||
Rank 3 got 936998224
|
||
Rank 4 got 1827129182
|
||
Rank 5 got 1951746047
|
||
Rank 6 got 2117359639
|
||
</pre>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h4 class="title">
|
||
<a name="mpi.tutorial.collectives.reduce"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.reduce" title="Reduce">Reduce</a>
|
||
</h4></div></div></div>
|
||
<p>
|
||
The <code class="computeroutput"><a class="link" href="../boost/mpi/reduce.html" title="Function reduce">reduce</a></code>
|
||
collective summarizes the values from each process into a single value
|
||
at the user-specified "root" process. The Boost.MPI <code class="computeroutput"><span class="identifier">reduce</span></code> operation is similar in spirit
|
||
to the STL <a href="http://www.sgi.com/tech/stl/accumulate.html" target="_top"><code class="computeroutput"><span class="identifier">accumulate</span></code></a> operation, because
|
||
it takes a sequence of values (one per process) and combines them via a
|
||
function object. For instance, we can randomly generate values in each
|
||
process and the compute the minimum value over all processes via a call
|
||
to <code class="computeroutput"><a class="link" href="../boost/mpi/reduce.html" title="Function reduce">reduce</a></code>
|
||
(<code class="computeroutput"><span class="identifier">random_min</span><span class="special">.</span><span class="identifier">cpp</span></code>):
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">time</span><span class="special">(</span><span class="number">0</span><span class="special">)</span> <span class="special">+</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span>
|
||
<span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span>
|
||
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="keyword">int</span> <span class="identifier">minimum</span><span class="special">;</span>
|
||
<span class="identifier">reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">minimum</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">>(),</span> <span class="number">0</span><span class="special">);</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"The minimum value is "</span> <span class="special"><<</span> <span class="identifier">minimum</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span>
|
||
<span class="identifier">reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">>(),</span> <span class="number">0</span><span class="special">);</span>
|
||
<span class="special">}</span>
|
||
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
The use of <code class="computeroutput"><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span></code>
|
||
indicates that the minimum value should be computed. <code class="computeroutput"><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span></code> is a binary function object that compares
|
||
its two parameters via <code class="computeroutput"><span class="special"><</span></code>
|
||
and returns the smaller value. Any associative binary function or function
|
||
object will work provided it's stateless. For instance, to concatenate
|
||
strings with <code class="computeroutput"><span class="identifier">reduce</span></code> one
|
||
could use the function object <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">plus</span><span class="special"><</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">></span></code> (<code class="computeroutput"><span class="identifier">string_cat</span><span class="special">.</span><span class="identifier">cpp</span></code>):
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">string</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">functional</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">names</span><span class="special">[</span><span class="number">10</span><span class="special">]</span> <span class="special">=</span> <span class="special">{</span> <span class="string">"zero "</span><span class="special">,</span> <span class="string">"one "</span><span class="special">,</span> <span class="string">"two "</span><span class="special">,</span> <span class="string">"three "</span><span class="special">,</span>
|
||
<span class="string">"four "</span><span class="special">,</span> <span class="string">"five "</span><span class="special">,</span> <span class="string">"six "</span><span class="special">,</span> <span class="string">"seven "</span><span class="special">,</span>
|
||
<span class="string">"eight "</span><span class="special">,</span> <span class="string">"nine "</span> <span class="special">};</span>
|
||
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">result</span><span class="special">;</span>
|
||
<span class="identifier">reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><</span> <span class="number">10</span><span class="special">?</span> <span class="identifier">names</span><span class="special">[</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()]</span>
|
||
<span class="special">:</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"many "</span><span class="special">),</span>
|
||
<span class="identifier">result</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">plus</span><span class="special"><</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">>(),</span> <span class="number">0</span><span class="special">);</span>
|
||
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"The result is "</span> <span class="special"><<</span> <span class="identifier">result</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
In this example, we compute a string for each process and then perform
|
||
a reduction that concatenates all of the strings together into one, long
|
||
string. Executing this program with seven processors yields the following
|
||
output:
|
||
</p>
|
||
<pre class="programlisting">The result is zero one two three four five six
|
||
</pre>
|
||
<h5>
|
||
<a name="mpi.tutorial.collectives.reduce.h0"></a>
|
||
<span class="phrase"><a name="mpi.tutorial.collectives.reduce.binary_operations_for_reduce"></a></span><a class="link" href="tutorial.html#mpi.tutorial.collectives.reduce.binary_operations_for_reduce">Binary
|
||
operations for reduce</a>
|
||
</h5>
|
||
<p>
|
||
Any kind of binary function objects can be used with <code class="computeroutput"><span class="identifier">reduce</span></code>.
|
||
For instance, and there are many such function objects in the C++ standard
|
||
<code class="computeroutput"><span class="special"><</span><span class="identifier">functional</span><span class="special">></span></code> header and the Boost.MPI header <code class="computeroutput"><span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">operations</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span></code>. Or, you can create your own function
|
||
object. Function objects used with <code class="computeroutput"><span class="identifier">reduce</span></code>
|
||
must be associative, i.e. <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="identifier">x</span><span class="special">,</span>
|
||
<span class="identifier">f</span><span class="special">(</span><span class="identifier">y</span><span class="special">,</span> <span class="identifier">z</span><span class="special">))</span></code> must be equivalent to <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="identifier">f</span><span class="special">(</span><span class="identifier">x</span><span class="special">,</span> <span class="identifier">y</span><span class="special">),</span> <span class="identifier">z</span><span class="special">)</span></code>. If they are also commutative (i..e,
|
||
<code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="identifier">x</span><span class="special">,</span> <span class="identifier">y</span><span class="special">)</span> <span class="special">==</span> <span class="identifier">f</span><span class="special">(</span><span class="identifier">y</span><span class="special">,</span>
|
||
<span class="identifier">x</span><span class="special">)</span></code>),
|
||
Boost.MPI can use a more efficient implementation of <code class="computeroutput"><span class="identifier">reduce</span></code>.
|
||
To state that a function object is commutative, you will need to specialize
|
||
the class <code class="computeroutput"><a class="link" href="../boost/mpi/is_commutative.html" title="Struct template is_commutative">is_commutative</a></code>.
|
||
For instance, we could modify the previous example by telling Boost.MPI
|
||
that string concatenation is commutative:
|
||
</p>
|
||
<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span>
|
||
|
||
<span class="keyword">template</span><span class="special"><></span>
|
||
<span class="keyword">struct</span> <span class="identifier">is_commutative</span><span class="special"><</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">plus</span><span class="special"><</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">>,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">></span>
|
||
<span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{</span> <span class="special">};</span>
|
||
|
||
<span class="special">}</span> <span class="special">}</span> <span class="comment">// end namespace boost::mpi</span>
|
||
</pre>
|
||
<p>
|
||
By adding this code prior to <code class="computeroutput"><span class="identifier">main</span><span class="special">()</span></code>, Boost.MPI will assume that string concatenation
|
||
is commutative and employ a different parallel algorithm for the <code class="computeroutput"><span class="identifier">reduce</span></code> operation. Using this algorithm,
|
||
the program outputs the following when run with seven processes:
|
||
</p>
|
||
<pre class="programlisting">The result is zero one four five six two three
|
||
</pre>
|
||
<p>
|
||
Note how the numbers in the resulting string are in a different order:
|
||
this is a direct result of Boost.MPI reordering operations. The result
|
||
in this case differed from the non-commutative result because string concatenation
|
||
is not commutative: <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="string">"x"</span><span class="special">,</span>
|
||
<span class="string">"y"</span><span class="special">)</span></code>
|
||
is not the same as <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="string">"y"</span><span class="special">,</span>
|
||
<span class="string">"x"</span><span class="special">)</span></code>,
|
||
because argument order matters. For truly commutative operations (e.g.,
|
||
integer addition), the more efficient commutative algorithm will produce
|
||
the same result as the non-commutative algorithm. Boost.MPI also performs
|
||
direct mappings from function objects in <code class="computeroutput"><span class="special"><</span><span class="identifier">functional</span><span class="special">></span></code>
|
||
to <code class="computeroutput"><span class="identifier">MPI_Op</span></code> values predefined
|
||
by MPI (e.g., <code class="computeroutput"><span class="identifier">MPI_SUM</span></code>,
|
||
<code class="computeroutput"><span class="identifier">MPI_MAX</span></code>); if you have your
|
||
own function objects that can take advantage of this mapping, see the class
|
||
template <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_op.html" title="Struct template is_mpi_op">is_mpi_op</a></code>.
|
||
</p>
|
||
<div class="warning"><table border="0" summary="Warning">
|
||
<tr>
|
||
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Warning]" src="../../../doc/src/images/warning.png"></td>
|
||
<th align="left">Warning</th>
|
||
</tr>
|
||
<tr><td align="left" valign="top"><p>
|
||
Due to the underlying MPI limitations, it is important to note that the
|
||
operation must be stateless.
|
||
</p></td></tr>
|
||
</table></div>
|
||
<h5>
|
||
<a name="mpi.tutorial.collectives.reduce.h1"></a>
|
||
<span class="phrase"><a name="mpi.tutorial.collectives.reduce.all_process_variant"></a></span><a class="link" href="tutorial.html#mpi.tutorial.collectives.reduce.all_process_variant">All process
|
||
variant</a>
|
||
</h5>
|
||
<p>
|
||
Like <a class="link" href="tutorial.html#mpi.tutorial.collectives.gather" title="Gather"><code class="computeroutput"><span class="identifier">gather</span></code></a>,
|
||
<code class="computeroutput"><span class="identifier">reduce</span></code> has an "all"
|
||
variant called <code class="computeroutput"><a class="link" href="../boost/mpi/all_reduce.html" title="Function all_reduce">all_reduce</a></code> that performs
|
||
the reduction operation and broadcasts the result to all processes. This
|
||
variant is useful, for instance, in establishing global minimum or maximum
|
||
values.
|
||
</p>
|
||
<p>
|
||
The following code (<code class="computeroutput"><span class="identifier">global_min</span><span class="special">.</span><span class="identifier">cpp</span></code>)
|
||
shows a broadcasting version of the <code class="computeroutput"><span class="identifier">random_min</span><span class="special">.</span><span class="identifier">cpp</span></code>
|
||
example:
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span>
|
||
<span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span>
|
||
<span class="keyword">int</span> <span class="identifier">minimum</span><span class="special">;</span>
|
||
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">all_reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">minimum</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">>());</span>
|
||
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"The minimum value is "</span> <span class="special"><<</span> <span class="identifier">minimum</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
In that example we provide both input and output values, requiring twice
|
||
as much space, which can be a problem depending on the size of the transmitted
|
||
data. If there is no need to preserve the input value, the output value
|
||
can be omitted. In that case the input value will be overridden with the
|
||
output value and Boost.MPI is able, in some situation, to implement the
|
||
operation with a more space efficient solution (using the <code class="computeroutput"><span class="identifier">MPI_IN_PLACE</span></code> flag of the MPI C mapping),
|
||
as in the following example (<code class="computeroutput"><span class="identifier">in_place_global_min</span><span class="special">.</span><span class="identifier">cpp</span></code>):
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span>
|
||
<span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span>
|
||
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">all_reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">>());</span>
|
||
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"The minimum value is "</span> <span class="special"><<</span> <span class="identifier">my_number</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
</div>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h3 class="title">
|
||
<a name="mpi.tutorial.user_data_types"></a><a class="link" href="tutorial.html#mpi.tutorial.user_data_types" title="User-defined data types">User-defined data types</a>
|
||
</h3></div></div></div>
|
||
<p>
|
||
The inclusion of <code class="computeroutput"><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span></code> in the previous examples is very important:
|
||
it makes values of type <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span></code>
|
||
serializable, so that they can be be transmitted using Boost.MPI. In general,
|
||
built-in C++ types (<code class="computeroutput"><span class="keyword">int</span></code>s, <code class="computeroutput"><span class="keyword">float</span></code>s, characters, etc.) can be transmitted
|
||
over MPI directly, while user-defined and library-defined types will need
|
||
to first be serialized (packed) into a format that is amenable to transmission.
|
||
Boost.MPI relies on the <a href="../../../libs/serialization/doc" target="_top">Boost.Serialization</a>
|
||
library to serialize and deserialize data types.
|
||
</p>
|
||
<p>
|
||
For types defined by the standard library (such as <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span></code>
|
||
or <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span></code>) and some types in Boost (such as
|
||
<code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">variant</span></code>), the <a href="../../../libs/serialization/doc" target="_top">Boost.Serialization</a>
|
||
library already contains all of the required serialization code. In these
|
||
cases, you need only include the appropriate header from the <code class="computeroutput"><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span></code> directory.
|
||
</p>
|
||
<p>
|
||
For types that do not already have a serialization header, you will first
|
||
need to implement serialization code before the types can be transmitted
|
||
using Boost.MPI. Consider a simple class <a class="link" href="tutorial.html#gps_position"><code class="computeroutput"><span class="identifier">gps_position</span></code></a> that contains members
|
||
<code class="computeroutput"><span class="identifier">degrees</span></code>, <code class="computeroutput"><span class="identifier">minutes</span></code>,
|
||
and <code class="computeroutput"><span class="identifier">seconds</span></code>. This class is
|
||
made serializable by making it a friend of <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">serialization</span><span class="special">::</span><span class="identifier">access</span></code>
|
||
and introducing the templated <code class="computeroutput"><span class="identifier">serialize</span><span class="special">()</span></code> function, as follows:<a name="gps_position"></a>
|
||
</p>
|
||
<pre class="programlisting"><span class="keyword">class</span> <span class="identifier">gps_position</span>
|
||
<span class="special">{</span>
|
||
<span class="keyword">private</span><span class="special">:</span>
|
||
<span class="keyword">friend</span> <span class="keyword">class</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">serialization</span><span class="special">::</span><span class="identifier">access</span><span class="special">;</span>
|
||
|
||
<span class="keyword">template</span><span class="special"><</span><span class="keyword">class</span> <span class="identifier">Archive</span><span class="special">></span>
|
||
<span class="keyword">void</span> <span class="identifier">serialize</span><span class="special">(</span><span class="identifier">Archive</span> <span class="special">&</span> <span class="identifier">ar</span><span class="special">,</span> <span class="keyword">const</span> <span class="keyword">unsigned</span> <span class="keyword">int</span> <span class="identifier">version</span><span class="special">)</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">ar</span> <span class="special">&</span> <span class="identifier">degrees</span><span class="special">;</span>
|
||
<span class="identifier">ar</span> <span class="special">&</span> <span class="identifier">minutes</span><span class="special">;</span>
|
||
<span class="identifier">ar</span> <span class="special">&</span> <span class="identifier">seconds</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">degrees</span><span class="special">;</span>
|
||
<span class="keyword">int</span> <span class="identifier">minutes</span><span class="special">;</span>
|
||
<span class="keyword">float</span> <span class="identifier">seconds</span><span class="special">;</span>
|
||
<span class="keyword">public</span><span class="special">:</span>
|
||
<span class="identifier">gps_position</span><span class="special">(){};</span>
|
||
<span class="identifier">gps_position</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">d</span><span class="special">,</span> <span class="keyword">int</span> <span class="identifier">m</span><span class="special">,</span> <span class="keyword">float</span> <span class="identifier">s</span><span class="special">)</span> <span class="special">:</span>
|
||
<span class="identifier">degrees</span><span class="special">(</span><span class="identifier">d</span><span class="special">),</span> <span class="identifier">minutes</span><span class="special">(</span><span class="identifier">m</span><span class="special">),</span> <span class="identifier">seconds</span><span class="special">(</span><span class="identifier">s</span><span class="special">)</span>
|
||
<span class="special">{}</span>
|
||
<span class="special">};</span>
|
||
</pre>
|
||
<p>
|
||
Complete information about making types serializable is beyond the scope
|
||
of this tutorial. For more information, please see the <a href="../../../libs/serialization/doc" target="_top">Boost.Serialization</a>
|
||
library tutorial from which the above example was extracted. One important
|
||
side benefit of making types serializable for Boost.MPI is that they become
|
||
serializable for any other usage, such as storing the objects to disk and
|
||
manipulated them in XML.
|
||
</p>
|
||
<p>
|
||
Some serializable types, like <a class="link" href="tutorial.html#gps_position"><code class="computeroutput"><span class="identifier">gps_position</span></code></a> above, have a fixed
|
||
amount of data stored at fixed offsets and are fully defined by the values
|
||
of their data member (most POD with no pointers are a good example). When
|
||
this is the case, Boost.MPI can optimize their serialization and transmission
|
||
by avoiding extraneous copy operations. To enable this optimization, users
|
||
must specialize the type trait <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>, e.g.:
|
||
</p>
|
||
<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span>
|
||
<span class="keyword">template</span> <span class="special"><></span>
|
||
<span class="keyword">struct</span> <span class="identifier">is_mpi_datatype</span><span class="special"><</span><span class="identifier">gps_position</span><span class="special">></span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{</span> <span class="special">};</span>
|
||
<span class="special">}</span> <span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
For non-template types we have defined a macro to simplify declaring a type
|
||
as an MPI datatype
|
||
</p>
|
||
<pre class="programlisting"><span class="identifier">BOOST_IS_MPI_DATATYPE</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">)</span>
|
||
</pre>
|
||
<p>
|
||
For composite traits, the specialization of <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code> may depend
|
||
on <code class="computeroutput"><span class="identifier">is_mpi_datatype</span></code> itself.
|
||
For instance, a <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">array</span></code> object is fixed only when the type
|
||
of the parameter it stores is fixed:
|
||
</p>
|
||
<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span>
|
||
<span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">T</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span> <span class="identifier">N</span><span class="special">></span>
|
||
<span class="keyword">struct</span> <span class="identifier">is_mpi_datatype</span><span class="special"><</span><span class="identifier">array</span><span class="special"><</span><span class="identifier">T</span><span class="special">,</span> <span class="identifier">N</span><span class="special">></span> <span class="special">></span>
|
||
<span class="special">:</span> <span class="keyword">public</span> <span class="identifier">is_mpi_datatype</span><span class="special"><</span><span class="identifier">T</span><span class="special">></span> <span class="special">{</span> <span class="special">};</span>
|
||
<span class="special">}</span> <span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
The redundant copy elimination optimization can only be applied when the
|
||
shape of the data type is completely fixed. Variable-length types (e.g.,
|
||
strings, linked lists) and types that store pointers cannot use the optimization,
|
||
but Boost.MPI will be unable to detect this error at compile time. Attempting
|
||
to perform this optimization when it is not correct will likely result in
|
||
segmentation faults and other strange program behavior.
|
||
</p>
|
||
<p>
|
||
Boost.MPI can transmit any user-defined data type from one process to another.
|
||
Built-in types can be transmitted without any extra effort; library-defined
|
||
types require the inclusion of a serialization header; and user-defined types
|
||
will require the addition of serialization code. Fixed data types can be
|
||
optimized for transmission using the <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code> type trait.
|
||
</p>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h3 class="title">
|
||
<a name="mpi.tutorial.communicators"></a><a class="link" href="tutorial.html#mpi.tutorial.communicators" title="Communicators">Communicators</a>
|
||
</h3></div></div></div>
|
||
<div class="toc"><dl class="toc">
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.communicators.managing">Managing communicators</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.communicators.cartesian_communicator">Cartesian
|
||
communicator</a></span></dt>
|
||
</dl></div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h4 class="title">
|
||
<a name="mpi.tutorial.communicators.managing"></a><a class="link" href="tutorial.html#mpi.tutorial.communicators.managing" title="Managing communicators">Managing communicators</a>
|
||
</h4></div></div></div>
|
||
<p>
|
||
Communication with Boost.MPI always occurs over a communicator. A communicator
|
||
contains a set of processes that can send messages among themselves and
|
||
perform collective operations. There can be many communicators within a
|
||
single program, each of which contains its own isolated communication space
|
||
that acts independently of the other communicators.
|
||
</p>
|
||
<p>
|
||
When the MPI environment is initialized, only the "world" communicator
|
||
(called <code class="computeroutput"><span class="identifier">MPI_COMM_WORLD</span></code>
|
||
in the MPI C and Fortran bindings) is available. The "world"
|
||
communicator, accessed by default-constructing a <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">mpi::communicator</a></code>
|
||
object, contains all of the MPI processes present when the program begins
|
||
execution. Other communicators can then be constructed by duplicating or
|
||
building subsets of the "world" communicator. For instance, in
|
||
the following program we split the processes into two groups: one for processes
|
||
generating data and the other for processes that will collect the data.
|
||
(<code class="computeroutput"><span class="identifier">generate_collect</span><span class="special">.</span><span class="identifier">cpp</span></code>)
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">vector</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
|
||
<span class="keyword">enum</span> <span class="identifier">message_tags</span> <span class="special">{</span><span class="identifier">msg_data_packet</span><span class="special">,</span> <span class="identifier">msg_broadcast_data</span><span class="special">,</span> <span class="identifier">msg_finished</span><span class="special">};</span>
|
||
|
||
<span class="keyword">void</span> <span class="identifier">generate_data</span><span class="special">(</span><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">local</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">);</span>
|
||
<span class="keyword">void</span> <span class="identifier">collect_data</span><span class="special">(</span><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">local</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">);</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="keyword">bool</span> <span class="identifier">is_generator</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><</span> <span class="number">2</span> <span class="special">*</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span> <span class="special">/</span> <span class="number">3</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">local</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">split</span><span class="special">(</span><span class="identifier">is_generator</span><span class="special">?</span> <span class="number">0</span> <span class="special">:</span> <span class="number">1</span><span class="special">);</span>
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">is_generator</span><span class="special">)</span> <span class="identifier">generate_data</span><span class="special">(</span><span class="identifier">local</span><span class="special">,</span> <span class="identifier">world</span><span class="special">);</span>
|
||
<span class="keyword">else</span> <span class="identifier">collect_data</span><span class="special">(</span><span class="identifier">local</span><span class="special">,</span> <span class="identifier">world</span><span class="special">);</span>
|
||
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
When communicators are split in this way, their processes retain membership
|
||
in both the original communicator (which is not altered by the split) and
|
||
the new communicator. However, the ranks of the processes may be different
|
||
from one communicator to the next, because the rank values within a communicator
|
||
are always contiguous values starting at zero. In the example above, the
|
||
first two thirds of the processes become "generators" and the
|
||
remaining processes become "collectors". The ranks of the "collectors"
|
||
in the <code class="computeroutput"><span class="identifier">world</span></code> communicator
|
||
will be 2/3 <code class="computeroutput"><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span></code>
|
||
and greater, whereas the ranks of the same collector processes in the
|
||
<code class="computeroutput"><span class="identifier">local</span></code> communicator will
|
||
start at zero. The following excerpt from <code class="computeroutput"><span class="identifier">collect_data</span><span class="special">()</span></code> (in <code class="computeroutput"><span class="identifier">generate_collect</span><span class="special">.</span><span class="identifier">cpp</span></code>)
|
||
illustrates how to manage multiple communicators:
|
||
</p>
|
||
<pre class="programlisting"><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">status</span> <span class="identifier">msg</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">probe</span><span class="special">();</span>
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">msg</span><span class="special">.</span><span class="identifier">tag</span><span class="special">()</span> <span class="special">==</span> <span class="identifier">msg_data_packet</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="comment">// Receive the packet of data</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">data</span><span class="special">;</span>
|
||
<span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="identifier">msg</span><span class="special">.</span><span class="identifier">source</span><span class="special">(),</span> <span class="identifier">msg</span><span class="special">.</span><span class="identifier">tag</span><span class="special">(),</span> <span class="identifier">data</span><span class="special">);</span>
|
||
|
||
<span class="comment">// Tell each of the collectors that we'll be broadcasting some data</span>
|
||
<span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">dest</span> <span class="special">=</span> <span class="number">1</span><span class="special">;</span> <span class="identifier">dest</span> <span class="special"><</span> <span class="identifier">local</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">dest</span><span class="special">)</span>
|
||
<span class="identifier">local</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="identifier">dest</span><span class="special">,</span> <span class="identifier">msg_broadcast_data</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">.</span><span class="identifier">source</span><span class="special">());</span>
|
||
|
||
<span class="comment">// Broadcast the actual data.</span>
|
||
<span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">local</span><span class="special">,</span> <span class="identifier">data</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
The code in this except is executed by the "master" collector,
|
||
e.g., the node with rank 2/3 <code class="computeroutput"><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span></code> in the <code class="computeroutput"><span class="identifier">world</span></code>
|
||
communicator and rank 0 in the <code class="computeroutput"><span class="identifier">local</span></code>
|
||
(collector) communicator. It receives a message from a generator via the
|
||
<code class="computeroutput"><span class="identifier">world</span></code> communicator, then
|
||
broadcasts the message to each of the collectors via the <code class="computeroutput"><span class="identifier">local</span></code> communicator.
|
||
</p>
|
||
<p>
|
||
For more control in the creation of communicators for subgroups of processes,
|
||
the Boost.MPI <code class="computeroutput"><a class="link" href="../boost/mpi/group.html" title="Class group">group</a></code>
|
||
provides facilities to compute the union (<code class="computeroutput"><span class="special">|</span></code>),
|
||
intersection (<code class="computeroutput"><span class="special">&</span></code>), and
|
||
difference (<code class="computeroutput"><span class="special">-</span></code>) of two groups,
|
||
generate arbitrary subgroups, etc.
|
||
</p>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h4 class="title">
|
||
<a name="mpi.tutorial.communicators.cartesian_communicator"></a><a class="link" href="tutorial.html#mpi.tutorial.communicators.cartesian_communicator" title="Cartesian communicator">Cartesian
|
||
communicator</a>
|
||
</h4></div></div></div>
|
||
<p>
|
||
A communicator can be organised as a cartesian grid, here a basic example:
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">vector</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">collectives</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">cartesian_communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">minimal</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
<span class="keyword">int</span> <span class="identifier">test_main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span> <span class="special">!=</span> <span class="number">24</span><span class="special">)</span> <span class="keyword">return</span> <span class="special">-</span><span class="number">1</span><span class="special">;</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">cartesian_dimension</span> <span class="identifier">dims</span><span class="special">[]</span> <span class="special">=</span> <span class="special">{{</span><span class="number">2</span><span class="special">,</span> <span class="keyword">true</span><span class="special">},</span> <span class="special">{</span><span class="number">3</span><span class="special">,</span><span class="keyword">true</span><span class="special">},</span> <span class="special">{</span><span class="number">4</span><span class="special">,</span><span class="keyword">true</span><span class="special">}};</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">cartesian_communicator</span> <span class="identifier">cart</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">cartesian_topology</span><span class="special">(</span><span class="identifier">dims</span><span class="special">));</span>
|
||
<span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">r</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">r</span> <span class="special"><</span> <span class="identifier">cart</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">r</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">cart</span><span class="special">.</span><span class="identifier">barrier</span><span class="special">();</span>
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">r</span> <span class="special">==</span> <span class="identifier">cart</span><span class="special">.</span><span class="identifier">rank</span><span class="special">())</span> <span class="special">{</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">c</span> <span class="special">=</span> <span class="identifier">cart</span><span class="special">.</span><span class="identifier">coordinates</span><span class="special">(</span><span class="identifier">r</span><span class="special">);</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"rk :"</span> <span class="special"><<</span> <span class="identifier">r</span> <span class="special"><<</span> <span class="string">" coords: "</span>
|
||
<span class="special"><<</span> <span class="identifier">c</span><span class="special">[</span><span class="number">0</span><span class="special">]</span> <span class="special"><<</span> <span class="char">' '</span> <span class="special"><<</span> <span class="identifier">c</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special"><<</span> <span class="char">' '</span> <span class="special"><<</span> <span class="identifier">c</span><span class="special">[</span><span class="number">2</span><span class="special">]</span> <span class="special"><<</span> <span class="char">'\n'</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
<span class="special">}</span>
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
</div>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h3 class="title">
|
||
<a name="mpi.tutorial.threading"></a><a class="link" href="tutorial.html#mpi.tutorial.threading" title="Threads">Threads</a>
|
||
</h3></div></div></div>
|
||
<p>
|
||
There are an increasing number of hybrid parallel applications that mix distributed
|
||
and shared memory parallelism. To know how to support that model, one need
|
||
to know what level of threading support is guaranteed by the MPI implementation.
|
||
There are 4 ordered level of possible threading support described by <code class="computeroutput"><a class="link" href="../boost/mpi/threading/level.html" title="Type level">mpi::threading::level</a></code>. At the
|
||
lowest level, you should not use threads at all, at the highest level, any
|
||
thread can perform MPI call.
|
||
</p>
|
||
<p>
|
||
If you want to use multi-threading in your MPI application, you should indicate
|
||
in the environment constructor your preferred threading support. Then probe
|
||
the one the library did provide, and decide what you can do with it (it could
|
||
be nothing, then aborting is a valid option):
|
||
</p>
|
||
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
|
||
<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
|
||
<span class="keyword">namespace</span> <span class="identifier">mt</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">threading</span><span class="special">;</span>
|
||
|
||
<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
|
||
<span class="special">{</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">mt</span><span class="special">::</span><span class="identifier">funneled</span><span class="special">);</span>
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">env</span><span class="special">.</span><span class="identifier">thread_level</span><span class="special">()</span> <span class="special"><</span> <span class="identifier">mt</span><span class="special">::</span><span class="identifier">funneled</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="identifier">env</span><span class="special">.</span><span class="identifier">abort</span><span class="special">(-</span><span class="number">1</span><span class="special">);</span>
|
||
<span class="special">}</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"I am process "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><<</span> <span class="string">" of "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span>
|
||
<span class="special"><<</span> <span class="string">"."</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
|
||
<span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="special">}</span>
|
||
</pre>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h3 class="title">
|
||
<a name="mpi.tutorial.skeleton_and_content"></a><a class="link" href="tutorial.html#mpi.tutorial.skeleton_and_content" title="Separating structure from content">Separating structure
|
||
from content</a>
|
||
</h3></div></div></div>
|
||
<p>
|
||
When communicating data types over MPI that are not fundamental to MPI (such
|
||
as strings, lists, and user-defined data types), Boost.MPI must first serialize
|
||
these data types into a buffer and then communicate them; the receiver then
|
||
copies the results into a buffer before deserializing into an object on the
|
||
other end. For some data types, this overhead can be eliminated by using
|
||
<code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>.
|
||
However, variable-length data types such as strings and lists cannot be MPI
|
||
data types.
|
||
</p>
|
||
<p>
|
||
Boost.MPI supports a second technique for improving performance by separating
|
||
the structure of these variable-length data structures from the content stored
|
||
in the data structures. This feature is only beneficial when the shape of
|
||
the data structure remains the same but the content of the data structure
|
||
will need to be communicated several times. For instance, in a finite element
|
||
analysis the structure of the mesh may be fixed at the beginning of computation
|
||
but the various variables on the cells of the mesh (temperature, stress,
|
||
etc.) will be communicated many times within the iterative analysis process.
|
||
In this case, Boost.MPI allows one to first send the "skeleton"
|
||
of the mesh once, then transmit the "content" multiple times. Since
|
||
the content need not contain any information about the structure of the data
|
||
type, it can be transmitted without creating separate communication buffers.
|
||
</p>
|
||
<p>
|
||
To illustrate the use of skeletons and content, we will take a somewhat more
|
||
limited example wherein a master process generates random number sequences
|
||
into a list and transmits them to several slave processes. The length of
|
||
the list will be fixed at program startup, so the content of the list (i.e.,
|
||
the current sequence of numbers) can be transmitted efficiently. The complete
|
||
example is available in <code class="computeroutput"><span class="identifier">example</span><span class="special">/</span><span class="identifier">random_content</span><span class="special">.</span><span class="identifier">cpp</span></code>. We
|
||
being with the master process (rank 0), which builds a list, communicates
|
||
its structure via a <code class="computeroutput"><a class="link" href="reference.html#boost.mpi.skeleton">skeleton</a></code>, then repeatedly
|
||
generates random number sequences to be broadcast to the slave processes
|
||
via <code class="computeroutput">content</code>:
|
||
</p>
|
||
<pre class="programlisting"><span class="comment">// Generate the list and broadcast its structure</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">list</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">l</span><span class="special">(</span><span class="identifier">list_len</span><span class="special">);</span>
|
||
<span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">skeleton</span><span class="special">(</span><span class="identifier">l</span><span class="special">),</span> <span class="number">0</span><span class="special">);</span>
|
||
|
||
<span class="comment">// Generate content several times and broadcast out that content</span>
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">content</span> <span class="identifier">c</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">get_content</span><span class="special">(</span><span class="identifier">l</span><span class="special">);</span>
|
||
<span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">i</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">i</span> <span class="special"><</span> <span class="identifier">iterations</span><span class="special">;</span> <span class="special">++</span><span class="identifier">i</span><span class="special">)</span> <span class="special">{</span>
|
||
<span class="comment">// Generate new random values</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">generate</span><span class="special">(</span><span class="identifier">l</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span> <span class="special">&</span><span class="identifier">random</span><span class="special">);</span>
|
||
|
||
<span class="comment">// Broadcast the new content of l</span>
|
||
<span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">c</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
|
||
<span class="special">}</span>
|
||
|
||
<span class="comment">// Notify the slaves that we're done by sending all zeroes</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">fill</span><span class="special">(</span><span class="identifier">l</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span> <span class="number">0</span><span class="special">);</span>
|
||
<span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">c</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
|
||
</pre>
|
||
<p>
|
||
The slave processes have a very similar structure to the master. They receive
|
||
(via the <code class="computeroutput"><a class="link" href="../boost/mpi/broadcast.html" title="Function broadcast">broadcast()</a></code> call) the skeleton of the
|
||
data structure, then use it to build their own lists of integers. In each
|
||
iteration, they receive via another <code class="computeroutput"><span class="identifier">broadcast</span><span class="special">()</span></code> the new content in the data structure and
|
||
compute some property of the data:
|
||
</p>
|
||
<pre class="programlisting"><span class="comment">// Receive the content and build up our own list</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">list</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">l</span><span class="special">;</span>
|
||
<span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">skeleton</span><span class="special">(</span><span class="identifier">l</span><span class="special">),</span> <span class="number">0</span><span class="special">);</span>
|
||
|
||
<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">content</span> <span class="identifier">c</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">get_content</span><span class="special">(</span><span class="identifier">l</span><span class="special">);</span>
|
||
<span class="keyword">int</span> <span class="identifier">i</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span>
|
||
<span class="keyword">do</span> <span class="special">{</span>
|
||
<span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">c</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
|
||
|
||
<span class="keyword">if</span> <span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">find_if</span>
|
||
<span class="special">(</span><span class="identifier">l</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span>
|
||
<span class="identifier">std</span><span class="special">::</span><span class="identifier">bind1st</span><span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">not_equal_to</span><span class="special"><</span><span class="keyword">int</span><span class="special">>(),</span> <span class="number">0</span><span class="special">))</span> <span class="special">==</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">())</span>
|
||
<span class="keyword">break</span><span class="special">;</span>
|
||
|
||
<span class="comment">// Compute some property of the data.</span>
|
||
|
||
<span class="special">++</span><span class="identifier">i</span><span class="special">;</span>
|
||
<span class="special">}</span> <span class="keyword">while</span> <span class="special">(</span><span class="keyword">true</span><span class="special">);</span>
|
||
</pre>
|
||
<p>
|
||
The skeletons and content of any Serializable data type can be transmitted
|
||
either via the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_4-bb">send</a></code> and <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_9-bb">recv</a></code> members of the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">communicator</a></code>
|
||
class (for point-to-point communicators) or broadcast via the <code class="computeroutput"><a class="link" href="../boost/mpi/broadcast.html" title="Function broadcast">broadcast()</a></code> collective. When separating
|
||
a data structure into a skeleton and content, be careful not to modify the
|
||
data structure (either on the sender side or the receiver side) without transmitting
|
||
the skeleton again. Boost.MPI can not detect these accidental modifications
|
||
to the data structure, which will likely result in incorrect data being transmitted
|
||
or unstable programs.
|
||
</p>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h3 class="title">
|
||
<a name="mpi.tutorial.performance_optimizations"></a><a class="link" href="tutorial.html#mpi.tutorial.performance_optimizations" title="Performance optimizations">Performance optimizations</a>
|
||
</h3></div></div></div>
|
||
<div class="toc"><dl class="toc">
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.performance_optimizations.serialization_optimizations">Serialization
|
||
optimizations</a></span></dt>
|
||
<dt><span class="section"><a href="tutorial.html#mpi.tutorial.performance_optimizations.homogeneous_machines">Homogeneous
|
||
Machines</a></span></dt>
|
||
</dl></div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h4 class="title">
|
||
<a name="mpi.tutorial.performance_optimizations.serialization_optimizations"></a><a class="link" href="tutorial.html#mpi.tutorial.performance_optimizations.serialization_optimizations" title="Serialization optimizations">Serialization
|
||
optimizations</a>
|
||
</h4></div></div></div>
|
||
<p>
|
||
To obtain optimal performance for small fixed-length data types not containing
|
||
any pointers it is very important to mark them using the type traits of
|
||
Boost.MPI and Boost.Serialization.
|
||
</p>
|
||
<p>
|
||
It was already discussed that fixed length types containing no pointers
|
||
can be using as <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>, e.g.:
|
||
</p>
|
||
<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span>
|
||
<span class="keyword">template</span> <span class="special"><></span>
|
||
<span class="keyword">struct</span> <span class="identifier">is_mpi_datatype</span><span class="special"><</span><span class="identifier">gps_position</span><span class="special">></span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{</span> <span class="special">};</span>
|
||
<span class="special">}</span> <span class="special">}</span>
|
||
</pre>
|
||
<p>
|
||
or the equivalent macro
|
||
</p>
|
||
<pre class="programlisting"><span class="identifier">BOOST_IS_MPI_DATATYPE</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">)</span>
|
||
</pre>
|
||
<p>
|
||
In addition it can give a substantial performance gain to turn off tracking
|
||
and versioning for these types, if no pointers to these types are used,
|
||
by using the traits classes or helper macros of Boost.Serialization:
|
||
</p>
|
||
<pre class="programlisting"><span class="identifier">BOOST_CLASS_TRACKING</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">,</span><span class="identifier">track_never</span><span class="special">)</span>
|
||
<span class="identifier">BOOST_CLASS_IMPLEMENTATION</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">,</span><span class="identifier">object_serializable</span><span class="special">)</span>
|
||
</pre>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h4 class="title">
|
||
<a name="mpi.tutorial.performance_optimizations.homogeneous_machines"></a><a class="link" href="tutorial.html#mpi.tutorial.performance_optimizations.homogeneous_machines" title="Homogeneous Machines">Homogeneous
|
||
Machines</a>
|
||
</h4></div></div></div>
|
||
<p>
|
||
More optimizations are possible on homogeneous machines, by avoiding MPI_Pack/MPI_Unpack
|
||
calls but using direct bitwise copy. This feature is enabled by default
|
||
by defining the macro <code class="computeroutput"><a class="link" href="../BOOST_MPI_HOMOGENEOUS.html" title="Macro BOOST_MPI_HOMOGENEOUS">BOOST_MPI_HOMOGENEOUS</a></code>
|
||
in the include file <code class="computeroutput"><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">config</span><span class="special">.</span><span class="identifier">hpp</span></code>.
|
||
That definition must be consistent when building Boost.MPI and when building
|
||
the application.
|
||
</p>
|
||
<p>
|
||
In addition all classes need to be marked both as is_mpi_datatype and as
|
||
is_bitwise_serializable, by using the helper macro of Boost.Serialization:
|
||
</p>
|
||
<pre class="programlisting"><span class="identifier">BOOST_IS_BITWISE_SERIALIZABLE</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">)</span>
|
||
</pre>
|
||
<p>
|
||
Usually it is safe to serialize a class for which is_mpi_datatype is true
|
||
by using binary copy of the bits. The exception are classes for which some
|
||
members should be skipped for serialization.
|
||
</p>
|
||
</div>
|
||
</div>
|
||
<div class="footnotes">
|
||
<br><hr style="width:100; text-align:left;margin-left: 0">
|
||
<div id="ftn.mpi.tutorial.f0" class="footnote"><p><a href="#mpi.tutorial.f0" class="para"><sup class="para">[11] </sup></a>
|
||
According to the MPI standard, initialization must take place at user's
|
||
initiative after once the main function has been called.
|
||
</p></div>
|
||
</div>
|
||
</div>
|
||
<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
|
||
<td align="left"></td>
|
||
<td align="right"><div class="copyright-footer">Copyright © 2005-2007 Douglas Gregor,
|
||
Matthias Troyer, Trustees of Indiana University<p>
|
||
Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||
file LICENSE_1_0.txt or copy at <ulink url="http://www.boost.org/LICENSE_1_0.txt">
|
||
http://www.boost.org/LICENSE_1_0.txt </ulink>)
|
||
</p>
|
||
</div></td>
|
||
</tr></table>
|
||
<hr>
|
||
<div class="spirit-nav">
|
||
<a accesskey="p" href="getting_started.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../mpi.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="c_mapping.html"><img src="../../../doc/src/images/next.png" alt="Next"></a>
|
||
</div>
|
||
</body>
|
||
</html>
|