On Sun, Nov 28, 2010 at 1:25 PM, Philipp Kraus wrote:

I understand the recv in this way, that it waits until a message is send. I think if I have no messages, the receive blocks the loop, do it? I would like to create the code in that way:

while (...) { while mpi-message-is-there    do something }

Indeed, `recv` is a blocking call. To receive a message only if one is available, you need to probe for it first, and then receive. Something along these lines:: while(...) { while(boost::optionalmpi::status status = p_comm.iprobe(mpi::any_source, TAG)) { // a message is available, receive it mpi::recv(status->source(), status->tag(), l_str); // process `l_str` }; // ... }; Caveat: use `status->source()` and `status->tag()` in the recv call to ensure you recv the message that you probed for (this may still fail if you're running iprobe/recv loops in several threads concurrently within the same MPI process). Best regards, Riccardo

Hi, On Sun, Nov 28, 2010 at 5:44 PM, Kraus Philipp wrote:

I'll get now some compiler warning in the boost::optional line

Boost/1.45/include/boost/mpi/status.hpp: In member function 'void machinelearning::tools::logger::listener(const boost::mpi::communicator&)': /Boost/1.45/include/boost/mpi/status.hpp:48: warning: dereferencing pointer '<anonymous>' does break strict-aliasing rules

Do you think this warnings can be create a problem?

I'm afraid I don't know. However, a similar topic seems to have been discussed on the Boost mailing list earlier: http://lists.boost.org/Archives/boost/2009/12/160236.php As far as I understand, the conclusion is that that warning is harmless for boost::optional. Best regards, Riccardo

On 2010-11-28 13:42:04 +0100, Riccardo Murri said:

On Sun, Nov 28, 2010 at 1:25 PM, Philipp Kraus wrote:

...

I understand the recv in this way, that it waits until a message is send. I think if I have no messages, the receive blocks the loop, do it? I would like to create the code in that way:

while (...) { while mpi-message-is-there    do something }

Indeed, `recv` is a blocking call. To receive a message only if one is available, you need to probe for it first, and then receive. Something along these lines::

while(...) { while(boost::optionalmpi::status status = p_comm.iprobe(mpi::any_source, TAG)) { // a message is available, receive it mpi::recv(status->source(), status->tag(), l_str); // process `l_str` }; // ... };

Caveat: use `status->source()` and `status->tag()` in the recv call to ensure you recv the message that you probed for (this may still fail if you're running iprobe/recv loops in several threads concurrently within the same MPI process).

Thanks for the hint with the treads. I have written my code, but at the moment there is a problem to receive all messages: Every process run a "startmethod", which creates on CPU 0 a thread (at the end of the methode a barrier synchronize all CPUs). inline void logger::startListener( const mpi::communicator& p_mpi ) { if ((p_mpi.size() == 1) || (m_listenerrunnging)) return; m_listenerrunnging = true; if (p_mpi.rank() == 0) boost::thread l_thread( boost::bind( &logger::listener, this, boost::cref(p_mpi)) ); p_mpi.barrier(); } The thread method: inline void logger::listener( const mpi::communicator& p_mpi ) { boost::lock_guardboost::mutex l_lock(m_muxfinalize); while (m_listenerrunnging) { while (boost::optionalmpi::status l_status = p_mpi.iprobe(mpi::any_source, LOGGER_MPI_TAG)) { p_mpi.recv( l_status->source(), l_status->tag(), l_str ); } boost::this_thread::yield(); } } The lister must be shutdown with this code: inline void logger::shutdownListener( const mpi::communicator& p_mpi ) { if (!m_listenerrunnging) return; m_listenerrunnging = false; boost::lock_guardboost::mutex l_lock(m_muxfinalize); p_mpi.barrier(); } At this time I have two problems. Sometimes the programm create a error with: Adress not mapped (it seems that the iprobe creates the problem). I can't reproduce the error. Another problem is, that I don't received all messages, the messages are send with isend command. I have try to create a loop with iprobe in my shutdown method to get the rest of the messages, but that doesn't work well. I would send "every time" messages to CPU 0. The CPU 0 should collecte the messages, the last point to collect all messages shoudl be the shutdown method. Thx for helping with the problem Phil

On 2010-11-29 17:29:22 +0100, Riccardo Murri said:

Hi Philipp,

On Mon, Nov 29, 2010 at 4:17 PM, Philipp Kraus wrote:

...

At this time I have two problems. Sometimes the programm create a error with: Adress not mapped (it seems that the iprobe creates the problem). I can't reproduce the error.

It's difficult to give any sensible advice on this without compilable code; the general advice is to try to create a minimal test source that can reproduce the problem. In doing the exercise, I often found out the problem myself...

A rough guess about your "address not mapped" error: if you are going to use MPI with threads, then *two* conditions must be met:

(1) You must initialize MPI with MPI_Init_thread() instead of MPI_Init(). The boost::mpi::communicator ctor uses MPI_Init(), so you must run the initialization yourself and *then* create the communicator object. For instance::

MPI_Init_thread(&argc, &argv, MPI_THREAD_SERIALIZED) // ... mpi::communicator world;

Do the Boost lib support the initialisation process? I have take a look into the Boost MPI classes, but it seems Boost calls only the MPI_init(). Do I understand it correct, that I must use the mpi.h for calling the MPI_Init_thread() or is there a boost support?

(2) Your MPI library must have been compiled with threads support. This is not the case with some Linux distributions, e.g., Rocks 5.x, so double-check that MPI_Init_threads() actually did its job.

I used self-compiled libs, so that's not a problem Thanks Phil

On 2010-12-01 20:43:51 +0100, Riccardo Murri said:

On Wed, Dec 1, 2010 at 8:18 PM, Philipp Kraus wrote:

...

...

A rough guess about your "address not mapped" error: if you are going to use MPI with threads, then *two* conditions must be met:

(1) You must initialize MPI with MPI_Init_thread() instead of MPI_Init().  The boost::mpi::communicator ctor uses MPI_Init(), so you must run the initialization yourself and *then* create the communicator object.  For instance::

     MPI_Init_thread(&argc, &argv, MPI_THREAD_SERIALIZED)      // ...      mpi::communicator world;

Do the Boost lib support the initialisation process? I have take a look into the Boost MPI classes, but it seems Boost calls only the MPI_init(). Do I understand it correct, that I must use the mpi.h for calling the MPI_Init_thread() or is there a boost support?

You must call MPI_Init_thread() from mpi.h before you instanciate a boost::mpi::communicator.

I use the C++ code MPI::Init_thread( argc, argv, MPI_THREAD_SERIALIZED ); The error with "adress not mapped" is not shown anymore. There is only one problem at the time: The CPU 0 does not catch every message. The main method: MPI::Init_thread( argc, argv, MPI_THREAD_SERIALIZED ); mpi::communicator loMPICom; tl::logger::getInstance()->startListener(loMPICom); tl::logger::getInstance()->write(loMPICom, tl::logger::warn, "test all"); tl::logger::getInstance()->shutdownListener(loMPICom); MPI::Finalize(); So every process shoudl create a message and the CPU 0 should catch them all. The write method sends the data with isend (non-blocking). The CPU 0 runs a thread in background with while (m_listenerrunnging) { while (boost::optionalmpi::status l_status = p_mpi.iprobe(mpi::any_source, LOGGER_MPI_TAG)) { std::string l_str; p_mpi.recv( l_status->source(), l_status->tag(), l_str ); } boost::this_thread::yield(); } I would like to create it in this way: If I call isend the message is send to CPU 0 and is saved until the iprobe returns a valid status, than the message is received with recv. If I call my shutdown method I must call iprobe until the status is not valid, do I? If I run the program some times, I receive at one run 1 message, sometimes 4 or 5, but it's not deterministic. I have tested the program with 8 CPUs, so I would receive everytime 8 messages. Do you have any idea ? Thanks Phil

Am 02.12.2010 um 18:00 schrieb Riccardo Murri:

Hi Philipp,

On Thu, Dec 2, 2010 at 4:12 PM, Philipp Kraus wrote:

...

There is only one problem at the time: The CPU 0 does not catch every message. The main method:

MPI::Init_thread( argc, argv, MPI_THREAD_SERIALIZED ); mpi::communicator loMPICom; tl::logger::getInstance()->startListener(loMPICom); tl::logger::getInstance()->write(loMPICom, tl::logger::warn, "test all"); tl::logger::getInstance()->shutdownListener(loMPICom); MPI::Finalize();

So every process shoudl create a message and the CPU 0 should catch them all. The write method sends the data with isend (non-blocking). The CPU 0 runs a thread in background with

while (m_listenerrunnging) { while (boost::optionalmpi::status l_status = p_mpi.iprobe(mpi::any_source, LOGGER_MPI_TAG)) { std::string l_str; p_mpi.recv( l_status->source(), l_status->tag(), l_str ); } boost::this_thread::yield(); }

This code for the background thread will only probe for new messages until m_listenerrunnging is true; as soon as it turns to false (in logger::getInstance()->shutdownListener?), it will stop and receive no more messages.

You probably want to make it run until all messages have arrived. How you do this depends on your application: if you know how many messages will be coming, just count them; if you don't, make ranks !=1 send an "end of transmission" message when they are done and only shutdown the listener when all "end of transmission" messages have been received. There might be other methods that are more suited to your application.

Okay, that answers my question: It is also my problem to solving the transmission. The MPI interface has no implementated code.

...

If I run the program some times, I receive at one run 1 message, sometimes 4 or 5, but it's not deterministic.

MPI guarantees reliable delivery of messages but not within a specific timeframe. IOW, messages will be received *if you wait long enough*. If you shut down the receiver end too early, messages will be lost.

I think that is the main problem, so I must send a EO? for close all transmission. Thanks PHil