Riccardo Murri wrote:
One MPI rank is started per compute node; all OpenMP threads may call mpi::isend(); only one will do mpi::iprobe()/mpi::recv().
Although the above error is in the mpi::communicator::recv(), serializing the mpi::isend() calls apparently solves the issue; similarly, the program runs fine if I run it on one node only; with some other (smaller) inputs, it runs fine as well. This leads me to think that it is a thread-safety issue with the MPI part. I have checked that the MPI library (OpenMPI 1.4.2) is initialized with MPI_Init_threads() and provides the threading level MPI_THREAD_MULTIPLE.
So, question: is there a (known) thread-safety issue with Boost.MPI, or should I definitely look somewhere else?
As a MPI+openMP user, we take great care to not have multiple omp thread to do MPI operation not because of Boost.MPI but of MPI way of handling things which are not that thread safe.