boost no longer builds mpi libraries when using intel mpi

newer
[BB++] Now is 3.5x faster than...

older
MS controlled email domains - not...

Franck Houssen

19 Jul 2017 19 Jul '17

2:26 p.m.

Hello, Just to get your attention to this : https://github.com/easybuilders/easybuild-easyconfigs/issues/1065 Franck

Show replies by date

Alain Miniussi

19 Jul 19 Jul

7:04 p.m.

...

Hello,

Just to get your attention to this : https://github.com/easybuilders/easybuild-easyconfigs/issues/1065 It hasn't build with that syntax in years (edit, just saw it's from 2017), Intel's MPI does not support the -show* options family (and don't

On 19/07/2017 16:26, Franck Houssen via Boost wrote: plan to). There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c... I have no clue where the "using mpi"... is it called a directive ?.. is implemented, and if I did, I don't know how to detect we're using Intel's Implementation with bjam, nor how to pass the retrieved options, nor to what. The documentation layout is not great, not sure if it improved, but the documentation system at the time was an in house tool with install instructions scattered here and there so... Cheers, Alain PS: you might want to post MPI related issues to the boost MPI mailing list (boost-mpi@lists.boost.org)

Edward Diener

8:57 p.m.

On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote:

...

...
Hello,

Just to get your attention to this : https://github.com/easybuilders/easybuild-easyconfigs/issues/1065 It hasn't build with that syntax in years (edit, just saw it's from 2017), Intel's MPI does not support the -show* options family (and don't

On 19/07/2017 16:26, Franck Houssen via Boost wrote: plan to). There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c... I have no clue where the "using mpi"... is it called a directive ?.. is implemented, and if I did, I don't know how to detect we're using Intel's Implementation with bjam, nor how to pass the retrieved options, nor to what.

The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html.

...

The documentation layout is not great, not sure if it improved, but the documentation system at the time was an in house tool with install instructions scattered here and there so...

Cheers,

Alain

PS: you might want to post MPI related issues to the boost MPI mailing list (boost-mpi@lists.boost.org)

Alain Miniussi

9:40 p.m.

On 19/07/2017 22:57, Edward Diener via Boost wrote:

...

On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote:

...
...
Hello,

Just to get your attention to this : https://github.com/easybuilders/easybuild-easyconfigs/issues/1065 It hasn't build with that syntax in years (edit, just saw it's from 2017), Intel's MPI does not support the -show* options family (and don't plan to). There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c... I have no clue where the "using mpi"... is it called a directive ?.. is implemented, and if I did, I don't know how to detect we're using Intel's Implementation with bjam, nor how to pass the retrieved

On 19/07/2017 16:26, Franck Houssen via Boost wrote: options, nor to what.

The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi. I remember greping mpi in the build system a few year ago, but could not find a way to touch it (it failed with no meaningful message).

Thks Alain

...

...
The documentation layout is not great, not sure if it improved, but the documentation system at the time was an in house tool with install instructions scattered here and there so...

Cheers,

Alain

PS: you might want to post MPI related issues to the boost MPI mailing list (boost-mpi@lists.boost.org)

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Edward Diener

10 p.m.

On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote:

...

On 19/07/2017 22:57, Edward Diener via Boost wrote:

...
On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote:

...
On 19/07/2017 16:26, Franck Houssen via Boost wrote:

...
Hello,

Just to get your attention to this : https://github.com/easybuilders/easybuild-easyconfigs/issues/1065 It hasn't build with that syntax in years (edit, just saw it's from 2017), Intel's MPI does not support the -show* options family (and don't plan to). There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c...

I have no clue where the "using mpi"... is it called a directive ?.. is implemented, and if I did, I don't know how to detect we're using Intel's Implementation with bjam, nor how to pass the retrieved options, nor to what.

The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi. I remember greping mpi in the build system a few year ago, but could not find a way to touch it (it failed with no meaningful message).

Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s).

...

Thks

Alain

...
...
The documentation layout is not great, not sure if it improved, but the documentation system at the time was an in house tool with install instructions scattered here and there so...

Cheers,

Alain

PS: you might want to post MPI related issues to the boost MPI mailing list (boost-mpi@lists.boost.org)

Belcourt, Kenneth

20 Jul 20 Jul

1:53 a.m.

New subject: [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost mailto:boost@lists.boost.org> wrote: On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote: On 19/07/2017 22:57, Edward Diener via Boost wrote: On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote: On 19/07/2017 16:26, Franck Houssen via Boost wrote: Hello, Just to get your attention to this : https://github.com/easybuilders/easybuild-easyconfigs/issues/1065 It hasn't build with that syntax in years (edit, just saw it's from 2017), Intel's MPI does not support the -show* options family (and don't plan to). There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c... I have no clue where the "using mpi"... is it called a directive ?.. is implemented, and if I did, I don't know how to detect we're using Intel's Implementation with bjam, nor how to pass the retrieved options, nor to what. The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi. I remember greping mpi in the build system a few year ago, but could not find a way to touch it (it failed with no meaningful message). Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s). Hi Alain, I think I’ve sent this before, but here’s the user-config.jam I use to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1. using mpi : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx ; using intel ; cd libs/mpi/test ../../../b2 -d+2 toolset=intel Does this work for you? Previously you’d mentioned IMPI not working with Boost.Build, can you remind me what that failure involved? Noel

Alain Miniussi

8:34 a.m.

New subject: [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

...

On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost mailto:boost@lists.boost.org> wrote:

On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote: On 19/07/2017 22:57, Edward Diener via Boost wrote: On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote: On 19/07/2017 16:26, Franck Houssen via Boost wrote: Hello,

Just to get your attention to this : https://github.com/easybuilders/easybuild-easyconfigs/issues/1065 It hasn't build with that syntax in years (edit, just saw it's from 2017), Intel's MPI does not support the -show* options family (and don't plan to). There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c... I have no clue where the "using mpi"... is it called a directive ?.. is implemented, and if I did, I don't know how to detect we're using Intel's Implementation with bjam, nor how to pass the retrieved options, nor to what.

The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi. I remember greping mpi in the build system a few year ago, but could not find a way to touch it (it failed with no meaningful message).

Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s).

Hi Alain,

I think I’ve sent this before, but here’s the user-config.jam I use to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1.

using mpi : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx ;

using intel ;

cd libs/mpi/test ../../../b2 -d+2 toolset=intel

Does this work for you? Previously you’d mentioned IMPI not working with Boost.Build, can you remind me what that failure involved? Sure. I try to take the story from the beginning so that all the involved

On 20/07/2017 03:53, Belcourt, Kenneth via Boost wrote: people can get what I see as the whole picture. Sorry for the boring, well known bits. The MPI people thought that the easiest way to compile MPI application was through compiler wrapper like mpicc that would call the underlying compiler with the right options. Those options include, but are not limited to[*], the -I<incl path> -L<lib path> -l<libname> option. Sometime (most of the time IMO) this is not good: we want to run the compiler directly with right options (after all, we're just want to use a library at that point). So the question is how to retrieve them. Some/most MPI implementations provides wrappers options to retrieve those. OpenMPI based implementation usually provide --showme:(compile|link) option, so adding the output of $(mpicc --showme:link) to the link flags will do the job for example. Mpich based implementation usually provides -compile_info -link_info (which work slightly differently). Most tools like bjam/cmake/autotool/<other I guess> tries to find the compile and link flags through those options. They try -show:compile, see if it succeed, and if it does not, try --compile_info. Intel is based on mpich, but both --compile_info/--link_info/-show* return the same output (a mix of link and compile information). Even more annoying, the -show* option, even when it fails, returns a success status on many versions[**], meaning the build tool does not detect that the option is not supported (FindMPI.cmake does some magic to detect the Intel problem on recent versions) and does not know it should try something else. As a result, with most currently deployed Intel's MPI, you need to explicitly set the include and lib path and libraries. And of course, some MPI implementations do not have wrappers. Real world sucks. Alain [*]: meaning it's not so obvious that the bjam solution of providing includes path, lib search and libs is equivalent. We could be missing something, obviously not critical. But that's another story. [**]: just saw that the very last one does not, which is a good thing, maybe there is hope for humanity and reporting issues to support had some effect after all. But the behavior is still problematic, both link and compilation queries returns the same options.

...

Noel

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Alain Miniussi

8:37 a.m.

New subject: [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

On 20/07/2017 10:34, Alain Miniussi via Boost wrote:

...

On 20/07/2017 03:53, Belcourt, Kenneth via Boost wrote:

...
On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost mailto:boost@lists.boost.org> wrote:

On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote: On 19/07/2017 22:57, Edward Diener via Boost wrote: On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote: On 19/07/2017 16:26, Franck Houssen via Boost wrote: Hello,

Just to get your attention to this : https://github.com/easybuilders/easybuild-easyconfigs/issues/1065 It hasn't build with that syntax in years (edit, just saw it's from 2017), Intel's MPI does not support the -show* options family (and don't plan to). There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c...

I have no clue where the "using mpi"... is it called a directive ?.. is implemented, and if I did, I don't know how to detect we're using Intel's Implementation with bjam, nor how to pass the retrieved options, nor to what.

The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi. I remember greping mpi in the build system a few year ago, but could not find a way to touch it (it failed with no meaningful message).

Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s).

Hi Alain,

I think I’ve sent this before, but here’s the user-config.jam I use to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1.

using mpi : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx ;

using intel ;

cd libs/mpi/test ../../../b2 -d+2 toolset=intel

Forgot to mention my toolset was intel-linux, so that could be an issue.

...

...
Does this work for you? Previously you’d mentioned IMPI not working with Boost.Build, can you remind me what that failure involved?

Sure. I try to take the story from the beginning so that all the involved people can get what I see as the whole picture. Sorry for the boring, well known bits.

The MPI people thought that the easiest way to compile MPI application was through compiler wrapper like mpicc that would call the underlying compiler with the right options. Those options include, but are not limited to[*], the -I<incl path> -L<lib path> -l<libname> option. Sometime (most of the time IMO) this is not good: we want to run the compiler directly with right options (after all, we're just want to use a library at that point). So the question is how to retrieve them.

Some/most MPI implementations provides wrappers options to retrieve those. OpenMPI based implementation usually provide --showme:(compile|link) option, so adding the output of $(mpicc --showme:link) to the link flags will do the job for example. Mpich based implementation usually provides -compile_info -link_info (which work slightly differently).

Most tools like bjam/cmake/autotool/<other I guess> tries to find the compile and link flags through those options. They try -show:compile, see if it succeed, and if it does not, try --compile_info.

Intel is based on mpich, but both --compile_info/--link_info/-show* return the same output (a mix of link and compile information). Even more annoying, the -show* option, even when it fails, returns a success status on many versions[**], meaning the build tool does not detect that the option is not supported (FindMPI.cmake does some magic to detect the Intel problem on recent versions) and does not know it should try something else. As a result, with most currently deployed Intel's MPI, you need to explicitly set the include and lib path and libraries.

And of course, some MPI implementations do not have wrappers. Real world sucks.

Alain

[*]: meaning it's not so obvious that the bjam solution of providing includes path, lib search and libs is equivalent. We could be missing something, obviously not critical. But that's another story. [**]: just saw that the very last one does not, which is a good thing, maybe there is hope for humanity and reporting issues to support had some effect after all. But the behavior is still problematic, both link and compilation queries returns the same options.

...
Noel

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Belcourt, Kenneth

5:55 p.m.

New subject: [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

...

On Jul 20, 2017, at 2:34 AM, Alain Miniussi wrote: On 20/07/2017 03:53, Belcourt, Kenneth via Boost wrote:

...
On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost mailto:boost@lists.boost.org> wrote: On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote: On 19/07/2017 22:57, Edward Diener via Boost wrote: On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote: On 19/07/2017 16:26, Franck Houssen via Boost wrote:

...

...
There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c...

The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi.

...

...
Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s).

...

...
I think I’ve sent this before, but here’s the user-config.jam I use to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1.

using mpi : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx ;

using intel ;

cd libs/mpi/test ../../../b2 -d+2 toolset=intel

Does this work for you?

...

Sure.

Good. Note that the above user-config.jam doesn’t require you to set the include, lib paths or libraries. So I’m not sure why you say this:

...

As a result, with most currently deployed Intel's MPI, you need to explicitly set the include and lib path and libraries.

Can you provide me an example of an Intel MPI that requires you to explicitly set the include, lib path and libraries? Intel MPI doesn’t require you to do this any more than any other MPI implementation does.

...

And of course, some MPI implementations do not have wrappers.

Right. Cray, for example, does not provide a separate wrapper script, and it works just fine with Boost.MPI and Boost.Build. Noel

Alain Miniussi

8:58 p.m.

New subject: [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

On 20/07/2017 19:55, Belcourt, Kenneth via Boost wrote:

...

...
On Jul 20, 2017, at 2:34 AM, Alain Miniussi wrote: On 20/07/2017 03:53, Belcourt, Kenneth via Boost wrote:

...
On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost mailto:boost@lists.boost.org> wrote: On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote: On 19/07/2017 22:57, Edward Diener via Boost wrote: On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote: On 19/07/2017 16:26, Franck Houssen via Boost wrote: There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c...

The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi. Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s). I think I’ve sent this before, but here’s the user-config.jam I use to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1.

using mpi : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx ;

using intel ;

cd libs/mpi/test ../../../b2 -d+2 toolset=intel

Does this work for you? Sure. Good. Note that the above user-config.jam doesn’t require you to set the include, lib paths or libraries. So I’m not sure why you say this:

...
As a result, with most currently deployed Intel's MPI, you need to explicitly set the include and lib path and libraries. Can you provide me an example of an Intel MPI that requires you to explicitly set the include, lib path and libraries? Intel MPI doesn’t require you to do this any more than any other MPI implementation does. On linux, IMPI 5.0.1.035 and a few versions before that one (I do not have access to them anymore). I hadn't tried to move back to the simple solution since then since the ugly one still worked, but it seems to be working again with 5.0.3. It uses the same options for compilation and link (since --compile_info and -link_info boths falls back on "echo") but it works.

Maybe we should change the doc then. Alain

...

...
And of course, some MPI implementations do not have wrappers. Right. Cray, for example, does not provide a separate wrapper script, and it works just fine with Boost.MPI and Boost.Build.

Noel

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Alain Miniussi

9:05 p.m.

New subject: [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

On 20/07/2017 22:58, Alain Miniussi wrote:

...

On 20/07/2017 19:55, Belcourt, Kenneth via Boost wrote:

...
...
...
On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost mailto:boost@lists.boost.org> wrote: On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote: On 19/07/2017 22:57, Edward Diener via Boost wrote: On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote: On 19/07/2017 16:26, Franck Houssen via Boost wrote: There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c...

The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi. Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s). I think I’ve sent this before, but here’s the user-config.jam I use to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1.

using mpi : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx ;

using intel ;

cd libs/mpi/test ../../../b2 -d+2 toolset=intel

Does this work for you? Sure. Good. Note that the above user-config.jam doesn’t require you to set

On Jul 20, 2017, at 2:34 AM, Alain Miniussi wrote: On 20/07/2017 03:53, Belcourt, Kenneth via Boost wrote: the include, lib paths or libraries. So I’m not sure why you say this:

...
As a result, with most currently deployed Intel's MPI, you need to explicitly set the include and lib path and libraries. Can you provide me an example of an Intel MPI that requires you to explicitly set the include, lib path and libraries? Intel MPI doesn’t require you to do this any more than any other MPI implementation does. On linux, IMPI 5.0.1.035 and a few versions before that one (I do not have access to them anymore). I hadn't tried to move back to the simple solution since then since the ugly one still worked, but it seems to be working again with 5.0.3. It uses the same options for compilation and link (since --compile_info and -link_info boths falls back on "echo") but it works.

FWIW, I guess the problem was that, from ???? to 5.0.1: [alainm@gurney ~]$ mpiicc -show:compile icc: command line warning #10006: ignoring unknown option '-show:compile' /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/crt1.o: In function `_start': (.text+0x20): undefined reference to `main' [alainm@gurney ~]$ echo $? 0 [alainm@gurney ~]$ While, starting 5.0.3 (at least): [alainm@tagir ~]$ mpiicc -show:compile icc: command line warning #10006: ignoring unknown option '-show:compile' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crt1.o: dans la fonction « _start »: (.text+0x20): référence indéfinie vers « main » [alainm@tagir ~]$ echo $? 1 [alainm@tagir ~]$ Still: [alainm@tagir ~]$ mpiicc -compile_info icc -I/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/include -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread [alainm@tagir ~]$ mpiicc -link_info icc -I/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/include -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread [alainm@tagir ~]$ But it works, so...

...

Maybe we should change the doc then.

Alain

...
...
And of course, some MPI implementations do not have wrappers. Right. Cray, for example, does not provide a separate wrapper script, and it works just fine with Boost.MPI and Boost.Build.

Noel

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Belcourt, Kenneth

9:22 p.m.

New subject: [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

...

On Jul 20, 2017, at 3:05 PM, Alain Miniussi wrote:

On 20/07/2017 22:58, Alain Miniussi wrote:

...
On 20/07/2017 19:55, Belcourt, Kenneth via Boost wrote:

...
...
On Jul 20, 2017, at 2:34 AM, Alain Miniussi wrote: On 20/07/2017 03:53, Belcourt, Kenneth via Boost wrote:

...
On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost mailto:boost@lists.boost.org> wrote: On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote: On 19/07/2017 22:57, Edward Diener via Boost wrote: On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote: On 19/07/2017 16:26, Franck Houssen via Boost wrote: There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c...

The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi. Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s). I think I’ve sent this before, but here’s the user-config.jam I use to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1.

using mpi : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx ;

using intel ;

cd libs/mpi/test ../../../b2 -d+2 toolset=intel

Does this work for you? Sure. Good. Note that the above user-config.jam doesn’t require you to set the include, lib paths or libraries. So I’m not sure why you say this:

...
As a result, with most currently deployed Intel's MPI, you need to explicitly set the include and lib path and libraries. Can you provide me an example of an Intel MPI that requires you to explicitly set the include, lib path and libraries? Intel MPI doesn’t require you to do this any more than any other MPI implementation does. On linux, IMPI 5.0.1.035 and a few versions before that one (I do not have access to them anymore). I hadn't tried to move back to the simple solution since then since the ugly one still worked, but it seems to be working again with 5.0.3. It uses the same options for compilation and link (since --compile_info and -link_info boths falls back on "echo") but it works.

FWIW, I guess the problem was that, from ???? to 5.0.1: [alainm@gurney ~]$ mpiicc -show:compile icc: command line warning #10006: ignoring unknown option '-show:compile' /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/crt1.o: In function `_start': (.text+0x20): undefined reference to `main' [alainm@gurney ~]$ echo $? 0 [alainm@gurney ~]$

While, starting 5.0.3 (at least): [alainm@tagir ~]$ mpiicc -show:compile icc: command line warning #10006: ignoring unknown option '-show:compile' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crt1.o: dans la fonction « _start »: (.text+0x20): référence indéfinie vers « main » [alainm@tagir ~]$ echo $? 1 [alainm@tagir ~]$

Still: [alainm@tagir ~]$ mpiicc -compile_info icc -I/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/include -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread [alainm@tagir ~]$ mpiicc -link_info icc -I/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/include -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread [alainm@tagir ~]$

But it works, so…

Good. I agree that it can be ugly and relies on the compiler ignoring passed linker flags (and the linker ignoring compiler flags), and other less than ideal implementation details. To me, it's most important that it works correctly and it sounds like it does for you. Please let me know if you hit roadblocks.

...

Maybe we should change the doc then.

Fair enough. Since we’re both here, how do we address PR 46 (Winter is coming)? I now have HPC C++11 mostly compliant compilers though we still see significant performance degradataion (20-30% slower runtimes) building with C++11, than with C++98. Should we start by cherry-picking those commits that don’t require C++11 and try to move Boost.MPI to require C++11 for Boost 1.66? What are your thoughts on this?

Alain Miniussi

10:49 p.m.

New subject: [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

On 20/07/2017 23:22, Belcourt, Kenneth via Boost wrote:

...

...
On Jul 20, 2017, at 3:05 PM, Alain Miniussi wrote:

On 20/07/2017 22:58, Alain Miniussi wrote:

...
On 20/07/2017 19:55, Belcourt, Kenneth via Boost wrote:

...
...
On Jul 20, 2017, at 2:34 AM, Alain Miniussi wrote: On 20/07/2017 03:53, Belcourt, Kenneth via Boost wrote:

...
On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost mailto:boost@lists.boost.org> wrote: On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote: On 19/07/2017 22:57, Edward Diener via Boost wrote: On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote: On 19/07/2017 16:26, Franck Houssen via Boost wrote: There is a specific piece of doc for that in: http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c...

The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi. Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s). I think I’ve sent this before, but here’s the user-config.jam I use to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1.

using mpi : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx ;

using intel ;

cd libs/mpi/test ../../../b2 -d+2 toolset=intel

Does this work for you? Sure. Good. Note that the above user-config.jam doesn’t require you to set the include, lib paths or libraries. So I’m not sure why you say this:

...
As a result, with most currently deployed Intel's MPI, you need to explicitly set the include and lib path and libraries. Can you provide me an example of an Intel MPI that requires you to explicitly set the include, lib path and libraries? Intel MPI doesn’t require you to do this any more than any other MPI implementation does. On linux, IMPI 5.0.1.035 and a few versions before that one (I do not have access to them anymore). I hadn't tried to move back to the simple solution since then since the ugly one still worked, but it seems to be working again with 5.0.3. It uses the same options for compilation and link (since --compile_info and -link_info boths falls back on "echo") but it works.

FWIW, I guess the problem was that, from ???? to 5.0.1: [alainm@gurney ~]$ mpiicc -show:compile icc: command line warning #10006: ignoring unknown option '-show:compile' /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/crt1.o: In function `_start': (.text+0x20): undefined reference to `main' [alainm@gurney ~]$ echo $? 0 [alainm@gurney ~]$

While, starting 5.0.3 (at least): [alainm@tagir ~]$ mpiicc -show:compile icc: command line warning #10006: ignoring unknown option '-show:compile' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crt1.o: dans la fonction « _start »: (.text+0x20): référence indéfinie vers « main » [alainm@tagir ~]$ echo $? 1 [alainm@tagir ~]$

Still: [alainm@tagir ~]$ mpiicc -compile_info icc -I/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/include -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread [alainm@tagir ~]$ mpiicc -link_info icc -I/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/include -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread [alainm@tagir ~]$

But it works, so… Good. I agree that it can be ugly and relies on the compiler ignoring passed linker flags (and the linker ignoring compiler flags), and other less than ideal implementation details. To me, it's most important that it works correctly and it sounds like it does for you. Please let me know if you hit roadblocks.

...
Maybe we should change the doc then. Fair enough.

Since we’re both here, how do we address PR 46 (Winter is coming)? I now have HPC C++11 mostly compliant compilers though we still see significant performance degradataion (20-30% slower runtimes) building with C++11, than with C++98. Should we start by cherry-picking those commits that don’t require C++11 and try to move Boost.MPI to require C++11 for Boost 1.66?

What are your thoughts on this? I do build both with and without c++11 activated on the current distrib and master (intel 15-17, and some g++). So could you (maybe through a specific issue on github) provide me with the error messages/compiler version ? I'd like to give it a try, maybe we don't have to choose for 1.66.

Belcourt, Kenneth

11:22 p.m.

New subject: [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

...

On Jul 20, 2017, at 4:49 PM, Alain Miniussi wrote: On 20/07/2017 23:22, Belcourt, Kenneth via Boost wrote:

...
...
On Jul 20, 2017, at 3:05 PM, Alain Miniussi wrote:

On 20/07/2017 22:58, Alain Miniussi wrote:

...
On 20/07/2017 19:55, Belcourt, Kenneth via Boost wrote:

...
...
On Jul 20, 2017, at 2:34 AM, Alain Miniussi wrote: On 20/07/2017 03:53, Belcourt, Kenneth via Boost wrote: > On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost mailto:boost@lists.boost.org> wrote: > On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote: > On 19/07/2017 22:57, Edward Diener via Boost wrote: > On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote: > On 19/07/2017 16:26, Franck Houssen via Boost wrote: > There is a specific piece of doc for that in: > http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c... > > The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html. > Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi. > Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s). > I think I’ve sent this before, but here’s the user-config.jam I use to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1. > > using mpi > : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx > ; > > using intel > ; > > cd libs/mpi/test > ../../../b2 -d+2 toolset=intel > > Does this work for you? Sure. Good. Note that the above user-config.jam doesn’t require you to set the include, lib paths or libraries. So I’m not sure why you say this:

...
As a result, with most currently deployed Intel's MPI, you need to explicitly set the include and lib path and libraries. Can you provide me an example of an Intel MPI that requires you to explicitly set the include, lib path and libraries? Intel MPI doesn’t require you to do this any more than any other MPI implementation does. On linux, IMPI 5.0.1.035 and a few versions before that one (I do not have access to them anymore). I hadn't tried to move back to the simple solution since then since the ugly one still worked, but it seems to be working again with 5.0.3. It uses the same options for compilation and link (since --compile_info and -link_info boths falls back on "echo") but it works.

FWIW, I guess the problem was that, from ???? to 5.0.1: [alainm@gurney ~]$ mpiicc -show:compile icc: command line warning #10006: ignoring unknown option '-show:compile' /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/crt1.o: In function `_start': (.text+0x20): undefined reference to `main' [alainm@gurney ~]$ echo $? 0 [alainm@gurney ~]$

While, starting 5.0.3 (at least): [alainm@tagir ~]$ mpiicc -show:compile icc: command line warning #10006: ignoring unknown option '-show:compile' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crt1.o: dans la fonction « _start »: (.text+0x20): référence indéfinie vers « main » [alainm@tagir ~]$ echo $? 1 [alainm@tagir ~]$

Still: [alainm@tagir ~]$ mpiicc -compile_info icc -I/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/include -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread [alainm@tagir ~]$ mpiicc -link_info icc -I/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/include -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread [alainm@tagir ~]$

But it works, so… Good. I agree that it can be ugly and relies on the compiler ignoring passed linker flags (and the linker ignoring compiler flags), and other less than ideal implementation details. To me, it's most important that it works correctly and it sounds like it does for you. Please let me know if you hit roadblocks.

...
Maybe we should change the doc then. Fair enough.

Since we’re both here, how do we address PR 46 (Winter is coming)? I now have HPC C++11 mostly compliant compilers though we still see significant performance degradataion (20-30% slower runtimes) building with C++11, than with C++98. Should we start by cherry-picking those commits that don’t require C++11 and try to move Boost.MPI to require C++11 for Boost 1.66?

What are your thoughts on this? I do build both with and without c++11 activated on the current distrib and master (intel 15-17, and some g++). So could you (maybe through a specific issue on github) provide me with the error messages/compiler version ? I'd like to give it a try, maybe we don't have to choose for 1.66.

Sure enough, I’ll do that.

2670

Age (days ago)

2671

Last active (days ago)

List overview

Download

13 comments

4 participants

participants (4)

Alain Miniussi
Belcourt, Kenneth
Edward Diener
Franck Houssen