What is the situation of the spirit library in the trunk? The serialization library now depends upon the latest spirit to parse xml archives. But now serialization can't be built for a number of compilers which previously could handle this library. This was not a surprise to me as its not unusual to require some tweaks to get this to work. But now I'm surprised that this hasn't resolved the issue. The newer version of spirit has been around for at least a year, and the serializaiton library xml grammar is very simple compared to other spirit applications so I didn't anticipate any major problems - but it seems that we now ahve them. Could spirit library developers take a look at this and give us an idea of what we should do about this? Is anyone working on this? Should we just roll back and try again before the next release? Robert Ramey
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 26 Oct 2010 21:33:54 -0800 "Robert Ramey" <ramey@rrsd.com> wrote:
What is the situation of the spirit library in the trunk?
The serialization library now depends upon the latest spirit to parse xml archives.
But now serialization can't be built for a number of compilers which previously could handle this library. This was not a surprise to me as its not unusual to require some tweaks to get this to work. But now I'm surprised that this hasn't resolved the issue. The newer version of spirit has been around for at least a year, and the serializaiton library xml grammar is very simple compared to other spirit applications so I didn't anticipate any major problems - but it seems that we now ahve them.
Could spirit library developers take a look at this and give us an idea of what we should do about this? Is anyone working on this? Should we just roll back and try again before the next release?
Ramey, I haven't really had a chance to look into this -too- much. However, I have compiled Serialization multiple times on the major development toolchains (Intel 11 on windows and linux; mingw 4.3, 4.4 and 4.5 on windows; VS 10 and VS 7.1 on windows; gcc 4.5, 4.4, 4.3, 4.2 and 4.1 on linux; clang 2.9 on linux). I have no problems on any of these compilers anymore; to my knowledge, nobody else is having issues, either (I spent a good amount of last week working on that). The only outstanding issue is the test matrix. Yes, admittedly, the grammar stuff is a bit on the slow side compile-time wise. However, I cannot reproduce these 300 second timeouts that the build machines have been reporting. I haven't had a chance to talk to the people running the relevant machines, but I am having a hard time buying that those machines are taking more than five minutes to instantiate my Spirit grammar. Let's take a look at the results from Intel's compiler. Intel is benched as being one of the slowest compilers out there, but one of the best optimizers ( http://macles.blogspot.com/2010/08/intel-atom-icc-gcc-clang.html). Compile [2010-10-27 01:49:54 UTC]: fail "/sierra/Sntools/extras/compilers/intel/Compiler/11.0/081/bin/intel64/icpc" -c -xc++ -O0 -g -w1 -inline-level=0 -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_SERIALIZATION_DYN_LINK=1 -I".." -c -o "/scratch/boost/results/boost/bin.v2/libs/serialization/build/intel-linux-11.0/debug/xml_wgrammar.o" "../libs/serialization/src/xml_wgrammar.cpp" 300 second time limit exceeded The test box is running Red Hat, and has an impressive Dual Duo-Core Intel Xeon (3Ghz) with 8Gb RAM. My machine is running an experimental Linux kernel built with Clang and full debugging symbols (not to mention my kernel was not optimized). My hardware is an Intel Core 2 Duo, and 2Gb RAM. I was running X, procmail and firefox while testing (I suspect the Red Hat server has none of those installed, much less running during tests). My times: wash@Pegasus:~/boost/status$ time icpc -c -xc++ -O0 -g -w1 -inline-level=0 -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_SERIALIZATION_DYN_LINK=1 -I".." -c -o "/scratch/boost/results/boost/bin.v2/libs/serialization/build/intel-linux-11.0/debug/xml_wgrammar.o" "../libs/serialization/src/xml_wgrammar.cpp" real 5m54.892s user 5m44.142s sys 0m1.128s I just can't imagine the Xeon being that slow, relative to my numbers... - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkzHuPQACgkQO/fqqIuE2t6aCwCcCVXmh7f507VCYbYqK4PlAQvx K+UAoK+gnnLwmlu6hmPBSVgqHzqTSkLu =Iwc9 -----END PGP SIGNATURE-----
Compile [2010-10-27 01:49:54 UTC]: fail
"/sierra/Sntools/extras/compilers/intel/Compiler/11.0/081/bin/intel64/icpc" -c -xc++ -O0 -g -w1 -inline-level=0 -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_SERIALIZATION_DYN_LINK=1 -I".." -c -o "/scratch/boost/results/boost/bin.v2/libs/serialization/build/intel-
Bryce Lelbach wrote: linux-11.0/debug/xml_wgrammar.o"
"../libs/serialization/src/xml_wgrammar.cpp"
300 second time limit exceeded
The test box is running Red Hat, and has an impressive Dual Duo-Core Intel Xeon (3Ghz) with 8Gb RAM. My machine is running an experimental Linux kernel built with Clang and full debugging symbols (not to mention my kernel was not optimized). My hardware is an Intel Core 2 Duo, and 2Gb RAM. I was running X, procmail and firefox while testing (I suspect the Red Hat server has none of those installed, much less running during tests). My times:
wash@Pegasus:~/boost/status$ time icpc -c -xc++ -O0 -g -w1 -inline-level=0 -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_SERIALIZATION_DYN_LINK=1 -I".." -c -o "/scratch/boost/results/boost/bin.v2/libs/serialization/build/intel-
linux-11.0/debug/xml_wgrammar.o"
"../libs/serialization/src/xml_wgrammar.cpp"
real 5m54.892s user 5m44.142s sys 0m1.128s
I just can't imagine the Xeon being that slow, relative to my numbers...
I don't suppose Boost tests are being run on a dedicated machine... - Volodya
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 27 Oct 2010 09:35:13 +0400 Vladimir Prus <vladimir@codesourcery.com> wrote:
Bryce Lelbach wrote:
Compile [2010-10-27 01:49:54 UTC]: fail
"/sierra/Sntools/extras/compilers/intel/Compiler/11.0/081/bin/intel64/icpc" -c -xc++ -O0 -g -w1 -inline-level=0 -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_SERIALIZATION_DYN_LINK=1 -I".." -c -o "/scratch/boost/results/boost/bin.v2/libs/serialization/build/intel- linux-11.0/debug/xml_wgrammar.o" "../libs/serialization/src/xml_wgrammar.cpp"
300 second time limit exceeded
The test box is running Red Hat, and has an impressive Dual Duo-Core Intel Xeon (3Ghz) with 8Gb RAM. My machine is running an experimental Linux kernel built with Clang and full debugging symbols (not to mention my kernel was not optimized). My hardware is an Intel Core 2 Duo, and 2Gb RAM. I was running X, procmail and firefox while testing (I suspect the Red Hat server has none of those installed, much less running during tests). My times:
wash@Pegasus:~/boost/status$ time icpc -c -xc++ -O0 -g -w1 -inline-level=0 -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_SERIALIZATION_DYN_LINK=1 -I".." -c -o "/scratch/boost/results/boost/bin.v2/libs/serialization/build/intel- linux-11.0/debug/xml_wgrammar.o" "../libs/serialization/src/xml_wgrammar.cpp"
real 5m54.892s user 5m44.142s sys 0m1.128s
I just can't imagine the Xeon being that slow, relative to my numbers...
I don't suppose Boost tests are being run on a dedicated machine...
- Volodya
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Just in case anyone misses it, I posted an RFC to the Boost.Users list about this. I think this is a good chance to discuss Spirit porting questions that may well become relevant if somebody was to decide to port Wave to Spirit 2.x someday. I am going to spend some time tommorow refactoring the grammar. I will try to get it into a state where it can pass all the tests (VACPP, Intel, MSVC) in the allocated 300 timeout on the test matrix. I still think it is good idea to consider pulling the new grammar at this point in time, but with SVN, it is a bit painful to do branching. I think if I can get the following done, we should consider keeping it in trunk: * Reduce the number of warnings with GCC and MSVC * Passes tests on VACPP, Intel and MSVC - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkzJGV8ACgkQO/fqqIuE2t7OjQCgs3e3D5+u1qVoLaCy+dE1iUE+ hIgAoMxT0irDiqrqSle+yEwB5rVKRb0n =CPxh -----END PGP SIGNATURE-----
Bryce Lelbach wrote:
Could spirit library developers take a look at this and give us an idea of what we should do about this? Is anyone working on this? Should we just roll back and try again before the next release?
Ramey, I haven't really had a chance to look into this -too- much. However, I have compiled Serialization multiple times on the major development toolchains (Intel 11 on windows and linux; mingw 4.3, 4.4 and 4.5 on windows; VS 10 and VS 7.1 on windows; gcc 4.5, 4.4, 4.3, 4.2 and 4.1 on linux; clang 2.9 on linux). I have no problems on any of these compilers anymore; to my knowledge, nobody else is having issues, either (I spent a good amount of last week working on that).
FYI, on my machine, I have trouble building with msvc7.1 and I see the test matrix has the same problem.
The only outstanding issue is the test matrix. Yes, admittedly, the grammar stuff is a bit on the slow side compile-time wise. However, I cannot reproduce these 300 second timeouts that the build machines have been reporting. I haven't had a chance to talk to the people running the relevant machines, but I am having a hard time buying that those machines are taking more than five minutes to instantiate my Spirit grammar.
Let's take a look at the results from Intel's compiler.
<snip> I understand that not everyone is on the same page here. I'm hoping that this can be reconciled soon. I notice that the almost all the spirit tests pass on these compilers - and I don't think the serialization grammar is one of the simplest ones around. How can wave be compiling and the serialization grammar not be? I'm hoping that someone from the spirit group might take a look at this. There is also the VACPP compiler - AIX IBM which also has to be addressed. I know that those on the IBM compiler team have invested a sincere effort to make sure that their compiler passes the boost tests in general and the serialization library in particular. I feel we cannot let them down. Basically, my view is that we cannot ship something that doesn't work at least as well as the previous version. Robert Ramey
On 10/27/10 3:30 PM, Robert Ramey wrote:
Bryce Lelbach wrote:
Could spirit library developers take a look at this and give us an idea of what we should do about this? Is anyone working on this? Should we just roll back and try again before the next release?
Ramey, I haven't really had a chance to look into this -too- much. However, I have compiled Serialization multiple times on the major development toolchains (Intel 11 on windows and linux; mingw 4.3, 4.4 and 4.5 on windows; VS 10 and VS 7.1 on windows; gcc 4.5, 4.4, 4.3, 4.2 and 4.1 on linux; clang 2.9 on linux). I have no problems on any of these compilers anymore; to my knowledge, nobody else is having issues, either (I spent a good amount of last week working on that).
FYI, on my machine, I have trouble building with msvc7.1 and I see the test matrix has the same problem.
The only outstanding issue is the test matrix. Yes, admittedly, the grammar stuff is a bit on the slow side compile-time wise. However, I cannot reproduce these 300 second timeouts that the build machines have been reporting. I haven't had a chance to talk to the people running the relevant machines, but I am having a hard time buying that those machines are taking more than five minutes to instantiate my Spirit grammar.
Let's take a look at the results from Intel's compiler.
<snip>
I understand that not everyone is on the same page here. I'm hoping that this can be reconciled soon. I notice that the almost all the spirit tests pass on these compilers - and I don't think the serialization grammar is one of the simplest ones around. How can wave be compiling and the serialization grammar not be? I'm hoping that someone from the spirit group might take a look at this.
There is also the VACPP compiler - AIX IBM which also has to be addressed. I know that those on the IBM compiler team have invested a sincere effort to make sure that their compiler passes the boost tests in general and the serialization library in particular. I feel we cannot let them down.
Basically, my view is that we cannot ship something that doesn't work at least as well as the previous version.
I agree. I think it's better to let it mature first. I am not testing on VC7.1 anymore, but there were some discussions a while back that this has been addressed. At any rate, I don't think it (the port) is ready for prime time yet. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 27 Oct 2010 15:00:35 +0800 Joel de Guzman <joel@boost-consulting.com> wrote:
On 10/27/10 3:30 PM, Robert Ramey wrote:
Bryce Lelbach wrote:
Could spirit library developers take a look at this and give us an idea of what we should do about this? Is anyone working on this? Should we just roll back and try again before the next release?
Ramey, I haven't really had a chance to look into this -too- much. However, I have compiled Serialization multiple times on the major development toolchains (Intel 11 on windows and linux; mingw 4.3, 4.4 and 4.5 on windows; VS 10 and VS 7.1 on windows; gcc 4.5, 4.4, 4.3, 4.2 and 4.1 on linux; clang 2.9 on linux). I have no problems on any of these compilers anymore; to my knowledge, nobody else is having issues, either (I spent a good amount of last week working on that).
FYI, on my machine, I have trouble building with msvc7.1 and I see the test matrix has the same problem.
The only outstanding issue is the test matrix. Yes, admittedly, the grammar stuff is a bit on the slow side compile-time wise. However, I cannot reproduce these 300 second timeouts that the build machines have been reporting. I haven't had a chance to talk to the people running the relevant machines, but I am having a hard time buying that those machines are taking more than five minutes to instantiate my Spirit grammar.
Let's take a look at the results from Intel's compiler.
<snip>
I understand that not everyone is on the same page here. I'm hoping that this can be reconciled soon. I notice that the almost all the spirit tests pass on these compilers - and I don't think the serialization grammar is one of the simplest ones around. How can wave be compiling and the serialization grammar not be? I'm hoping that someone from the spirit group might take a look at this.
There is also the VACPP compiler - AIX IBM which also has to be addressed. I know that those on the IBM compiler team have invested a sincere effort to make sure that their compiler passes the boost tests in general and the serialization library in particular. I feel we cannot let them down.
Basically, my view is that we cannot ship something that doesn't work at least as well as the previous version.
I agree. I think it's better to let it mature first. I am not testing on VC7.1 anymore, but there were some discussions a while back that this has been addressed. At any rate, I don't think it (the port) is ready for prime time yet.
Regards,
Agreed. I'll put some more work into this week. Hopefully it will be ready for next release (which will be, what, the main 1.45 release?). - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkzHzrYACgkQO/fqqIuE2t4u1ACglDa53IdbXuL5AZut4n5lvShh qfYAoJiJN/dQC3jyaNEmc5nJ82+EqCFM =iDd7 -----END PGP SIGNATURE-----
Bryce Lelbach wrote:
Basically, my view is that we cannot ship something that doesn't work at least as well as the previous version.
I agree. I think it's better to let it mature first. I am not testing on VC7.1 anymore, but there were some discussions a while back that this has been addressed. At any rate, I don't think it (the port) is ready for prime time yet.
Regards,
Agreed. I'll put some more work into this week. Hopefully it will be ready for next release (which will be, what, the main 1.45 release?).
OK - here is what I would like to see: a) back the upgrade to spirit/qi out of the trunk b) we'll let the trunk tests cycle a couple of days c) I'll then merge the serialization trunk into the release branch. d) we'll release boost 1.45 - I would hope this would be in the next 10 days or sor e) The spirit upgrade can then be put back into the trunk f) trunk testing can then still be used to test the new spirit/qi xml_iarchive g) hopefully the developers of spirit/qi and xml_iarchive can get all this resolved by the time 1.46 comes out. Moving the new spirit/qi to xml_iarchive to a branch won't help because the difficultities are only showing up on the test matrix machines. Robert Ramey
On 10/27/2010 09:46 AM, Robert Ramey wrote:
Moving the new spirit/qi to xml_iarchive to a branch won't help because the difficultities are only showing up on the test matrix machines.
Hi Robert - While branches are not helpful for the testing phase in themselves, they are a great tool for managing various independent changes to a library. If the xml changes were in a branch they could be easily applied and removed and re-applied to the trunk at any time. I'm constantly amazed at the lack of branches in general for boost development. They are trivial to create and make 'cherry-picking' changes very easy. Considering how we (as a community) try to use the trunk as a stable testing ground, the judicious application of branches seems the only sensible way to develop. best regards - michael -- Michael Caisse Object Modeling Designs www.objectmodelingdesigns.com
On Thu, Oct 28, 2010 at 12:03 AM, Michael Caisse <boost@objectmodelingdesigns.com> wrote:
On 10/27/2010 09:46 AM, Robert Ramey wrote:
Moving the new spirit/qi to xml_iarchive to a branch won't help because the difficultities are only showing up on the test matrix machines.
Hi Robert -
While branches are not helpful for the testing phase in themselves, they are a great tool for managing various independent changes to a library. If the xml changes were in a branch they could be easily applied and removed and re-applied to the trunk at any time.
+1 This is so much easier in Git too, but I don't want to bring that into the discussion (which I just did, heh :P).
I'm constantly amazed at the lack of branches in general for boost development. They are trivial to create and make 'cherry-picking' changes very easy. Considering how we (as a community) try to use the trunk as a stable testing ground, the judicious application of branches seems the only sensible way to develop.
Actually, in subversion, branches are painful. Especially in a large repo/project like Boost. Maybe that's why branches aren't so popular. Just my speculation though. ;) -- Dean Michael Berris deanberris.com
On Wed, Oct 27, 2010 at 3:00 PM, Joel de Guzman <joel@boost-consulting.com> wrote:
On 10/27/10 3:30 PM, Robert Ramey wrote:
Basically, my view is that we cannot ship something that doesn't work at least as well as the previous version.
I agree. I think it's better to let it mature first. I am not testing on VC7.1 anymore, but there were some discussions a while back that this has been addressed. At any rate, I don't think it (the port) is ready for prime time yet.
+1. My suggestion would be (if it's even possible) to move the spirit 2.x port to a branch other than trunk, and then just merge that back into trunk when the parser has been "proven" after 1.45. My thinking is that 1.45 should go out with the urgent fixes to the issues its facing with 1.44 WRT serialization, then schedule the spirit port for a subsequent release. I don't think there's a rush here unless it's an important fix that addresses a broken feature. HTH -- Dean Michael Berris deanberris.com
Bryce Lelbach wrote:
Could spirit library developers take a look at this and give us an idea of what we should do about this? Is anyone working on this? Should we just roll back and try again before the next release?
Ramey, I haven't really had a chance to look into this -too- much. However, I have compiled Serialization multiple times on the major development toolchains (Intel 11 on windows and linux; mingw 4.3, 4.4 and 4.5 on windows; VS 10 and VS 7.1 on windows; gcc 4.5, 4.4, 4.3, 4.2 and 4.1 on linux; clang 2.9 on linux). I have no problems on any of these compilers anymore; to my knowledge, nobody else is having issues, either (I spent a good amount of last week working on that).
FYI, on my machine, I have trouble building with msvc7.1 and I see the test matrix has the same problem.
Make sure you have SP1 installed. Without that service pack the compiler is worthless.
The only outstanding issue is the test matrix. Yes, admittedly, the grammar stuff is a bit on the slow side compile-time wise. However, I cannot reproduce these 300 second timeouts that the build machines have been reporting. I haven't had a chance to talk to the people running the relevant machines, but I am having a hard time buying that those machines are taking more than five minutes to instantiate my Spirit grammar.
Let's take a look at the results from Intel's compiler.
<snip>
I understand that not everyone is on the same page here. I'm hoping that this can be reconciled soon. I notice that the almost all the spirit tests pass on these compilers - and I don't think the serialization grammar is one of the simplest ones around. How can wave be compiling and the serialization grammar not be?
Wave uses Spirit.Classic. It has not been ported/rewritten yet.
I'm hoping that someone from the spirit group might take a look at this.
There is also the VACPP compiler - AIX IBM which also has to be addressed. I know that those on the IBM compiler team have invested a sincere effort to make sure that their compiler passes the boost tests in general and the serialization library in particular. I feel we cannot let them down.
Basically, my view is that we cannot ship something that doesn't work at least as well as the previous version.
Agreed. As both XML grammars (Spirit.Classic and Spirit.Qi) are compatible, you can always decide to leave both of them in the code base, switching back for older/non-conforming compilers. Regards Hartmut --------------- http://boost-spirit.com
participants (7)
-
Bryce Lelbach -
Dean Michael Berris -
Hartmut Kaiser -
Joel de Guzman -
Michael Caisse -
Robert Ramey -
Vladimir Prus