[bcp] Bcp namespace rename causes compile errors on multiple libraries (type_traits, range, spirit, foreach, regex, iostreams) in 1.43
Has bcp namespace renaming been tested against all Boost libraries in 1.43? I'm getting compile errors in (or caused by) the following libraries: type_traits, range, spirit, foreach, regex, iostreams (more details later in this message). The compile errors are from a full bjam build of Boost. I haven't compiled tests or built against my work project code, so I suspect there's more errors in header-only libraries. I'm using g++ 4.1.2 on Linux (although these errors are not compiler specific - I'll also be compiling with VC++ 8.0 at some point). All of these errors are easily fixed, and are "boost" text substitution problems. I was able to get all libraries to build after my changes, but I would rather not use hacked-up source (I prefer to work with the Boost developers and install non-modified Boost source into our Clearcase COTS VOB). However, with these (obvious) compile errors, I'm wondering if the bcp namespace facilities are being tested with all Boost libraries? If not, can I work with John Maddock or someone to set up a test process? Besides doing a full extract / rename and build of the libraries, as many examples and tests as possible should also be compiled to find errors in header-only libraries not brought in to the library source build. Of course it's possible all of the errors I've found have been introduced with new code in 1.43 (which still brings up the need to do a full bcp test before a Boost release), or that I somehow incorrectly used bcp, or there's fixes that haven't been applied to the mainline in SVN. My use case is a perfect fit for the bcp namespace facilities - my work team delivers an SDK (which links to and uses various Boost libraries) to another group which also links to and uses Boost libraries in their own application (everything builds into the same executable). Currently our work team is constrained to using the same Boost version, but we'd like to break this constraint (in particular, for other groups to use our SDK). We use (or will use) enough Boost libraries that bcp on the ones we care about pulls in about 80% of Boost, so I went ahead and ran bcp on all of Boost. Details of the compile failures: -- In type_traits: integral_promotion.hpp, promote.hpp, floating_point_promotion.hpp, header guard probs - original code #ifndef FILE_boost_type_traits_integral_promotion_hpp_INCLUDED #define FILE_boost_type_traits_integral_promotion_hpp_INCLUDED After running bcp with "MyProj_Boost" for the namespace, the header guard is: #ifndef FILE_boost_type_traits_integral_promotion_hpp_INCLUDED #define FILE_MyProj_Boost_type_traits_integral_promotion_hpp_INCLUDED -- In range/config.hpp there's two macros where bcp namespace substitution is incorrectly being applied to the "boost" prefix in boost_range_array: #define BOOST_RANGE_ARRAY_REF() (boost_range_array) #define BOOST_RANGE_ARRAY_REF() (&boost_range_array) Similarly, in range/detail/implementation_help.hpp, the following lines have incorrect substitution: boost_range_silence_warning( boost_range_array ); -- In Phoenix and Wave, there's a bit of namespace substitution problems, where "namespace phoenix" (in multiple files, such as spirit/home/classic/phoenix) is being replaced with "MyProjectphoenix", and in wave/grammars/cpp_intlit_grammer.hpp and cpp_chlit_grammar.hpp there's "namespace phx=phoenix". -- In regex/v4/regex_traits.hpp: BOOST_MPL_HAS_XXX_TRAIT_DEF(boost_extensions_tag) The "boost" prefix in boost_extensions_tag is improperly substituted. -- In foreach.hpp, "boost_foreach_argument_dependent_lookup_hack_value" is not substituted in the enum, but in macros where the enum is used, the substitution is occurring. In iostreams/close.hpp, close_boost_stream on line 143 is being improperly substituted. --- As already mentioned, there's easy changes for each of these. But, is a better answer to make the bcp search and substitution smarter? (I have no idea how easy or hard that would be.) Cliff
Has bcp namespace renaming been tested against all Boost libraries in 1.43? I'm getting compile errors in (or caused by) the following libraries: type_traits, range, spirit, foreach, regex, iostreams (more details later in this message). The compile errors are from a full bjam build of Boost. I haven't compiled tests or built against my work project code, so I suspect there's more errors in header-only libraries.
Thanks for the report - the namespace substitution was tested, but we don't have automated tests for it - in fact full tests would be rather difficult to contrive, because as well as requiring that the modified code compile on it's own, we should also require that two different Boost versions work side by side.
I'm using g++ 4.1.2 on Linux (although these errors are not compiler specific - I'll also be compiling with VC++ 8.0 at some point).
All of these errors are easily fixed, and are "boost" text substitution problems. I was able to get all libraries to build after my changes, but I would rather not use hacked-up source (I prefer to work with the Boost developers and install non-modified Boost source into our Clearcase COTS VOB).
However, with these (obvious) compile errors, I'm wondering if the bcp namespace facilities are being tested with all Boost libraries? If not, can I work with John Maddock or someone to set up a test process? Besides doing a full extract / rename and build of the libraries, as many examples and tests as possible should also be compiled to find errors in header-only libraries not brought in to the library source build.
If you have the facilities to run a full test run after bcp substitution that would be great - the results would need to be compared to the results obtained without the substitution.
Of course it's possible all of the errors I've found have been introduced with new code in 1.43 (which still brings up the need to do a full bcp test before a Boost release), or that I somehow incorrectly used bcp, or there's fixes that haven't been applied to the mainline in SVN.
I suspect that the heuristics used aren't quite up to the job: I don't remember the details, but simply performing a global search and replace for "boost" doesn't quite work either :-( I will look into and try and fix all the issues you identify as soon as I can, Thanks for reporting these! John.
You may give a try to this script I written once and use for my needs: The script: http://art-blog.no-ip.info/files/rename.py Some background: http://stackoverflow.com/questions/836875 It would work as it replaces both ".*BOOST.*" and ".*boost.*" tokens and does this for macros as well. There is a small but important difference - it renames also includes so you do includes as #include <my_boost_namespace/shared_ptr.hpp> Instead of #include <boost/shared_ptr.hpp> So this actually allows to use two version of boost in same unit. Important notice: Boost.Regex defines extern "C" symbols for POSIX regex API as like regexec(A|W) and regcomp(A|W) without boost prefix. So you should still be very careful about using two versions of regex in same unit. Artyom
Thanks for the report - the namespace substitution was tested, but we don't have automated tests for it - in fact full tests would be rather difficult to contrive, because as well as requiring that the modified code compile on it's own, we should also require that two different Boost versions work side by side.
Since the problem I ran into were all compile failures, I'm guessing it's relatively recent code added to the various Boost libraries. And while I understand a full "system level" test takes some effort, I would think it's not too hard to do a "perform bcp on all libraries, then perform the general bjam that builds everything" (in essence, this is what I did). This would at least verify that everything will compile after the extract. But I fully understand the limited time and resources that are available. :)
If you have the facilities to run a full test run after bcp substitution that would be great - the results would need to be compared to the results obtained without the substitution.
That would definitely be good - I have the facilities on at least a couple of common compilers / platforms, but my available time might be limited. Let's continue the discussion, and when I can get to this again (I'm guessing in about two weeks), I'll see what I can do. (For now, my work project will continue using 1.38.) Independent of any contribution I can make, defining the verification process is a good step (or just warning users of the bcp namespace renaming facility that it has only been tested up to some particular date or milestone). Since the Boost codebase is constantly changing, the "verification" milestone will be moving as well. The release process is probably "costly" enough already, so there might be reluctance to add "perform the full bcp namespace rename testing step to verify that nothing has broken" into the process. And of course, if something is "broken", then there's the working with the developers to change the code (and delaying the release). Or, if the bcp "search and replace" can be made smart enough to work around the source code issues found by me (and similar issues in the future), that might be a better approach. A full "compare test results" would still be a good step, but at least it would be expected that compile failures won't happen.
I will look into and try and fix all the issues you identify as soon as I can,
Thanks! Keep me in the discussions (if the coordination continues on the Boost dev mailing list, I'll follow it there).
Thanks for reporting these!
No prob, and as already mentioned, I'll help out as much as time allows. Cliff
I will look into and try and fix all the issues you identify as soon as I can,
Thanks! Keep me in the discussions (if the coordination continues on the Boost dev mailing list, I'll follow it there).
I believe bcp on current Trunk is now doing the right thing by all the Boost libraries. HTH, John.
John Maddock, 6/10/2010: I believe bcp on current Trunk is now doing the right thing by all the Boost libraries.
Hi John - wanted to report that bjam builds for the Boost libraries now work on both VS2005 and g++ 4.1.2 using the namespace rename (and alias) features of bcp. This was using a SVN snapshot from a couple of weeks ago. I just downloaded the beta release candidate and will try a more in-depth test. Hopefully I can do this before the 1.44 release is ready. What I've done (with the snapshot from a couple of weeks ago) and what I still would like to do with the beta release candidate: Done: Performed bcp extract and namespace rename / alias on all Boost libraries. Invoked bjam in Boost root and built (the small subset) of libraries that require a build, VS2005 (Windows), g++ 4.12 (Linux). Would like to do: Perform test and / or example builds for all Boost libraries, VS2005, g++ 4.1.2 (this would hopefully catch any header-only library namespace problems). Will do: Build our own application with the renamed namespaces (we are currently using only a small subset of Boost, but that will change soon). I'll let you know right away if I run into problems. There's a few enhancements that would be nice for the bcp namespace features, but I'll post those in a separate e-mail in a few minutes. Thanks for the updates! Cliff
participants (3)
-
Artyom -
Cliff Green -
John Maddock