Aleksey, Many thanks for your detailed reply. I think the second scenario with pre-generated headers convinces me more, since our compilation times are not optimal at all :(. But as you stated in the book: more work for compiler means less work at runtime. These were not exactly the words, but I can remember the meaning. I have one more question: I tried to use the preprocess_map.py script, but encounter some problems. Running it on Windows with Python 2.5 causes the following output: D:\projects\libraries\boost_1_33_0\libs\mpl\preprocessed>preprocess_map.py Usage: preprocess_map.py <mode> <boost_root> [<source_file>] Purpose: updates preprocessed version(s) of the header(s) in "boost\mpl\map\aux_\preprocessed" directory Example: the following command will re-generate and update all 'apply.hpp' headers: preprocess_map.py all f:\cvs\boost apply.cpp How do I submit here the max map size? Or did you mean that I should write a similar script, which pregenerates headers by my own? If I look inside of this script, there is one call to main which passes 3 parameters, but probably this does not work. If I pass parameters as in this example, I get an error that gcc is not installed. Ok, I will install it and put it into my PATH env var, but still the question, how do specify the size of max map? Is there any way to include pre-generated headers for 150 elements into the next boost release? Probably the unnumbered mpl::map template could go until 50 types to save compilation time, but if someone wishes to go above, he/she could use numbered map templates like map150. The problem, why I am going to suggest it, is that if someone pre-generates the header and updates the version, it is possibly, that pre-generated headers will be removed and project maintainer (not usually programmer who generated these headers) will suddenly get compiler errors and will have to repeat this procedure again. And some other question: Dave said, that mpl::map had some bug fixes from the 1.33 release to upcoming 1.34. Will these preprocessing scripts break the changes? Many thanks for your effort and time, Ovanes -----Original Message----- From: Aleksey Gurtovoy [mailto:agurtovoy@meta-comm.com] Sent: Saturday, November 04, 2006 3:29 AM To: boost-users@lists.boost.org Cc: Ovanes Markarian; dave@boost-consulting.com Subject: Re: [mpl::map] generating map for 150 types Hi Ovanes,

I need to define a map with int_<1...150>->Type1...150 pairs.

Im source file I define:

#define BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS #define BOOST_MPL_LIMIT_MAP_SIZE 150

#include <boost/mpl/map.hpp>

A word of caution: redefining BOOST_MPL_LIMIT_MAP_SIZE is not recommended. Doing so globally will make you pay for what you probably don't use (unless all present and future 'map' usages in your project demand the 150-elements limit), and doing so within a translation unit will lead to an ODR violation. If you need to construct a sequence beyond the default size, use its numbered form (http://www.boost.org/libs/mpl/doc/refmanual/variadic-sequence.html).

When compiling I get an error in the file: ...\boost\include\boost-1_33_1\boost\mpl\map.hpp on line: 36 fatal error C1083: Cannot open include file: 'boost/mpl/map/map150.hpp': No such file or directory

Development environment is Visual C++ 8 Express Edition

The help states:

[...] What the BOOST_MPL_LIMIT_MAP_SIZE docs don't say is that there is a default upper limit of 50 elements for the macro's value: #define BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS #define BOOST_MPL_LIMIT_MAP_SIZE 50 #include <boost/mpl/map.hpp> // compiles This upper limit is overridable as well, but since touching BOOST_MPL_LIMIT_MAP_SIZE is not recommended anyway, I'll instead demonstrate how to bump the number of map's numbered forms up to 150. The simpliest way would be just this: #include <boost/mpl/map/map50.hpp> #include <boost/preprocessor/iterate.hpp> namespace boost { namespace mpl { # define BOOST_PP_ITERATION_PARAMS_1 \ (3,(51, 150, <boost/mpl/map/aux_/numbered.hpp>)) # include BOOST_PP_ITERATE() }} // use map150<....> with the caveat that the extra 100 map templates are generated on the fly using the preprocessor, which isn't ideal from the compilation time standpoint. Alternatively, you can pre-generate these extra templates once and for all using the '$BOOST_ROOT/libs/mpl/preprocessed/preprocess_map.py' script and the existing numbered map files in the '$BOOST_ROOT/boost/mpl/map/' and '$BOOST_ROOT/libs/mpl/preprocessed/map/' subdirectories as an example. HTH, -- Aleksey Gurtovoy MetaCommunications Engineering

Ovanes Markarian writes:

Many thanks for your detailed reply. I think the second scenario with pre-generated headers convinces me more, since our compilation times are not optimal at all :(.

I'd still suggest to try out the first version, before engaging into this particlular optimization, to see whether there are other showstoppers to what you are trying to do: the library has not been throughly tested with the sequence limits this high, and, depending on the particular compiler, you might run into internal structure overflows, ICEs and the likes, which may force you to adpot a different approach.

I have one more question: I tried to use the preprocess_map.py script, but encounter some problems.

[...]

How do I submit here the max map size? Or did you mean that I should write a similar script, which pregenerates headers by my own?

Sorry for not being specific enough about this; the script generates the numbered 'map' forms basing entirely on the corresponding source files / headers in the '$BOOST_ROOT/libs/mpl/preprocessed/map/' and '$BOOST_ROOT/boost/mpl/map/' directories. To bump up the current 50 elements limit to, say, 150 you need to provide the corresponding 'mapN.cpp'/'mapN.hpp' files for N up to 150, using the existing ones as an example. In the simpliest case, assuming that you don't want the same level of headers granularity the library adheres to and are okay with jumping from 50 to 150 in one step, '$BOOST_ROOT/boost/mpl/map/map150.hpp' would look like this: #ifndef BOOST_MPL_MAP_MAP150_HPP_INCLUDED #define BOOST_MPL_MAP_MAP150_HPP_INCLUDED #if !defined(BOOST_MPL_PREPROCESSING_MODE) # include <boost/mpl/map/map50.hpp> #endif #include <boost/mpl/aux_/config/use_preprocessed.hpp> #if !defined(BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS) \ && !defined(BOOST_MPL_PREPROCESSING_MODE) # define BOOST_MPL_PREPROCESSED_HEADER map150.hpp # include <boost/mpl/map/aux_/include_preprocessed.hpp> #else # include <boost/preprocessor/iterate.hpp> namespace boost { namespace mpl { # define BOOST_PP_ITERATION_PARAMS_1 \ (3,(51, 150, <boost/mpl/map/aux_/numbered.hpp>)) # include BOOST_PP_ITERATE() }} #endif // BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS #endif // BOOST_MPL_MAP_MAP150_HPP_INCLUDED ... and the corresponding 'map150.cpp' like this: #define BOOST_MPL_PREPROCESSING_MODE #include <boost/config.hpp> #include <boost/mpl/map/map150.hpp>

If I look inside of this script, there is one call to main which passes 3 parameters, but probably this does not work. If I pass parameters as in this example, I get an error that gcc is not installed.

Oh, right, sorry I forgot to mention it.

Ok, I will install it and put it into my PATH env var, but still the question, how do specify the size of max map?

Please see the above; the '<mode>' and '<boost_root>' are the only arguments you need to specify explicitly.

Is there any way to include pre-generated headers for 150 elements into the next boost release?

There is a cost of doing so (http://article.gmane.org/gmane.comp.lib.boost.user/7057): Increasing the default limit to, let's say, 100 elements means plus ~120 KB to the size of the distribution, and I'm not sure the need is widespread enough to make everybody pay for it. This has been a while ago, but my concern didn't go away :).

Probably the unnumbered mpl::map template could go until 50 types to save compilation time, but if someone wishes to go above, he/she could use numbered map templates like map150. The problem, why I am going to suggest it, is that if someone pre-generates the header and updates the version, it is possibly, that pre-generated headers will be removed and project maintainer (not usually programmer who generated these headers) will suddenly get compiler errors and will have to repeat this procedure again.

IMHO this is a general issue: how do you manage a third-party library sources in presence of a need to make local patches to them, and keep these patches from getting overwritten accidentally? (Unless you always work with the HEAD / latest & greatest sources _and_ have a direct channel to the library maintainer, local patches are inevitable). Our answer to this question here at work is: 1) Always maintain a patch directory alongside with the root directory for the library sources, e.g.: boost_root/... boost_patches/... 2) Make the patch directory precede the original sources in the list of include paths. Do this and #1 at the very moment you import the library in your repository. 3) When a need for a patch occurs: a) Isomorphically copy the affected / add the new files into the patch directory, leaving the originals untouched, and patch the copies. If you need to delete the file, don't, but, depending on the use case, override it with an empty one or the one containing an #error directive / redirecting #include. b) Rebuild the library if needed (making sure to enforce #2). 4) When upgrading to a new version of the library, diff the patches against the new originals and delete/keep/adjust them depending on the results.

And some other question: Dave said, that mpl::map had some bug fixes from the 1.33 release to upcoming 1.34. Will these preprocessing scripts break the changes?

No; I just double-checked, and all the fixes are present in the original headers used to generate the preprocessed versions. HTH, -- Aleksey Gurtovoy MetaCommunications Engineering