[mpl::map] generating map for 150 types
Hi!
I need to define a map with int_<1...150>->Type1...150 pairs.
Im source file I define:
#define BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS
#define BOOST_MPL_LIMIT_MAP_SIZE 150
#include
Hi Ovanes,
I need to define a map with int_<1...150>->Type1...150 pairs.
Im source file I define:
#define BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS #define BOOST_MPL_LIMIT_MAP_SIZE 150
#include
A word of caution: redefining BOOST_MPL_LIMIT_MAP_SIZE is not recommended. Doing so globally will make you pay for what you probably don't use (unless all present and future 'map' usages in your project demand the 150-elements limit), and doing so within a translation unit will lead to an ODR violation. If you need to construct a sequence beyond the default size, use its numbered form (http://www.boost.org/libs/mpl/doc/refmanual/variadic-sequence.html).
When compiling I get an error in the file: ...\boost\include\boost-1_33_1\boost\mpl\map.hpp on line: 36 fatal error C1083: Cannot open include file: 'boost/mpl/map/map150.hpp': No such file or directory
Development environment is Visual C++ 8 Express Edition
The help states: BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS is an boolean configuration macro regulating library's internal use of preprocessed headers. When defined, it instructs the MPL to discard the pre-generated headers found in boost/mpl/aux_/preprocessed directory and use preprocessor metaprogramming techniques to generate the necessary versions of the library components on the fly.
In this implementation of the library, the macro is not defined by default. To change the default configuration, define BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS before including any library header.
Note that the text above does _not_ say that discarding the the pre-generated headers -- Aleksey Gurtovoy MetaCommunications Engineering
Hi Ovanes,
I need to define a map with int_<1...150>->Type1...150 pairs.
Im source file I define:
#define BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS #define BOOST_MPL_LIMIT_MAP_SIZE 150
#include
A word of caution: redefining BOOST_MPL_LIMIT_MAP_SIZE is not recommended. Doing so globally will make you pay for what you probably don't use (unless all present and future 'map' usages in your project demand the 150-elements limit), and doing so within a translation unit will lead to an ODR violation. If you need to construct a sequence beyond the default size, use its numbered form (http://www.boost.org/libs/mpl/doc/refmanual/variadic-sequence.html).
When compiling I get an error in the file: ...\boost\include\boost-1_33_1\boost\mpl\map.hpp on line: 36 fatal error C1083: Cannot open include file: 'boost/mpl/map/map150.hpp': No such file or directory
Development environment is Visual C++ 8 Express Edition
The help states:
[...]
What the BOOST_MPL_LIMIT_MAP_SIZE docs don't say is that there is a
default upper limit of 50 elements for the macro's value:
#define BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS
#define BOOST_MPL_LIMIT_MAP_SIZE 50
#include
Aleksey,
Many thanks for your detailed reply. I think the second scenario with
pre-generated headers convinces me more, since our compilation times are not
optimal at all :(. But as you stated in the book: more work for compiler
means less work at runtime. These were not exactly the words, but I can
remember the meaning.
I have one more question: I tried to use the preprocess_map.py script, but
encounter some problems. Running it on Windows with Python 2.5 causes the
following output:
D:\projects\libraries\boost_1_33_0\libs\mpl\preprocessed>preprocess_map.py
Usage:
preprocess_map.py <mode>
I need to define a map with int_<1...150>->Type1...150 pairs.
Im source file I define:
#define BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS #define BOOST_MPL_LIMIT_MAP_SIZE 150
#include
A word of caution: redefining BOOST_MPL_LIMIT_MAP_SIZE is not recommended. Doing so globally will make you pay for what you probably don't use (unless all present and future 'map' usages in your project demand the 150-elements limit), and doing so within a translation unit will lead to an ODR violation. If you need to construct a sequence beyond the default size, use its numbered form (http://www.boost.org/libs/mpl/doc/refmanual/variadic-sequence.html).
When compiling I get an error in the file: ...\boost\include\boost-1_33_1\boost\mpl\map.hpp on line: 36 fatal error C1083: Cannot open include file: 'boost/mpl/map/map150.hpp': No such file or directory
Development environment is Visual C++ 8 Express Edition
The help states:
[...]
What the BOOST_MPL_LIMIT_MAP_SIZE docs don't say is that there is a default
upper limit of 50 elements for the macro's value:
#define BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS
#define BOOST_MPL_LIMIT_MAP_SIZE 50
#include
"Ovanes Markarian"
Aleksey,
Many thanks for your detailed reply. I think the second scenario with pre-generated headers convinces me more, since our compilation times are not optimal at all :(. But as you stated in the book: more work for compiler means less work at runtime.
In this case you won't get any runtime speedups by using the method that's slower at compile time.
These were not exactly the words, but I can remember the meaning.
I'm not sure you got the meaning exactly right either ;-) -- Dave Abrahams Boost Consulting www.boost-consulting.com
Ovanes Markarian writes:
Many thanks for your detailed reply. I think the second scenario with pre-generated headers convinces me more, since our compilation times are not optimal at all :(.
I'd still suggest to try out the first version, before engaging into this particlular optimization, to see whether there are other showstoppers to what you are trying to do: the library has not been throughly tested with the sequence limits this high, and, depending on the particular compiler, you might run into internal structure overflows, ICEs and the likes, which may force you to adpot a different approach.
I have one more question: I tried to use the preprocess_map.py script, but encounter some problems.
[...]
How do I submit here the max map size? Or did you mean that I should write a similar script, which pregenerates headers by my own?
Sorry for not being specific enough about this; the script generates
the numbered 'map' forms basing entirely on the corresponding source
files / headers in the '$BOOST_ROOT/libs/mpl/preprocessed/map/' and
'$BOOST_ROOT/boost/mpl/map/' directories. To bump up the current 50
elements limit to, say, 150 you need to provide the corresponding
'mapN.cpp'/'mapN.hpp' files for N up to 150, using the existing ones
as an example.
In the simpliest case, assuming that you don't want the same level of
headers granularity the library adheres to and are okay with jumping
from 50 to 150 in one step, '$BOOST_ROOT/boost/mpl/map/map150.hpp'
would look like this:
#ifndef BOOST_MPL_MAP_MAP150_HPP_INCLUDED
#define BOOST_MPL_MAP_MAP150_HPP_INCLUDED
#if !defined(BOOST_MPL_PREPROCESSING_MODE)
# include
If I look inside of this script, there is one call to main which passes 3 parameters, but probably this does not work. If I pass parameters as in this example, I get an error that gcc is not installed.
Oh, right, sorry I forgot to mention it.
Ok, I will install it and put it into my PATH env var, but still the question, how do specify the size of max map?
Please see the above; the '<mode>' and '
Is there any way to include pre-generated headers for 150 elements into the next boost release?
There is a cost of doing so (http://article.gmane.org/gmane.comp.lib.boost.user/7057): Increasing the default limit to, let's say, 100 elements means plus ~120 KB to the size of the distribution, and I'm not sure the need is widespread enough to make everybody pay for it. This has been a while ago, but my concern didn't go away :).
Probably the unnumbered mpl::map template could go until 50 types to save compilation time, but if someone wishes to go above, he/she could use numbered map templates like map150. The problem, why I am going to suggest it, is that if someone pre-generates the header and updates the version, it is possibly, that pre-generated headers will be removed and project maintainer (not usually programmer who generated these headers) will suddenly get compiler errors and will have to repeat this procedure again.
IMHO this is a general issue: how do you manage a third-party library sources in presence of a need to make local patches to them, and keep these patches from getting overwritten accidentally? (Unless you always work with the HEAD / latest & greatest sources _and_ have a direct channel to the library maintainer, local patches are inevitable). Our answer to this question here at work is: 1) Always maintain a patch directory alongside with the root directory for the library sources, e.g.: boost_root/... boost_patches/... 2) Make the patch directory precede the original sources in the list of include paths. Do this and #1 at the very moment you import the library in your repository. 3) When a need for a patch occurs: a) Isomorphically copy the affected / add the new files into the patch directory, leaving the originals untouched, and patch the copies. If you need to delete the file, don't, but, depending on the use case, override it with an empty one or the one containing an #error directive / redirecting #include. b) Rebuild the library if needed (making sure to enforce #2). 4) When upgrading to a new version of the library, diff the patches against the new originals and delete/keep/adjust them depending on the results.
And some other question: Dave said, that mpl::map had some bug fixes from the 1.33 release to upcoming 1.34. Will these preprocessing scripts break the changes?
No; I just double-checked, and all the fixes are present in the original headers used to generate the preprocessed versions. HTH, -- Aleksey Gurtovoy MetaCommunications Engineering
Thanks a lot! I will give it as you describe it and post my experiences here :) With Kind Regards, Ovanes -----Original Message----- From: Aleksey Gurtovoy [mailto:agurtovoy@meta-comm.com] Sent: Monday, November 06, 2006 3:11 AM To: boost-users@lists.boost.org Cc: Ovanes Markarian; dave@boost-consulting.com Subject: Re: [mpl::map] generating map for 150 types Ovanes Markarian writes:
Many thanks for your detailed reply. I think the second scenario with pre-generated headers convinces me more, since our compilation times are not optimal at all :(.
I'd still suggest to try out the first version, before engaging into this particlular optimization, to see whether there are other showstoppers to what you are trying to do: the library has not been throughly tested with the sequence limits this high, and, depending on the particular compiler, you might run into internal structure overflows, ICEs and the likes, which may force you to adpot a different approach.
I have one more question: I tried to use the preprocess_map.py script, but encounter some problems.
[...]
How do I submit here the max map size? Or did you mean that I should write a similar script, which pregenerates headers by my own?
Sorry for not being specific enough about this; the script generates the
numbered 'map' forms basing entirely on the corresponding source files /
headers in the '$BOOST_ROOT/libs/mpl/preprocessed/map/' and
'$BOOST_ROOT/boost/mpl/map/' directories. To bump up the current 50 elements
limit to, say, 150 you need to provide the corresponding
'mapN.cpp'/'mapN.hpp' files for N up to 150, using the existing ones as an
example.
In the simpliest case, assuming that you don't want the same level of
headers granularity the library adheres to and are okay with jumping from 50
to 150 in one step, '$BOOST_ROOT/boost/mpl/map/map150.hpp'
would look like this:
#ifndef BOOST_MPL_MAP_MAP150_HPP_INCLUDED #define
BOOST_MPL_MAP_MAP150_HPP_INCLUDED
#if !defined(BOOST_MPL_PREPROCESSING_MODE)
# include
If I look inside of this script, there is one call to main which passes 3 parameters, but probably this does not work. If I pass parameters as in this example, I get an error that gcc is not installed.
Oh, right, sorry I forgot to mention it.
Ok, I will install it and put it into my PATH env var, but still the question, how do specify the size of max map?
Please see the above; the '<mode>' and '
Is there any way to include pre-generated headers for 150 elements into the next boost release?
There is a cost of doing so (http://article.gmane.org/gmane.comp.lib.boost.user/7057): Increasing the default limit to, let's say, 100 elements means plus ~120 KB to the size of the distribution, and I'm not sure the need is widespread enough to make everybody pay for it. This has been a while ago, but my concern didn't go away :).
Probably the unnumbered mpl::map template could go until 50 types to save compilation time, but if someone wishes to go above, he/she could use numbered map templates like map150. The problem, why I am going to suggest it, is that if someone pre-generates the header and updates the version, it is possibly, that pre-generated headers will be removed and project maintainer (not usually programmer who generated these headers) will suddenly get compiler errors and will have to repeat this procedure again.
IMHO this is a general issue: how do you manage a third-party library sources in presence of a need to make local patches to them, and keep these patches from getting overwritten accidentally? (Unless you always work with the HEAD / latest & greatest sources _and_ have a direct channel to the library maintainer, local patches are inevitable). Our answer to this question here at work is: 1) Always maintain a patch directory alongside with the root directory for the library sources, e.g.: boost_root/... boost_patches/... 2) Make the patch directory precede the original sources in the list of include paths. Do this and #1 at the very moment you import the library in your repository. 3) When a need for a patch occurs: a) Isomorphically copy the affected / add the new files into the patch directory, leaving the originals untouched, and patch the copies. If you need to delete the file, don't, but, depending on the use case, override it with an empty one or the one containing an #error directive / redirecting #include. b) Rebuild the library if needed (making sure to enforce #2). 4) When upgrading to a new version of the library, diff the patches against the new originals and delete/keep/adjust them depending on the results.
And some other question: Dave said, that mpl::map had some bug fixes from the 1.33 release to upcoming 1.34. Will these preprocessing scripts break the changes?
No; I just double-checked, and all the fixes are present in the original headers used to generate the preprocessed versions. HTH, -- Aleksey Gurtovoy MetaCommunications Engineering
participants (3)
-
Aleksey Gurtovoy -
David Abrahams -
Ovanes Markarian