[preprocessor metaprogramming] tools
Hello, I've noticed preprocessor metaprogramming has become more popular of late, both as a means to provide a cleaner alternative interface to a library (e.g. the DEFINE and ADAPT macros in Boost.Fusion or the proposed macro wrappers for Boost.TypeErasure) and as a means to emulate new language features (e.g. Boost.LocalFunction, Boost.Generic, and Boost.Contract). Sadly, IDE support for writing macro metaprograms is fairly poor in comparison to IDE support for writing regular C++ code. I wanted to share with you something I've found very useful for tracing/debugging macro metaprograms: the Eclipse CDT IDE has a "macro stepper" that allows you to step through the expansion of a macro step-by-step from the initial invocation to the final expansion. (This feature can be accessed by using the "Explore Macro Expansion" context menu item on a macro invocation.) I would be curious to hear what tools you have found useful for writing or debugging macro metaprograms. Regards, Nate
Nathan Ridge wrote
I've noticed preprocessor metaprogramming has become more popular of late, both as a means to provide a cleaner alternative interface to a library (e.g. the DEFINE and ADAPT macros in Boost.Fusion or the proposed macro wrappers for Boost.TypeErasure) and as a means to emulate new language features (e.g. Boost.LocalFunction, Boost.Generic, and Boost.Contract).
Sadly, IDE support for writing macro metaprograms is fairly poor in comparison to IDE support for writing regular C++ code.
I wanted to share with you something I've found very useful for tracing/debugging macro metaprograms: the Eclipse CDT IDE has a "macro stepper" that allows you to step through the expansion of a macro step-by-step from the initial invocation to the final expansion. (This feature can be accessed by using the "Explore Macro Expansion" context menu item on a macro invocation.)
I would be curious to hear what tools you have found useful for writing or debugging macro metaprograms.
I do this manually expanding the macros before compilation with -E -P compiler options (or /EP...) and inspecting by eye the lengthy expanded macro code. But I'm a vi guy that uses gdb and Linux command line -- so I like things raw ;) I was planning to use the Eclipse feature you indicated to benchmark macro expansion times/steps in order to optimize my pp meta-programs but I didn't get around to do that yet. Thanks. --Lorenzo -- View this message in context: http://boost.2283326.n4.nabble.com/preprocessor-metaprogramming-tools-tp4632... Sent from the Boost - Dev mailing list archive at Nabble.com.
I usually use the printf-style debugging for macros. So, if I want to see what the output is at a certain stage, I might add a DEBUG(), which is what I will look for in the preprocessor output. Heres an example, say we want to debug the TINY_size macros(taken from the C++ Template Metaprogramming appendix): #define TINY_print(z, n, data) data #define TINY_size(z, n, unused) \ template <BOOST_PP_ENUM_PARAMS(n, class T)> \ struct tiny_size< \ BOOST_PP_ENUM_PARAMS(n,T) \ BOOST_PP_COMMA_IF(n) \ BOOST_PP_ENUM( \ BOOST_PP_SUB(TINY_MAX_SIZE,n), TINY_print, none) \ > \ : mpl::int_<n> {}; BOOST_PP_REPEAT(TINY_MAX_SIZE, TINY_size, ~) #undef TINY_size #undef TINY_print Now maybe I want to see what each of the parameters are getting passed into TINY_size, I can then change it to this: #define TINY_size(z, n, unused) DEBUG(z, n, unused) \ template <BOOST_PP_ENUM_PARAMS(n, class T)> \ struct tiny_size< \ BOOST_PP_ENUM_PARAMS(n,T) \ BOOST_PP_COMMA_IF(n) \ BOOST_PP_ENUM( \ BOOST_PP_SUB(TINY_MAX_SIZE,n), TINY_print, none) \ > \ : mpl::int_<n> {}; Then I will look at the preprocessor output for DEBUG(...). Now if I want to check that the subtraction is correct I could add another statement for that(or perhaps replace the the other debug statement): #define TINY_size(z, n, unused) DEBUG(z, n, unused) \ DEBUG_SUB(BOOST_PP_SUB(TINY_MAX_SIZE,n)) \ template <BOOST_PP_ENUM_PARAMS(n, class T)> \ struct tiny_size< \ BOOST_PP_ENUM_PARAMS(n,T) \ BOOST_PP_COMMA_IF(n) \ BOOST_PP_ENUM( \ BOOST_PP_SUB(TINY_MAX_SIZE,n), TINY_print, none) \ > \ : mpl::int_<n> {}; Then I look for DEBUG_SUB(...) in the preprocessor output. This works quite well for simple things like that, but when using recursion and deferred expressions, I have not found an effective way. This is because its difficult to tell why a macro didn't expand, is it because it was painted blue, or does it need another scan? In this case it requires an expert understanding of the preprocessor, and stepping through the expansions manually. I wanted to share with you something I've found very useful for tracing/debugging macro metaprograms: the Eclipse CDT IDE has a "macro stepper" that allows you to step through the expansion of a macro step-by-step from the initial invocation to the final expansion. (This feature can be accessed by using the "Explore Macro Expansion" context menu item on a macro invocation.) This seems very nice, and probably can help especially when debugging deferred expressions. However, will it show when a macro is painted blue? That is really important to knowing why it didn't expand. Thanks, Paul
I usually use the printf-style debugging for macros. So, if I want to see what the output is at a certain stage, I might add a `DEBUG()`, which is what I will look for in the preprocessor output. Heres an example, say we want to debug the TINY_size macros(taken from the C++ Template Metaprogramming appendix): #define TINY_print(z, n, data) data #define TINY_size(z, n, unused) \ template <BOOST_PP_ENUM_PARAMS(n, class T)> \ struct tiny_size< \ BOOST_PP_ENUM_PARAMS(n,T) \ BOOST_PP_COMMA_IF(n) \ BOOST_PP_ENUM( \ BOOST_PP_SUB(TINY_MAX_SIZE,n), TINY_print, none) \ > \ : mpl::int_<n> {}; BOOST_PP_REPEAT(TINY_MAX_SIZE, TINY_size, ~) #undef TINY_size #undef TINY_print Now maybe I want to see what each of the parameters are getting passed into `TINY_size`, I can then change it to this: #define TINY_size(z, n, unused) DEBUG(z, n, unused) \ template <BOOST_PP_ENUM_PARAMS(n, class T)> \ struct tiny_size< \ BOOST_PP_ENUM_PARAMS(n,T) \ BOOST_PP_COMMA_IF(n) \ BOOST_PP_ENUM( \ BOOST_PP_SUB(TINY_MAX_SIZE,n), TINY_print, none) \ > \ : mpl::int_<n> {}; Then I will look at the preprocessor output for `DEBUG(...)`. Now if I want to check that the subtraction is correct I could add another statement for that(or perhaps replace the the other debug statement): #define TINY_size(z, n, unused) DEBUG(z, n, unused) \ DEBUG_SUB(BOOST_PP_SUB(TINY_MAX_SIZE,n)) \ template <BOOST_PP_ENUM_PARAMS(n, class T)> \ struct tiny_size< \ BOOST_PP_ENUM_PARAMS(n,T) \ BOOST_PP_COMMA_IF(n) \ BOOST_PP_ENUM( \ BOOST_PP_SUB(TINY_MAX_SIZE,n), TINY_print, none) \ > \ : mpl::int_<n> {}; Then I look for `DEBUG_SUB(...)` in the preprocessor output. This works quite well for simple things like that, but when using recursion and deferred expressions, I have not found an effective way. This is because its difficult to tell why a macro didn't expand, is it because it was painted blue, or does it need another scan? In this case it requires an expert understanding of the preprocessor, and stepping through the expansions manually.
I wanted to share with you something I've found very useful for tracing/debugging macro metaprograms: the Eclipse CDT IDE has a "macro stepper" that allows you to step through the expansion of a macro step-by-step from the initial invocation to the final expansion. (This feature can be accessed by using the "Explore Macro Expansion" context menu item on a macro invocation.)
This seems very nice, and probably can help especially when debugging deferred expressions. However, will it show when a macro is painted blue? That is really important to knowing why it didn't expand. Thanks, Paul
I wanted to share with you something I've found very useful for tracing/debugging macro metaprograms: the Eclipse CDT IDE has a "macro stepper" that allows you to step through the expansion of a macro step-by-step from the initial invocation to the final expansion. (This feature can be accessed by using the "Explore Macro Expansion" context menu item on a macro invocation.)
This seems very nice, and probably can help especially when debugging deferred expressions. However, will it show when a macro is painted blue? That is really important to knowing why it didn't expand.
Unfortunately it does not show that. That would be a nice addition. Regards, Nate
on Fri Jul 06 2012, Nathan Ridge <zeratul976-AT-hotmail.com> wrote:
I wanted to share with you something I've found very useful for tracing/debugging macro metaprograms: the Eclipse CDT IDE has a "macro stepper" that allows you to step through the expansion of a macro step-by-step from the initial invocation to the final expansion. (This feature can be accessed by using the "Explore Macro Expansion" context menu item on a macro invocation.)
This seems very nice, and probably can help especially when debugging deferred expressions. However, will it show when a macro is painted blue? That is really important to knowing why it didn't expand.
Unfortunately it does not show that. That would be a nice addition.
Wouldn't it be possible to build something really useful on top of the Wave library? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost
I wanted to share with you something I've found very useful for tracing/debugging macro metaprograms: the Eclipse CDT IDE has a "macro stepper" that allows you to step through the expansion of a macro step-by-step from the initial invocation to the final expansion. (This feature can be accessed by using the "Explore Macro Expansion" context menu item on a macro invocation.)
This seems very nice, and probably can help especially when debugging deferred expressions. However, will it show when a macro is painted blue? That is really important to knowing why it didn't expand.
Unfortunately it does not show that. That would be a nice addition.
Wouldn't it be possible to build something really useful on top of the Wave library?
The Wave tool ($BOOST_ROOT/tools/wave) allows generating traces for the expansion of macros. See the docs for it's done (http://www.boost.org/doc/libs/1_50_0/libs/wave/doc/tracing_facility.html). Macro tracing in Wave does not show when a macro has been painted blue, however it allows following in detail what happens during the expansion of a particular macro. The information about a macro being painted is available and could be added to the tracing facility easily. Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu
On 07/29/2012 09:38 AM, Hartmut Kaiser wrote:
I wanted to share with you something I've found very useful for tracing/debugging macro metaprograms: the Eclipse CDT IDE has a "macro stepper" that allows you to step through the expansion of a macro step-by-step from the initial invocation to the final expansion. (This feature can be accessed by using the "Explore Macro Expansion" context menu item on a macro invocation.)
This seems very nice, and probably can help especially when debugging deferred expressions. However, will it show when a macro is painted blue? That is really important to knowing why it didn't expand.
Unfortunately it does not show that. That would be a nice addition.
Wouldn't it be possible to build something really useful on top of the Wave library?
The Wave tool ($BOOST_ROOT/tools/wave) allows generating traces for the expansion of macros. See the docs for it's done (http://www.boost.org/doc/libs/1_50_0/libs/wave/doc/tracing_facility.html). Macro tracing in Wave does not show when a macro has been painted blue, however it allows following in detail what happens during the expansion of a particular macro. The information about a macro being painted is available and could be added to the tracing facility easily.
I have always found the Wave trace facility to be amazingly helpful in discovering how a macro should be expanded correctly. Just want to say thanks for this ability. Any further information that can be shown when expanding a macro, perhaps as a Wave trace option, is always welcome and appreciated by this programmer. Maybe something like a --verbose option.
I wanted to share with you something I've found very useful for tracing/debugging macro metaprograms: the Eclipse CDT IDE has a "macro stepper" that allows you to step through the expansion of a macro step-by-step from the initial invocation to the final expansion. (This feature can be accessed by using the "Explore Macro Expansion" context menu item on a macro invocation.)
This seems very nice, and probably can help especially when debugging deferred expressions. However, will it show when a macro is painted blue? That is really important to knowing why it didn't expand.
Unfortunately it does not show that. That would be a nice addition.
Wouldn't it be possible to build something really useful on top of the Wave library?
The Wave tool ($BOOST_ROOT/tools/wave) allows generating traces for the expansion of macros. See the docs for it's done
(http://www.boost.org/doc/libs/1_50_0/libs/wave/doc/tracing_facility.html) .
Macro tracing in Wave does not show when a macro has been painted blue, however it allows following in detail what happens during the expansion of a particular macro. The information about a macro being painted is available and could be added to the tracing facility easily.
I have always found the Wave trace facility to be amazingly helpful in discovering how a macro should be expanded correctly. Just want to say thanks for this ability. Any further information that can be shown when expanding a macro, perhaps as a Wave trace option, is always welcome and appreciated by this programmer. Maybe something like a --verbose option.
Well, thinking about it... The trace already (implicitly) has the information about a macro name being blue painted. The rescanning [...] output gives you the scope for which the macro under expansion is not available for further expansion (e.g. it is marked blue). Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu
On Mon, 30 Jul 2012 08:19:44 -0500, Hartmut Kaiser wrote:
Well, thinking about it... The trace already (implicitly) has the information about a macro name being blue painted. The rescanning [...] output gives you the scope for which the macro under expansion is not available for further expansion (e.g. it is marked blue).
Hartmut, does Wave still use a recursive expansion model? If so, while Wave emulates the correct results well, it gets there in a fashion that isn't technically correct. Debugging preprocessor metaprogramming naturally exposes that internal process. The theoretical process is essentially a stream-editing process with some (implementation-specific) method of dealing with the hidesets (i.e. virtual tokens or whatever). What would be ideal for debugging is a preprocessor that implemented this directly (such as the algorithm I posted a while back--which is nowhere near efficient enough for a real preprocessor). Otherwise, you are 1) really only debugging against Wave instead of against a standard and 2) learning the wrong things. Regards, Paul Mensonides
Well, thinking about it... The trace already (implicitly) has the information about a macro name being blue painted. The rescanning [...] output gives you the scope for which the macro under expansion is not available for further expansion (e.g. it is marked blue).
Hartmut, does Wave still use a recursive expansion model?
Yes. I have not re-implemented the macro-expansion since.
If so, while Wave emulates the correct results well, it gets there in a fashion that isn't technically correct. Debugging preprocessor metaprogramming naturally exposes that internal process.
The theoretical process is essentially a stream-editing process with some (implementation-specific) method of dealing with the hidesets (i.e. virtual tokens or whatever). What would be ideal for debugging is a preprocessor that implemented this directly (such as the algorithm I posted a while back--which is nowhere near efficient enough for a real preprocessor). Otherwise, you are 1) really only debugging against Wave instead of against a standard and 2) learning the wrong things.
Sure, I agree. However as we know, the differences are limited to a tiny set of corner cases. Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu
On 7/5/2012 3:02 AM, Nathan Ridge wrote:
Hello,
I've noticed preprocessor metaprogramming has become more popular of late, both as a means to provide a cleaner alternative interface to a library (e.g. the DEFINE and ADAPT macros in Boost.Fusion or the proposed macro wrappers for Boost.TypeErasure) and as a means to emulate new language features (e.g. Boost.LocalFunction, Boost.Generic, and Boost.Contract).
Sadly, IDE support for writing macro metaprograms is fairly poor in comparison to IDE support for writing regular C++ code.
I wanted to share with you something I've found very useful for tracing/debugging macro metaprograms: the Eclipse CDT IDE has a "macro stepper" that allows you to step through the expansion of a macro step-by-step from the initial invocation to the final expansion. (This feature can be accessed by using the "Explore Macro Expansion" context menu item on a macro invocation.)
I would be curious to hear what tools you have found useful for writing or debugging macro metaprograms.
The problem with debugging macro expansion is that the preprocessor one is using may be incorrectly expanding a macro and using other tools to look at the macro expansion will not show you the macro expansion which the incorrectly expanding preprocessor is doing. This is particularly the case with VC++. I have already queried on the Microsoft online forums if VC++ has a means of showing macro expansion step-by-step and the answer appears to be that it does not. This is not surprising since such a tool would further highlight VC++'s bugs in macro expansion, which they have pointedly declined to fix for many, many years. Boost already has a solution for viewing macro expansion in a 100% correct C++ preprocessor: Use the Boost wave with the tracing facility. This is very easy to do to see how a macro should be expanded. The Eclipse CDT IDE's "macro stepper" is a welcome edition to how a macro is expanded as long as the Eclipse C++ implementation's preprocessor is a 100% compliant preprocessor.
participants (7)
-
Dave Abrahams -
Edward Diener -
Hartmut Kaiser -
lcaminiti -
Nathan Ridge -
paul Fultz -
Paul Mensonides