Hi, Suppose I have the following macro: #define DO(x)\ BOOST_PP_CAT(STEP3, BOOST_PP_CAT(STEP2, BOOST_PP_EXPAND(STEP1 x))) #define STEP2STEP1 Now, DO(5) should IMO expand into STEP35. For some reason, VC71 expands it into STEP3 5. Where does the space come from? Can one CAT two tokens, and get a space between them? Is it a bug in VC71 or is it supposed to be like this? If I use BOOST_PP_SEQ_CAT outside, it does what I expect. Can anybody explain why? (the above exersize may not make sence out of context. I don't think the context is relevant, although I can provide it if necessary) Thanks in advance. Regards, Arkadiy
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Arkadiy Vertleyb
Hi,
Suppose I have the following macro:
#define DO(x)\ BOOST_PP_CAT(STEP3, BOOST_PP_CAT(STEP2, BOOST_PP_EXPAND(STEP1 x)))
#define STEP2STEP1
Now, DO(5) should IMO expand into STEP35. For some reason, VC71 expands it into STEP3 5. Where does the space come from?
It's a VC expansion order bug. (Note that the EXPAND should have no effect on this code.) The space comes from the space between 'STEP1' and 'x'. That space *should* have no effect, but doesn't because VC sucks. One problem, easily demonstrated here, is that VC is operating on text, not tokens. E.g. #define AB 123 #define ID(x) x ID( ID(A)B ) --> 123 I.e. the 'ID(A)B' causes juxtaposition, which VC reinterprets as token merging (i.e. concatenation without concatenation). This is flat out totally wrong. VC *should* be operating on tokens (specifically, preprocessing tokens). The juxtaposition should merely produce two adjacent, but distinct, tokens. The compiler is never, under any circumstances, allowed to introduce whitespace until translation phase 7, where "whitespace characters separating tokens are no longer significant." Let me clarify with another example: #define ID(x) x ID(+)+ This code *does not* result in the increment operator ("++"). This code *does not* result in "+ +"--either the textual sense or the token-oriented sense (i.e. token(+) space token(+)). This code results in the + token immediately followed by another + token with no intervening whitespace. Note that this is *not* a problem because the compiler should not be retokenizing anything. Each preprocessing token resulting from the preprocessor should be directly converted to a "regular" token (in translation phase 7). A standalone preprocessor (i.e. one that outputs the texual result) must insert spaces to prevent errant retokenization. However, it can only do this at the very last moment--after all othe preprocessing tasks are performed (especially after all stringizing is done and after all <header-name> tokens are formed). (In case it isn't already abundantly clear, I despise the VC preprocessor.) The bug in VC that causes your problem isn't directly caused by text-based operation, but it is related to it. It is also related to VC not doing evaluation order correctly. It is probably doing concatenation at the same time as scanning for macro expansion, or something similar. In any case, you're code should work as expected (except that EXPAND has no effect related to what you are doing). VC is wrong--not just in this example, but in general.
Can one CAT two tokens, and get a space between them?
No.
Is it a bug in VC71 or is it supposed to be like this?
It is one of many bugs in VC's preprocessor. That preprocessor is is flat-out terrible and is close to the worst preprocessor implementation that exists is current compilers.
If I use BOOST_PP_SEQ_CAT outside, it does what I expect. Can anybody explain why?
How are you using SEQ_CAT? I ask because SEQ_CAT does an ordered concatenation from left-to-right. A simple translation of the above: SEQ_CAT((STEP3)(STEP2)(STEP1 5)) Will never result in a single STEP2STEP1 token (which is supposed to expand to nothing). Instead, it will result, successively in STEP3 -> STEP3STEP2 -> STEP3STEP2STEP1 5.
(the above exersize may not make sence out of context. I don't think the context is relevant, although I can provide it if necessary)
No, the context isn't necessary, though there may be a better way to do whatever it is you're trying to do. The probable reason that SEQ_CAT is working is that SEQ_CAT is doing more workarounds to force expansion on VC that should have happened already (possibly, should have happened a long time ago). Regards, Paul Mensonides
How are you using SEQ_CAT? I ask because SEQ_CAT does an ordered concatenation from left-to-right. A simple translation of the above:
SEQ_CAT((STEP3)(STEP2)(STEP1 5))
Will never result in a single STEP2STEP1 token (which is supposed to expand to nothing). Instead, it will result, successively in STEP3 -> STEP3STEP2 -> STEP3STEP2STEP1 5.
(the above exersize may not make sence out of context. I don't think the context is relevant, although I can provide it if necessary)
No, the context isn't necessary, though there may be a better way to do whatever it is you're trying to do. The probable reason that SEQ_CAT is working is
"Paul Mensonides" <pmenso57@comcast.net> wrote that
SEQ_CAT is doing more workarounds to force expansion on VC that should have happened already (possibly, should have happened a long time ago).
I guess I need to provide the context :-) I have a macro parameter that describes parameters of a template. This macro parameter can come in one of two flavours, for example: 1) (class)(unsigned int)(bool) 2) a number. This means that the template has n type parameters. I want to apply some "magic" to this macro parameter, that: - in case 1 would not affect it; - in case 2 convert it to something like (class)(class)(class) My first attempt was to paste something on the left, like: #define DO(x) BOOST_PP_CAT(STEP1, x) #define STEP1(x) (x) #define STEP11 (class) #define STEP12 (class)(class) ... To naive -- can't paste anything to '(' :-(( By the way, VC7 did allow me to do this -- this approach worked there. It is GCC who complained (Not that this is good about VC71. If you disregard PP, the VC71 is pretty good IMO, but far too tolerant). So, I re-implemented it like this: #define DO(x) BOOST_PP_SEQ_CAT(\ (STEP3)\ (BOOST_PP_CAT(STEP2, BOOST_PP_EXPAND(STEP1 x))) ) #define STEP1(x) _(x) #define STEP3STEP2_(x) (x) #define STEP2STEP1 #define STEP31 (class) #define STEP32 (class)(class) ... Basically STEP1 is needed to add something to the sequence you can paste to, STEP2 -- to remove the sideffect from the number, and STEP3 -- to do the work as in the original implementation. So: (int)(class) --> STEP1 (int)(class) --> _(int)(class) --> STEP2_(int)(class) --> STEP3STEP2_(int)(class) --> (int)(class) 2 --> STEP1 2 -->STEP2STEP1 2 --> [space]2 --> STEP32 -->(class)(class) This also explains why I need EXPAND. So do you think anything better can be done? Regards, Arkadiy
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Arkadiy Vertleyb
I guess I need to provide the context :-)
I have a macro parameter that describes parameters of a template. This macro parameter can come in one of two flavours, for example:
1) (class)(unsigned int)(bool)
2) a number. This means that the template has n type parameters.
I want to apply some "magic" to this macro parameter, that:
- in case 1 would not affect it; - in case 2 convert it to something like (class)(class)(class)
This will work on most compilers (including VC)... #include <boost/preprocessor/control/iif.hpp> #include <boost/preprocessor/detail/is_unary.hpp> #include <boost/preprocessor/repetition/repeat.hpp> #include <boost/preprocessor/tuple/eat.hpp> #define TRANSLATE(x) \ BOOST_PP_IIF( \ BOOST_PP_IS_UNARY(x), \ x BOOST_PP_TUPLE_EAT(3), BOOST_PP_REPEAT \ )(x, TRANSLATE_2, ~) \ /**/ #define TRANSLATE_2(z, n, _) (class) TRANSLATE( (class)(unsigned)(bool) ) // --> (class)(unsigned)(bool) TRANSLATE( 3 ) // --> (class)(class)(class) This will work on most preprocessors, but not on those that use the Borland configuration. Thanks to Daniel James, I do have a workaround for that bug for Borland in particular, but I don't have verification that it works on other preprocessors that use that configuration (so it has to wait until I can separate the implementations).
My first attempt was to paste something on the left, like:
#define DO(x) BOOST_PP_CAT(STEP1, x) #define STEP1(x) (x) #define STEP11 (class) #define STEP12 (class)(class) ...
To naive -- can't paste anything to '(' :-((
Yep (unfortunately--making this defined behavior as a no-op or a retokenization is on my wish list for a variety of reasons).
By the way, VC7 did allow me to do this -- this approach worked there. It is GCC who complained (Not that this is good about VC71. If you disregard PP, the VC71 is pretty good IMO, but far too tolerant).
Yes. I was merely releasing some frustration. VC has gotten significantly better in all other areas (with the notably exception, IMO, of everything related to managed code). Regards, Paul Mensonides
"Paul Mensonides" <pmenso57@comcast.net> wrote
This will work on most compilers (including VC)...
#include <boost/preprocessor/control/iif.hpp> #include <boost/preprocessor/detail/is_unary.hpp> #include <boost/preprocessor/repetition/repeat.hpp> #include <boost/preprocessor/tuple/eat.hpp>
#define TRANSLATE(x) \ BOOST_PP_IIF( \ BOOST_PP_IS_UNARY(x), \ x BOOST_PP_TUPLE_EAT(3), BOOST_PP_REPEAT \ )(x, TRANSLATE_2, ~) \ /**/ #define TRANSLATE_2(z, n, _) (class)
TRANSLATE( (class)(unsigned)(bool) ) // --> (class)(unsigned)(bool)
TRANSLATE( 3 ) // --> (class)(class)(class)
This will work on most preprocessors, but not on those that use the Borland configuration. Thanks to Daniel James, I do have a workaround for that bug for Borland in particular, but I don't have verification that it works on other preprocessors that use that configuration (so it has to wait until I can separate the implementations).
This definitely has a benefit of not having to provide multiple defines (but rather rely on ones in the Boost.Preprocessor). However, it uses some internal feature, IS_UNARY, which I guess is not a huge drawback since the suggestion comes from the author :-) The portability issue is somewhat of concern -- I would not like to compromise it because of this nicety feature... Do you think my other suggestion might be more portable? #define TRANSLATE(x) BOOST_PP_SEQ_CAT(\ (STEP3)\ (BOOST_PP_CAT(STEP2, BOOST_PP_EXPAND(STEP1 x))) ) #define STEP1(x) _(x) #define STEP3STEP2_(x) (x) #define STEP2STEP1 #define STEP31 (class) #define STEP32 (class)(class) ... Regards, Arkadiy
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Arkadiy Vertleyb
This will work on most preprocessors, but not on those that use the Borland configuration. Thanks to Daniel James, I do have a workaround for that bug for Borland in particular, but I don't have verification that it works on other preprocessors that use that configuration (so it has to wait until I can separate the implementations).
This definitely has a benefit of not having to provide multiple defines (but rather rely on ones in the Boost.Preprocessor). However, it uses some internal feature, IS_UNARY, which I guess is not a huge drawback since the suggestion comes from the author :-)
It would be a public interface if I could make it work across all supported preprocessors. That might be the case now, but I don't have access to either Sun's or IBM's preprocessors--both of which use the Borland configuration. The predicate definition itself is pretty straightforward if you take away the workarounds: #define CAT(a, b) PRIMITIVE_CAT(a, b) #define PRIMITIVE_CAT(a, b) a ## b #define SPLIT(i, im) PRIMITIVE_CAT(SPLIT_, i)(im) #define SPLIT_0(a, b) a #define SPLIT_1(a, b) b #define IS_NULLARY(x) \ SPLIT(0, CAT(IS_NULLARY_R_, IS_NULLARY_C x)) \ /**/ #define IS_NULLARY_C() 1 #define IS_NULLARY_R_1 1, ~ #define IS_NULLARY_R_IS_NULLARY_C 0, ~ #define IS_UNARY(x) IS_NULLARY(IS_UNARY_C x) #define IS_UNARY_C(a) () #define IS_BINARY(x) IS_NULLARY(IS_BINARY_C x) #define IS_BINARY_C(a, b) () #define IS_TERNARY(x) IS_NULLARY(IS_TERNARY_C x) #define IS_TERNARY_C(a, b, c) () // etc. The library already includes these macros complete with a mess of workarounds. However, the library doesn't yet contain a Borland compatible version--credits for this one go to Daniel James: #include <boost/preprocessor/detail/split.hpp> #include <boost/preprocessor/punctuation/comma.hpp> #include <boost/preprocessor/punctuation/paren.hpp> #include <boost/preprocessor/tuple/eat.hpp> #define IS_NULLARY(x) \ IS_NULLARY_2(BOOST_PP_SPLIT(1, IS_NULLARY_C x BOOST_PP_COMMA() 0)) \ /**/ #define IS_NULLARY_2(x) x #define IS_NULLARY_C() \ ~, 1 BOOST_PP_RPAREN() \ BOOST_PP_TUPLE_EAT(2) BOOST_PP_LPAREN() ~ \ /**/ #define IS_UNARY(x) IS_NULLARY(IS_UNARY_C x) #define IS_UNARY_C(a) () This should work on Borland's preprocessor, but I can't speak for Sun or IBM.
The portability issue is somewhat of concern -- I would not like to compromise it because of this nicety feature...
Do you think my other suggestion might be more portable?
#define TRANSLATE(x) BOOST_PP_SEQ_CAT(\ (STEP3)\ (BOOST_PP_CAT(STEP2, BOOST_PP_EXPAND(STEP1 x))) ^^^^^^^
The main thing that the Borland configuration works around is this situation. If 'x' is not parenthetic (i.e. the argument list to STEP1), the Borland preprocessor "merges" it similar to token-pasting, but (if I remember correctly) it does so without the possibility of constructing a macro name. E.g. #define A(args) B args #define B(a, b) a + b A( 1 ) -> B1 A( (x, y) ) -> x + y Daniel's version above, effectively just doesn't care that it is doing that or not. It works by shifting the parameters when the "attempt-to-expand" macro actually does invoke. I think that you're best course of action is to use the library's IS_UNARY, but define you're own IS_UNARY for Borland (as above). If you're concerned with supporting IBM and Sun, then you can fall back on macro replication and undefined behavior (concatenation to left-parethesis) because you're dealing with a finite set of possible inputs (i.e. numbers). I don't think that it is possible to use a single implementation that will work on all preprocessors. Note that kind of stuff tends to occur whenever you get into this kind of low-level detection stuff. For example, the automatic recursion in the library does a similar thing. For most preprocessor's that merely means attempting to invoke a macro (where success will produce a particular detectable output). When the particular macro doesn't expand, it is detected, so the mechanism moves on to the next one (technically speaking, it doesn't move on to the "next" one, it does a binary search). The Borland configuration, OTOH, cannot do this because (at the time) I didn't have an IS_NULLARY (or similar) that I could use to test whether a macro expanded. Without it, instead of testing for whether the result is X or not X, I have to test whether the result is X or Y or Z (and so on). IOW, I have to have literally *thousands* of extra macros just for that configuration. Now, I do have an IS_NULLARY implementation that will work on Borland, but other preprocessors (namely IBM and Sun) are using that configuration. Having those predicates is so useful that it is worth having an additional configuration (and all that that implies) for Borland. I haven't done that yet because I'm looking at quite a few more drastic changes (like unifying library state) as well as "preparing" the library for variadics (etc.). Regards, Paul Mensonides
participants (2)
-
Arkadiy Vertleyb -
Paul Mensonides