Hi, this post originates from thread "concatenation of power of negative number" in comp.std.c++ (http://tinyurl.com/asn95). Consider this file: ---- #include <boost/preprocessor/cat.hpp> BOOST_PP_CAT(1e, -1) ---- According to Paul Mensonides' reply in the thread, it should be eventually interpreted as <1e> ## <-> <1> thus the pp-number "1e" is concatenated with "-" obtaining the pp-number "1e-" and then "1" is appended as a separate token. The final result would thus be "1e- 1" (notice the blank between "-" and "1"). However, if run into Wave, the program above produces the error message: error: pasting the following two tokens does not give a valid preprocessing token: "e" and "-" Who's right? I'd agree with Paul Mensonides, because "1e" is a valid pp token which is to be formed before applying the ## operator. But my understanding is quite limited in this area so I commit myself to the Gurus. Ganesh
Alberto Ganesh Barbati wrote:
this post originates from thread "concatenation of power of negative number" in comp.std.c++ (http://tinyurl.com/asn95). Consider this file:
---- #include <boost/preprocessor/cat.hpp> BOOST_PP_CAT(1e, -1) ----
According to Paul Mensonides' reply in the thread, it should be eventually interpreted as
<1e> ## <-> <1>
thus the pp-number "1e" is concatenated with "-" obtaining the pp-number "1e-" and then "1" is appended as a separate token. The final result would thus be "1e- 1" (notice the blank between "-" and "1").
However, if run into Wave, the program above produces the error message:
error: pasting the following two tokens does not give a valid preprocessing token: "e" and "-"
Who's right? I'd agree with Paul Mensonides, because "1e" is a valid pp token which is to be formed before applying the ## operator. But my understanding is quite limited in this area so I commit myself to the Gurus.
Paul is right and Wave is wrong. That's a known issue with Wave, it stems from the fact that Wave currently works on C++ tokens, not pp-tokens as mandated by the Standard. This will be fixed in the future, when I change Wave to operate on pp-tokens, rescanning these into C++ tokens after preprocessing. If you need to preprocess this kind of constructs with Wave, you may want to use the --variadics command line option, allowing concatenation of unrelated tokens. Regards Hartmut
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Hartmut Kaiser
Who's right? I'd agree with Paul Mensonides, because "1e" is a valid pp token which is to be formed before applying the ## operator. But my understanding is quite limited in this area so I commit myself to the Gurus.
Paul is right and Wave is wrong. That's a known issue with Wave, it stems from the fact that Wave currently works on C++ tokens, not pp-tokens as mandated by the Standard. This will be fixed in the future, when I change Wave to operate on pp-tokens, rescanning these into C++ tokens after preprocessing.
Just so everyone else understands the difference... A "preprocessing token" is a more permissive version of a regular underlying language token. Most preprocessing tokens are exactly the same as tokens (such as operators and punctuators like ++, etc.). Some are slightly different. For example, keywords don't exist yet, so they are just identifier preprocessing tokens. The biggest difference of all is the pp-number preprocessing token. Regular number tokens are divided up into various types (e.g. hex, floating-point, etc.), but pp-numbers are general enough to encompass all of them at once without specifically knowing about any of them. As an example, the preprocessing token 0__a is a valid pp-number. Eventually, preprocessing tokens get converted into regular tokens (as the result of preprocessing when they are passed on to the parser and immediately prior to evaluating the controlling expression in an #if or #elif). Some preprocessing tokens cannot be converted to regular tokens (such as 0__a), and if that conversion is attempted, it is an error. Note that not all preprocessing tokens are converted--only those that are passed on to the parser (etc.). E.g. #define EAT(x) EAT( 0.0.0.0.0 ) ^^^^^^^^^ The marked part is a single pp-number preprocessing token, but because it doesn't appear in the output of the preprocessor, no attempt should ever be make to convert it to a regular token. The point being, that preprocessing tokens that cannot be converted can still be used without it being an error--which is probably *why* preprocessing tokens exist as a separate token type. What this means for an implementation is that pp-number preprocessing tokens, unlike the numeric tokens in the underlying language, cannot be (exclusively) stored as numeric values, they are more like identifiers or string-literals. Regards, Paul Mensonides
Paul Mensonides wrote:
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Hartmut Kaiser
Who's right? I'd agree with Paul Mensonides, because "1e" is a valid pp token which is to be formed before applying the ## operator. But my understanding is quite limited in this area so I commit myself to the Gurus. Paul is right and Wave is wrong. That's a known issue with Wave, it stems from the fact that Wave currently works on C++ tokens, not pp-tokens as mandated by the Standard. This will be fixed in the future, when I change Wave to operate on pp-tokens, rescanning these into C++ tokens after preprocessing.
Just so everyone else understands the difference... A "preprocessing token" is a more permissive version of a regular underlying language token. Most preprocessing tokens are exactly the same as tokens (such as operators and punctuators like ++, etc.). Some are slightly different. For example, keywords don't exist yet, so they are just identifier preprocessing tokens. The biggest difference of all is the pp-number preprocessing token. Regular number tokens are divided up into various types (e.g. hex, floating-point, etc.), but pp-numbers are general enough to encompass all of them at once without specifically knowing about any of them. As an example, the preprocessing token 0__a is a valid pp-number. Eventually, preprocessing tokens get converted into regular tokens (as the result of preprocessing when they are passed on to the parser and immediately prior to evaluating the controlling expression in an #if or #elif). Some preprocessing tokens cannot be converted to regular tokens (such as 0__a), and if that conversion is attempted, it is an error. Note that not all preprocessing tokens are converted--only those that are passed on to the parser (etc.). E.g.
#define EAT(x)
EAT( 0.0.0.0.0 ) ^^^^^^^^^
The marked part is a single pp-number preprocessing token, but because it doesn't appear in the output of the preprocessor, no attempt should ever be make to convert it to a regular token. The point being, that preprocessing tokens that cannot be converted can still be used without it being an error--which is probably *why* preprocessing tokens exist as a separate token type. What this means for an implementation is that pp-number preprocessing tokens, unlike the numeric tokens in the underlying language, cannot be (exclusively) stored as numeric values, they are more like identifiers or string-literals.
I have never understood the definition of a pp-number in the C++ standard. I am using the 1998 version and I read: ppnumber: digit . digit ppnumber digit ppnumber nondigit ppnumber e sign ppnumber E sign ppnumber . Since "ppnumber digit" and "ppnumber nondigit" covers all sequences beginning either with a "digit" or a ".digit" followed by anything else which is not a whitespace character or a comment, why specify the last three items ?
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Edward Diener
I have never understood the definition of a pp-number in the C++ standard. I am using the 1998 version and I read:
ppnumber: digit . digit ppnumber digit ppnumber nondigit ppnumber e sign ppnumber E sign ppnumber .
Since "ppnumber digit" and "ppnumber nondigit" covers all sequences beginning either with a "digit" or a ".digit" followed by anything else which is not a whitespace character or a comment, why specify the last three items ?
Despite the name, 'nondigit' is not any non-digit character. It is just the alphabetic characters, the underscore, and universal character names (subject to annex E). Thus, the other productions are for the decimal point (.) and for the sign (+ or -). Regards, Paul Mensonides
Paul Mensonides wrote:
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Edward Diener
I have never understood the definition of a pp-number in the C++ standard. I am using the 1998 version and I read:
ppnumber: digit . digit ppnumber digit ppnumber nondigit ppnumber e sign ppnumber E sign ppnumber .
Since "ppnumber digit" and "ppnumber nondigit" covers all sequences beginning either with a "digit" or a ".digit" followed by anything else which is not a whitespace character or a comment, why specify the last three items ?
Despite the name, 'nondigit' is not any non-digit character. It is just the alphabetic characters, the underscore, and universal character names (subject to annex E).
Thanks, I see it now. The definition of non-digit ( and digit ) unfortunately was made not in Preprocessing Numbers section, where it was first mentioned, but in the Identifiers section. Thanks for pointing this out to me.
participants (4)
-
Alberto Ganesh Barbati -
Edward Diener -
Hartmut Kaiser -
Paul Mensonides