Paul Mensonides wrote:
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Hartmut Kaiser
Who's right? I'd agree with Paul Mensonides, because "1e" is a valid pp token which is to be formed before applying the ## operator. But my understanding is quite limited in this area so I commit myself to the Gurus. Paul is right and Wave is wrong. That's a known issue with Wave, it stems from the fact that Wave currently works on C++ tokens, not pp-tokens as mandated by the Standard. This will be fixed in the future, when I change Wave to operate on pp-tokens, rescanning these into C++ tokens after preprocessing.
Just so everyone else understands the difference... A "preprocessing token" is a more permissive version of a regular underlying language token. Most preprocessing tokens are exactly the same as tokens (such as operators and punctuators like ++, etc.). Some are slightly different. For example, keywords don't exist yet, so they are just identifier preprocessing tokens. The biggest difference of all is the pp-number preprocessing token. Regular number tokens are divided up into various types (e.g. hex, floating-point, etc.), but pp-numbers are general enough to encompass all of them at once without specifically knowing about any of them. As an example, the preprocessing token 0__a is a valid pp-number. Eventually, preprocessing tokens get converted into regular tokens (as the result of preprocessing when they are passed on to the parser and immediately prior to evaluating the controlling expression in an #if or #elif). Some preprocessing tokens cannot be converted to regular tokens (such as 0__a), and if that conversion is attempted, it is an error. Note that not all preprocessing tokens are converted--only those that are passed on to the parser (etc.). E.g.
#define EAT(x)
EAT( 0.0.0.0.0 ) ^^^^^^^^^
The marked part is a single pp-number preprocessing token, but because it doesn't appear in the output of the preprocessor, no attempt should ever be make to convert it to a regular token. The point being, that preprocessing tokens that cannot be converted can still be used without it being an error--which is probably *why* preprocessing tokens exist as a separate token type. What this means for an implementation is that pp-number preprocessing tokens, unlike the numeric tokens in the underlying language, cannot be (exclusively) stored as numeric values, they are more like identifiers or string-literals.
I have never understood the definition of a pp-number in the C++ standard. I am using the 1998 version and I read: ppnumber: digit . digit ppnumber digit ppnumber nondigit ppnumber e sign ppnumber E sign ppnumber . Since "ppnumber digit" and "ppnumber nondigit" covers all sequences beginning either with a "digit" or a ".digit" followed by anything else which is not a whitespace character or a comment, why specify the last three items ?