...

Shouldn't Boost.Wave preprocess "11LL" into two tokens "11" and "LL" if support for long long suffixes is disabled? In a current version I get lexer error.

Using the Wave tool on a file containing nothing but '11LL\n' I get:

test.cpp(1): warning: long long suffixes are not allowed in pure C++ mode, enable long_long mode to allow these: 11LL

Is that wrong?

It would be ok if Wave tool printed this warning and preprocessing continued outputting two tokens '11' and 'LL'. Instead of this it throws wave::cpplexer::lexing_exception and stops preprocessing as if '11LL' is invalid character sequence. '11LL' is in no way different from '11ABC' and the latter produces two tokens as expected. This kind of behavior leads to some problems. For example, you can't use Boost.Wave for preprocessing many of the boost files without enabling support for long long suffixes because boost/limit.hpp uses them (but they are ifdef'ed away if you define some macroses).

...

Which version are you using?

Boost 1.33.1beta

...

Which compiler?

MSVC .Net 2005

Paul Mensonides rote:

...

It would be ok if Wave tool printed this warning and preprocessing continued outputting two tokens '11' and 'LL'. Instead of this it throws wave::cpplexer::lexing_exception and stops preprocessing as if '11LL' is invalid character sequence. '11LL' is in no way different from '11ABC' and the latter produces two tokens as expected.

Both '11LL' and '11ABC' are valid _preprocessing tokens_. However, with long long support, the former is a valid _token_ but the latter is not. At the point where preprocessing tokens are converted into tokens, an error should occur, not two tokens. That point is directly between where preprocessing ends and the rest of the phases of translation. Normally, a standalone preprocessor doesn't go all the way in the conversion phase, but Wave does because it is producing tokens by going into phase 7. IMO, there are only two options here: 1) Wave should yield preprocessing tokens instead of tokens, or 2) Wave should yield an error trying to translate '11ABC' into a token (or '11LL' if long long is not explicitly enabled--it isn't part of standard C++ yet).

Since Wave produces tokens and not preprocessing tokens I believe the current behaviour is the correct one.

...

This kind of behavior leads to some problems. For example, you can't use Boost.Wave for preprocessing many of the boost files without enabling support for long long suffixes because boost/limit.hpp uses them (but they are ifdef'ed away if you define some macroses).

Personally, I think that Wave should produce preprocessing tokens. An iterator adaptor could be applied to produce tokens (or more accurately, do the checking that tokens require).

Will have to think about this. One of the goals of the Wave library was not only to make available a preprocessor but also lexer components producing C++ tokens usable in other contexts as well. Wave currently doesn't follow strictly the mandated compilation phases. This has to be changed in the future anyway. Regards Hartmut