Test1 posted below produces: INT SPACE EOF SPACE IDENTIFIER SEMICOLON NEWLINE EOF While Test2 produces: INT SPACE IDENTIFIER SEMICOLON NEWLINE EOF The EOF in the third line of Test1 output is produced by the TEST macro expansion. If this expected behaviour? I found this surprising. (MSVC10, Boost 1.51) Regards, Klaus ---- Code: typedef cpplexer::lex_token<> token_type; typedef context< string::iterator, cpplexer::lex_iterator<token_type> > lexer_context_type; void Test1() { string instring("int TEST x;\n"); lexer_context_type ctx(instring.begin(), instring.end(), ""); ctx.set_language(language_support(support_c99)); ctx.add_macro_definition("TEST="); for (lexer_context_type::iterator_type it = ctx.begin(); it != ctx.end(); ++it) cout << get_token_name(token_id(*it)) << endl; } void Test2() { string instring("#define TEST\nint TEST x;\n"); lexer_context_type ctx(instring.begin(), instring.end(), ""); ctx.set_language(language_support(support_c99)); for (lexer_context_type::iterator_type it = ctx.begin(); it != ctx.end(); ++it) cout << get_token_name(token_id(*it)) << endl; }