Re: [Boost-users] [WAVE] position of tokens

9 Nov 2006

      ...
I changed the real_positions example to use a new token type carrying both,
the original and the corrected positions. This makes it a bit more tricky,
though, because now you have to explicitely instantiate some of the internal
Wave library template types for this new token type. These explicit
instantiations are normally done in the Wave library code (for the default
token type).
I think this probably is the optimal solution and it is not difficult
to instantiate the templates in my library although I expect it to
slow down compilation considerably (?).
...
Note: the new token type is essentially a copy of the default token type
which has added the second position instance. For simplicity reasons I
removed the class allocators, though. You'll have to re-add these, if
needed. Please look at the file cpp_lex_token.hpp for reference).
In which instances would I need to use these class allocators? What
are they used for in Wave with the standard token type?
...
...
Yes. It was very easy to work with your code so that should
be no problem. Are the real_positions set before or after the
tokens are send to the preprocessing hooks?
The corrected position is set inside the generated_token() pp hook. IIUC
correctly you're asking, whether the corrected token position will be
available in other hooks as well. The answer is no. The generated_token pp
hook is call at the very last point before the preprocessed token gets
returned to the calling application. I see no other way to do this, because
only at this point I know everything about the final token sequence. Any
ideas?
An idea would be to allow calling of generated_token() from one of the
other hooks. For instance, if you in rescanned_macro() know when the
first macro has been fully texpanded then you can call
generated_token() at that point on every token provided to
rescanned_macro(). The reason why I say first macro is that a macro in
it's formal definition can call another macro, so first macro calls
second macro etc.

The suggestion I made above can probably also be considered an
optimization (?) because you could also imagine recomputing the
correct positions on every call to rescanned_macro.
Do you see any reason for this not working?

Thanks
Andreas
...
...
...
...
On 11/7/06, Hartmut Kaiser  wrote:
...
Andreas,
...
I need some advice regarding Wave. What I want to do is
...
...
with Wave on a token stream and extract the token stream
representing macro calls after expansion. The twist is
...
...
the token stream after expansion to have the positions
...
...
preprocessor would see on the already expanded output
from Wave.
E.g:
#define MACRO_CALL int x;
int main(){
     MACRO
};
Exands into
#line 2
int main(){
     int x;
};
Where 'int x' should have a position of line 3
instead of line 1
after expansion. I also want the column number to
represent what we
see on line 3 instead of what we see on line 1. Is
to operate
that I want
that another
this possible ?
...
Ok, I've added a new pp hook (what else? :-P) to Wave:
...
//
...
/////
//
//  The function 'generated_token' will be called by
////////////////////////////////////////////////////////////////////
the library
...
...
whenever a //  token is about to be returned from the library.
//
//  The parameter 'ctx' is a reference to the context
object used for
//  instantiating the preprocessing iterators by the user.
//
//  The parameter 't' is the token about to be returned
from the library.
//  This function may alter the token, but in this case it
must be //
implemented with a corresponding signature:
//
//      Token const&
//      generated_token(Context const& ctx, Token& t);
//
//  which makes it possible to modify the token in place.
//
//  The default behavior is to return the token passed as the
parameter //  without modification.
//
//////////////////////////////////////////////////////////////
/////////////
...
template 
    Token const&
    generated_token(Context const& ctx, Token const& t)
    { return t; }
This should help solving your problem.
To show how this hook function may help you I added a
new sample
...
'real_positions' to Wave demonstrating its use. Everything
is in the
Boost CVS::HEAD.
HTH
Regards Hartmut
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users

Re: [Boost-users] [WAVE] position of tokens

Andreas Sæbjørnsen