Re: [Boost-users] [WAVE] position of tokens

9 Nov 2006

      Andreas Sæbjørnsen wrote:
...
...
...
* the position of a token in the original source code is 
gone(default in standard Wave)
What do you mean by that?
What I need is both the original position and the position 
from the real_positions example available on a token.  This 
makes it possible to locate a token using the real_positions 
(which is equivalent to positions on output from a 
preprocessor) and then get the original source code position 
from that token. Do you see any other way of achieving this?
I changed the real_positions example to use a new token type carrying both,
the original and the corrected positions. This makes it a bit more tricky,
though, because now you have to explicitely instantiate some of the internal
Wave library template types for this new token type. These explicit
instantiations are normally done in the Wave library code (for the default
token type).

Note: the new token type is essentially a copy of the default token type
which has added the second position instance. For simplicity reasons I
removed the class allocators, though. You'll have to re-add these, if
needed. Please look at the file cpp_lex_token.hpp for reference).
...
...
...
* the line positioning is incorrect/non-standard (see
below for
...
an example).
This does not depend on the Wave library itself, but it's a 
matter of 
the real_positions example. Do you mind to submit a fix for that (I 
now changed the sample to output the detailed token 
information, which 
makes it easier to track the output)?
Yes. It was very easy to work with your code so that should 
be no problem. Are the real_positions set before or after the 
tokens are send to the preprocessing hooks?
The corrected position is set inside the generated_token() pp hook. IIUC
correctly you're asking, whether the corrected token position will be
available in other hooks as well. The answer is no. The generated_token pp
hook is call at the very last point before the preprocessed token gets
returned to the calling application. I see no other way to do this, because
only at this point I know everything about the final token sequence. Any
ideas?

HTH
Regards Hartmut
...
Thanks
Andrea
...
...
On 11/7/06, Hartmut Kaiser  wrote:
...
Andreas,
...
I need some advice regarding Wave. What I want to do is
...
...
with Wave on a token stream and extract the token stream 
representing macro calls after expansion. The twist is
...
...
the token stream after expansion to have the positions
...
...
preprocessor would see on the already expanded output 
from Wave.
E.g:
#define MACRO_CALL int x;
int main(){
     MACRO
};
Exands into
#line 2
int main(){
     int x;
};
Where 'int x' should have a position of line 3 
instead of line 1 
after expansion. I also want the column number to
represent what we
see on line 3 instead of what we see on line 1. Is
to operate
that I want
that another
this possible ?
...
Ok, I've added a new pp hook (what else? :-P) to Wave:
...
//
...
/////
//
//  The function 'generated_token' will be called by
////////////////////////////////////////////////////////////////////
the library
...
...
whenever a //  token is about to be returned from the library.
//
//  The parameter 'ctx' is a reference to the context
object used for
//  instantiating the preprocessing iterators by the user.
//
//  The parameter 't' is the token about to be returned
from the library.
//  This function may alter the token, but in this case it
must be //
implemented with a corresponding signature:
//
//      Token const&
//      generated_token(Context const& ctx, Token& t);
//
//  which makes it possible to modify the token in place.
//
//  The default behavior is to return the token passed as the 
parameter //  without modification.
//
//////////////////////////////////////////////////////////////
/////////////
...
template 
    Token const&
    generated_token(Context const& ctx, Token const& t)
    { return t; }
This should help solving your problem.
To show how this hook function may help you I added a
new sample
...
'real_positions' to Wave demonstrating its use. Everything
is in the
Boost CVS::HEAD.
HTH
Regards Hartmut
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users