wave lexed_tokens sample has a lexing error on an empty file
Hello, I pulled down a copy of boost 1.34 from CVS today and built the lexed_tokens sample. It seemed to work well, except for running it on an empty file: $./lexed_tokens bar.c bar.c(1): error: generic lexing error: invalid character '\000' in input stream $ This used to work in boost 1.33.1: $ ./lexed_tokens bar.c EOF (#402) at bar.c ( 1/ 1): >< $ Leo
Leo Davis wrote:
Hello,
I pulled down a copy of boost 1.34 from CVS today and built the lexed_tokens sample. It seemed to work well, except for running it on an empty file:
$./lexed_tokens bar.c bar.c(1): error: generic lexing error: invalid character '\000' in input stream $
This used to work in boost 1.33.1:
$ ./lexed_tokens bar.c EOF (#402) at bar.c ( 1/ 1): >< $
Leo
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I was able to get the behavior I wanted by applying the following patch in libs/wave/src/cpplexer/re2clex. I don't know if this is a good idea or not. I wasn't able to get something that looked correct out of re2c, so I don't have those patched. Leo
Leo Davis wrote:
I pulled down a copy of boost 1.34 from CVS today and built the lexed_tokens sample. It seemed to work well, except for running it on an empty file:
$./lexed_tokens bar.c bar.c(1): error: generic lexing error: invalid character '\000' in input stream $
This used to work in boost 1.33.1:
$ ./lexed_tokens bar.c EOF (#402) at bar.c ( 1/ 1): >< $
Leo
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I was able to get the behavior I wanted by applying the following patch in libs/wave/src/cpplexer/re2clex. I don't know if this is a good idea or not. I wasn't able to get something that looked correct out of re2c, so I don't have those patched.
I'm not able to reproduce this problem in the CVS::HEAD version. I get you expected output as you mention above. Have you tried this with the CVS version? Regards Hartmut
Hartmut Kaiser wrote:
Leo Davis wrote:
I pulled down a copy of boost 1.34 from CVS today and built the lexed_tokens sample. It seemed to work well, except for running it on an empty file:
$./lexed_tokens bar.c bar.c(1): error: generic lexing error: invalid character '\000' in input stream $
This used to work in boost 1.33.1:
$ ./lexed_tokens bar.c EOF (#402) at bar.c ( 1/ 1): >< $
Leo
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I was able to get the behavior I wanted by applying the following patch in libs/wave/src/cpplexer/re2clex. I don't know if this is a good idea or not. I wasn't able to get something that looked correct out of re2c, so I don't have those patched.
I'm not able to reproduce this problem in the CVS::HEAD version. I get you expected output as you mention above. Have you tried this with the CVS version?
Yes, I updated from CVS::HEAD again today and rebuilt the lib and lexed_tokens. I still get the "invalid character" error for an empty file. I'm using gcc 4.1.2 on Linux 2.6. Leo
Leo,
I was able to get the behavior I wanted by applying the following patch in libs/wave/src/cpplexer/re2clex. I don't know if this is a good idea or not. I wasn't able to get something that looked correct out of re2c, so I don't have those patched.
I'm not able to reproduce this problem in the CVS::HEAD version. I get you expected output as you mention above. Have you tried this with the CVS version?
Yes, I updated from CVS::HEAD again today and rebuilt the lib and lexed_tokens. I still get the "invalid character" error for an empty file.
I'm using gcc 4.1.2 on Linux 2.6.
I applied your patch to the *.re files and regenerated the *.inc files. It's in Boost CVS::HEAD. Could you verify please? Regards Hartmut
Hartmut Kaiser wrote:
Leo,
I was able to get the behavior I wanted by applying the following patch in libs/wave/src/cpplexer/re2clex. I don't know if this is a good idea or not. I wasn't able to get something that looked correct out of re2c, so I don't have those patched. I'm not able to reproduce this problem in the CVS::HEAD version. I get you expected output as you mention above. Have you tried this with the CVS version? Yes, I updated from CVS::HEAD again today and rebuilt the lib and lexed_tokens. I still get the "invalid character" error for an empty file.
I'm using gcc 4.1.2 on Linux 2.6.
I applied your patch to the *.re files and regenerated the *.inc files. It's in Boost CVS::HEAD. Could you verify please?
Sure. I updated from CVS::HEAD and rebuilt lexed_tokens, which works correctly (like Boost 1.33.1) again. I also rebuilt my static analysis tool (it's basically a hacked up version of lexed_tokens) and analyzed my sources and got the same answers as before (minus the "invalid character" error). Thanks! Leo
participants (2)
-
Hartmut Kaiser -
Leo Davis