Joel de Guzman wrote:
The main selling point of lexertl as far as boost goes is probably speed. lexertl lexers are both fast to construct (typically under 100 milliseconds on a modern machine) and fast to tokenise input (the generated state machine uses the flex technique of equivalence classes).
I personally like the fact that you can dump the DFA as data and process it later with any language you like too.
Spirit is a parser, please don't compare apples and oranges. You cannot implement, say, Wave, with just a lexer. I suggest you don't go the "this-is-better-that-is-better" route. Spirit has its own lexer too, FYI. It's called SLex. See Slex in http://tinyurl.com/29mcn
Oh and BTW, if you want to talk about speed, matching the speed of Flex is not good enough. The thing to beat is Re2C! Hartmut and I shall see how lexertl fares soon. Wave has an adaptable front end where you can choose your own lexer. Re2C (http://re2c.org/) is one of them.
Yes, and Slex is the other one. BTW, Slex seems to be very similar to lexertl (with the exeption of not allowing to optimize the constructed state machine tables - but this is not a principal issue, merely lack of time). Wave is based on a modular (layered) design. The lexer sits on top of the input (character) stream, producing C++ tokens, exposed via an iterator interface. The preprocessing component consumes the lexer iterators and is almost completely independent. The only dependency is that both have to use the same token type (which is a template parameter to both). It is very easy to interface a different lexer to the preprocessor. The cpp_tokens example (libs/wave/samples/cpp_tokens) demonstrates this by using the Slex based lexer. I'll elaborate on this in another mail during the next couple of days. Regards Hartmut