OvermindDL1 wrote:

...

One of my main current interests is parsing. Trying to decide among the choices:

- Regex - Spirit - Xpressive

Depends on what you are wanting to parse. If you want to do, say, a search and replace in a file, Xpressive is best, if you want to parse data structures and you want the absolute best speed and a completely unambiguous grammar, Spirit2.1 for sure. Do not bother with Regex itself as Xpressive can do everything Regex can, but more and better.

Thanks so much!, OvermindDL1... Allow me to describe my target data. I initially had a bunch of files with lines like this: Variable Name = Variable Value These are some examples: -------------------------------------------------------------------- My Favorite Baseball Player = George Herman "Babe" Ruth What did you do on Christmas = I rested, computed the % mortgage and visited my brother + sister. (the above should be in a single line) Favorite Curse = That umpire is a #&*%! -------------------------------------------------------------------- I quickly solved the above parsing with Regex like this: string variable = "([A-Za-z0-9][\\w\\h\\(\\)\\-\\.,/&]*)"; char equal_sign = '='; string value = "(.+)"; assignment = variable + equal_sign + value; After retrieving the LHS and the RHS I store them for subsequent use in a map data structure. My data, however, just became a bit more challenging. It is now divided into blocks: [Unique ID 1] Variable Name = Variable Value Variable Name = Variable Value Variable Name = Variable Value [Unique ID 2] Variable Name = Variable Value Variable Name = Variable Value Variable Name = Variable Value [Unique ID 3] Variable Name = Variable Value Variable Name = Variable Value Variable Name = Variable Value (etc.) Again, I would like to store the new format in a map, using the Unique ID as key to retrieve the block of lines underneath each ID. At this stage, I am wondering whether to continue using true and tried (and learned!) Regex, or get my feet wet into more powerful tools, such as the one recommended by Overmind (Xpressive). How does Xpressive compare with ANTLR? I am torn between them. TIA, -Ramon

This makes me wonder how Xpressive and Spirit compare, both do compiled parsing statements right? Can Spirit somehow be seen as Xpressive + more? Why use Xpressive at all then? Best, Dee On Sun, Sep 13, 2009 at 5:59 AM, OvermindDL1 wrote:

On Sat, Sep 12, 2009 at 1:40 PM, Ramon F Herrera wrote:

...

OvermindDL1 wrote:

...

...

One of my main current interests is parsing. Trying to decide among the choices:

- Regex - Spirit - Xpressive

Depends on what you are wanting to parse.  If you want to do, say, a search and replace in a file, Xpressive is best, if you want to parse data structures and you want the absolute best speed and a completely unambiguous grammar, Spirit2.1 for sure.  Do not bother with Regex itself as Xpressive can do everything Regex can, but more and better.

Thanks so much!, OvermindDL1...

Allow me to describe my target data. I initially had a bunch of files with lines like this:

Variable Name = Variable Value

These are some examples:

-------------------------------------------------------------------- My Favorite Baseball Player = George Herman "Babe" Ruth

What did you do on Christmas = I rested, computed the % mortgage and visited my brother + sister.

(the above should be in a single line)

Favorite Curse = That umpire is a #&*%! --------------------------------------------------------------------

I quickly solved the above parsing with Regex like this:

string variable = "([A-Za-z0-9][\\w\\h\\(\\)\\-\\.,/&]*)"; char equal_sign = '='; string value    = "(.+)"; assignment      = variable + equal_sign + value;

After retrieving the LHS and the RHS I store them for subsequent use in a map data structure.

My data, however, just became a bit more challenging. It is now divided into blocks:

[Unique ID 1] Variable Name = Variable Value Variable Name = Variable Value Variable Name = Variable Value

[Unique ID 2] Variable Name = Variable Value Variable Name = Variable Value Variable Name = Variable Value

[Unique ID 3] Variable Name = Variable Value Variable Name = Variable Value Variable Name = Variable Value

(etc.)

Again, I would like to store the new format in a map, using the Unique ID as key to retrieve the block of lines underneath each ID.

Actually, that kind of stuff is very easy to do in Spirit2.1 (in the boost trunk or Boost 1.41), it can auto-fill your structures and everything, and it is very fast.

On Sat, Sep 12, 2009 at 1:40 PM, Ramon F Herrera wrote:

...

At this stage, I am wondering whether to continue using true and tried (and learned!) Regex, or get my feet wet into more powerful tools, such as the one recommended by Overmind (Xpressive).

As stated, Xpressive can do all Regex can do, but you can also do static regex's (compiled by the C++ grammar, much faster then a string regex), but Spirit2.1 would still be a lot faster overall (it has been timed against a lot of things, and it blows even Xpressive's static parsers away).

On Sat, Sep 12, 2009 at 1:40 PM, Ramon F Herrera wrote:

...

How does Xpressive compare with ANTLR? I am torn between them.

Xpressive and ANTLR are two different things.  ANTLR is like a not-as-powerful-and-slower Spirit2.1, a full grammar parser, where Xpressive is just a regex parser. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

On Sat, Sep 12, 2009 at 7:04 PM, Ramon F Herrera wrote:

Diederick C. Niehorster wrote:

...

This makes me wonder how Xpressive and Spirit compare, both do compiled parsing statements right?

Can Spirit somehow be seen as Xpressive + more? Why use Xpressive at all then?

Best, Dee

Hi Dee,

While I am far from being an expert (and hope to read answers from people more qualified than myself), I was wondering the same thing.

I venture a guess. You draw the line here:

Several tools (Regex, Xpressive, Perl) can grab data, based on regular expressions. They can validate whether some statement is a correct expression of the target "language".

The more advanced tools, however, can actually put that data into action. The passive data becomes a set of executable statements using Spirit, ANTLR, etc. You specify things like: "every time you find the verb "such-and-such" call my function XYZ with the needed parameters.

Callback functions are the dividing line.

Static Xpressive (not dynamic Xpressive, you can think of dynamic Xpressive as being Boost.Regex exactly) and Spirit are both C++ DSEL's and compile to rather fast code, but Xpressive is a REGEX parser, and as such has limitations, where Spirit2.1 is a PEG (Parsing Expression Grammer as I recall, wiki it) grammar. PEG's are nice in that they are unambiguous, they are faster, then have unlimited lookahead, etc... Spirit can also be bound to just about anything in any way in the C++ world, with built-in parsers for a ton of things (everything from POD's to the STL to many Boost libraries like Fusion and such), and it is quite easy to make your own new things as well. For a comparison of Spirit2.1 with ANTLR, Spirit2.1 has been shown to be faster in execution speed, the code is a great deal shorter, and ANTLR's actions pale in comparison to Spirit2.1's versions, plus the fact you do not need to pre-parse code with an external app like you have to do with ANTLR. But yea, to learn what Spirit2.1 is built off of, look up PEG's on wikipedia, and yet Spirit2.1 is still so much more powerful then that. The documentation for it is in trunk. But as an example, I wrote this grammer a free hours ago, it is a relatively nasty looking one that I really should break up into easier to read parts, but it works quite well in my testing thus far: static boost::spirit::qi::rule uri_decode_rule; uint_parser _string_value_hex; uri_decode_rule = (lit('%') >> _string_value_hex[_val=boost::spirit::_1]) | (lit('+') >> eps[_val=' ']) | char_[_val=boost::spirit::_1] ; -no_case[+(uri_decode_rule-char_(':')) >> lit("://")] //scheme >> -no_case[+(uri_decode_rule-lit('/'))] // netloc >> ( lit('/') >> ( (+(uri_decode_rule-char_("/?;#")) >> -(lit(';') >> (+(uri_decode_rule-char_(";=?/#")) >> -(lit('=')

...

(+(uri_decode_rule-char_(";,?/#")))%lit(',')))%lit(';')) // params (ex: ;param=val1,val2,val3;p2) ) % lit('/')) >> // path -(lit('?') >> -((+(uri_decode_rule-char_("&=;#")) >> -(lit('=') +(uri_decode_rule-char_("&;#")))) % omit[char_("&;")] >> omit[-char_("&;")])) >> // query -(lit('#') >> *uri_decode_rule) // fragment )

It parses a URI (a type that only my system will generate, so I am not sure it follows the spec exactly, but it works very well for my purpose, and it is very fast). Only thing it does is parse all the info into this: struct uri { std::string scheme; std::string netloc; typedef std::vector< std::pair< std::string, std::vectorstd::string > > params_t; typedef std::vector< std::pairstd::string,params_t > path_t; path_t path; typedef std::vector< std::pairstd::string,std::string > query_t; query_t query; std::string fragment; };

OvermindDL1 wrote:

But yea, to learn what Spirit2.1 is built off of, look up PEG's on wikipedia, and yet Spirit2.1 is still so much more powerful then that. The documentation for it is in trunk. But as an example, I wrote this grammer a free hours ago, it is a relatively nasty looking one that I really should break up into easier to read parts, but it works quite well in my testing thus far:

[snip code] It would help if you use fewer indent spaces and using declarations ;-) Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net

Ramon F Herrera wrote:

OvermindDL1 wrote:

...

...

One of my main current interests is parsing. Trying to decide among the choices:

- Regex - Spirit - Xpressive

Depends on what you are wanting to parse. If you want to do, say, a search and replace in a file, Xpressive is best, if you want to parse data structures and you want the absolute best speed and a completely unambiguous grammar, Spirit2.1 for sure. Do not bother with Regex itself as Xpressive can do everything Regex can, but more and better.

Thanks so much!, OvermindDL1...

Allow me to describe my target data. I initially had a bunch of files with lines like this:

[...] By the looks of your description, Spirit2 sounds just like the right tool alright. Check out the tutorial: http://tinyurl.com/ozdsjo Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net