Re: [Boost-users] [Spirit] Qi lexeme only taking the first word

It's been a long while since I've used spirit::qi. But What it looks like is happeneing in your setup is something liek this, When you have: qi::rule full_id; the attribute is vector<string> When it matches id >> *(char_('.') >> id) this has an attribute of vector>> or something similar. spirit appears to compare your target attribute with the synthesised attribute of the parser and for any (trailing?) members of the synthesised attribute that do not match in your attribute, it marks them as unused_type and they are not assigned. You can see overload of assign to is used in your example if you breakpoint it -> boost\spirit\home\qi\detail\assign_to.hpp line 399. It appears in boost\spirit\home\qi\operator\sequence_base.hpp line 74, where the predicate traits::attribute_not_unused is passed to spirit::any_if (boost\spirit\home\support\algorithm\any_if.hpp line 186.) it will basically discard attributes where the LHS sequence is not matched with the RHS. You can see this in your example by adding an additional member to struct full_id_t { std::string val; std::vectorstd::string others; }; BOOST_FUSION_ADAPT_STRUCT(AST::full_id_t, val, others) Your missing bits will appear in this std::vector, as they are now not silently discarded. http://coliru.stacked-crooked.com/a/51f16c6deff45309 I think what the problem fundamentally is the attribute propagation is different when you have a string to when you have a vector<string> as in your two examples. the first kicks in whatever logic exists to flatten the LHS attribute into a string, the second takes the first element, assigns it and marks the rest as unused. One thing you can do is use qi::asstd::string()[ id >> *(char_('.') >> id) ] to force conversion of synthesised attribute to a string to happen before it is assigned to your attribute. http://coliru.stacked-crooked.com/a/6a060343a390f037 I've only had a quick look and this is pretty half hearted analysis. You'll really have to dig deep to find out exactly what is going on, but I suspect this is somewhat along the right lines. ________________________________ From: Boost-users on behalf of Michael Powell via Boost-users Sent: 06 November 2018 23:03 To: boost-users@lists.boost.org Cc: Michael Powell Subject: Re: [Boost-users] [Spirit] Qi lexeme only taking the first word On Tue, Nov 6, 2018 at 5:40 PM Michael Powell wrote:

On Tue, Nov 6, 2018 at 5:01 PM Michael Powell wrote:

...

Hello,

I've got a couple of rules that are perplexing to me. First,

rule id %= lexeme[qi::alpha >> *char_("A-Za-z0-9_")];

In and of itself, id is working fine. Then I've got a "full id":

rule full_id %= id >> *(char_('.') >> id);

Where:

struct full_id_t { std::string val; };

full_id_t::val is quite intentional for reasons elsewhere in the grammar.

The perplexity comes in, it seems lexeme is only shaving off the first word as the val.

For instance, parsing "two.oranges.red.test", I receive back "two" in the AST.

Perhaps I should defer specifying the lexeme part of id until later?

I elaborated a little on the "simple" full id sub-grammar, but I cannot repro using the GCC compiler. I'm wondering if this has anything to do with the VS2017 fpos issue?

http://coliru.stacked-crooked.com/a/adeb42ce2f19b0fd

Or there may be insufficient context in the web compiler to adequately demo.

I got a repro: http://coliru.stacked-crooked.com/a/069a44296240be7e Although the reasons as to why I do not know. It is a difference in attribute synthesis. When full_id synthesizes a std::string(), the conversion to full_id_t() "just works" magically. I'm guessing by happy accident based on the std::string val being the only member (adaptation, etc). But when I change the synthesis to be its "true" type, that is, AST::full_id_t(), suddenly I see the same behavior. Really and truly, I do not know why. Everything else being equal why would one approach be any different than the other? Anyone with some Spirit, Fusion, AST, insights? Thanks! For now, I'll run with it as has been exposed here, but it's a bit troubling to me not knowing the difference.

...

Thoughts? Suggestions?

Thank you!

Best regards,

Michael Powell

Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users