Re: [boost] [tokenizer] tokenizing by strings rather than chars
Andrzej Krzemienski wrote:
I want to parse numbers separated by "new-lines", however, on my windows system, a "new-line" is represented by two consecutive characters: 13, 10 (or in other words by string "\n\r"). So, only if I encounter a sequence of these two chars do I want to cut the token.
Spirit.Qi would do that handily. Something like this should work: namespace qi = boost::spirit::qi; std::vector<int> data; qi::phrase_parse(input.begin(), input.end(), qi::int_, qi::eol, data);
Thanks for the reply. And could I ask a similar question? I wanted to simplify my real task to make it shorter and clearer to understand, and it look like I changed it to a different one. My apologies. My real task is to check if the data in Windows clip-board is a column of data (copied from a grid or table). Such a column is just text separated with "\n\r" sequences. So, for example these are valid data: "12" - a column of height 1 with data "12", " a\n\r b" - a column of height 2 with two with data " a" and " b" "-1\n\r\n\r3" - a column of height 3 with datum "-1", empty cell, and datum "3". The spirit solution above will skip the empty cell in my third example. On the other hand, I do need the information about empty cells too. It really looks more like a tokenizing task. Or is there a way to do it easily in Spirit? Regards, &rzej
Andrzej Krzemienski wrote:
Rob Stewart wrote:
Andrzej Krzemienski wrote:
Spirit.Qi would do that handily. Something like this should work: namespace qi = boost::spirit::qi; std::vector<int> data; qi::phrase_parse(input.begin(), input.end(), *qi::int_, qi::eol, data);
Thanks for the reply. And could I ask a similar question? I wanted to simplify my real task to make it shorter and clearer to understand, and it look like I changed it to a different one. My apologies.
NP
My real task is to check if the data in Windows clip-board is a column of data (copied from a grid or table). Such a column is just text separated with "\n\r" sequences. So, for example these are valid data: "12" - a column of height 1 with data "12", " a\n\r b" - a column of height 2 with two with data " a" and " b" "-1\n\r\n\r3" - a column of height 3 with datum "-1", empty cell, and datum "3".
The spirit solution above will skip the empty cell in my third example. On the other hand, I do need the information about empty cells too.
In that case, you'd use qi::parse(), because the qi::eol characters become significant (you don't want them skipped -- ignored). However, there's an issue: you need to know whether you actually got a value since you need to account for empty cells. Presuming that there's a sentinel value to use to indicate no value, such as std::numeric_limits<int>::min(), then, this should work: qi::parse(input.begin(), input.end(), *( ( qi::int_ | qi::attr((std::numeric_limits<int>::min)()) ) >> qi::eol ), data); If there is no sentinel value, then you could use a std::vector<boost::optional<int>> and qi::attr(boost::optional<int>()). HTH _____ Rob Stewart robert.stewart@sig.com Software Engineer using std::disclaimer; Dev Tools & Components Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
participants (2)
-
Andrzej Krzemienski -
Stewart, Robert