Jeremy Many thanks for this! Paul Paul Grenyer pjgrenyer@iee.org ----- Original Message ----- From: Jeremy Siek To: Boost-Users@yahoogroups.com Sent: Friday, November 23, 2001 8:53 PM Subject: Re: [Boost-Users] boost tokenizer Hi Paul, By default the char_delimiters_separator (which is the default TokenizerFunction for token_iterator and tokenizer) splits on any punctuation, which I assume includes '_'. To explicitly specify the splitting characters, you pass a string of the separator characters into the constructor, like this: char_delimiters_separator<char> sep(false, " |"); // create token iterator or tokenizer using sep Cheers, Jeremy On 11/23/01 10:21 AM, "pjgrenyer@iee.org" wrote:

Dear All

I'm using the boost tokenizer to tokenise lines similar to this one:

first_initial |23|1|

This gives me:

first initial 23 1

but what I actually want is:

first_initial 23 1

Does anyone know how to get it to ignore the underscore?

Thanks Paul

-- Jeremy Siek http://www.osl.iu.edu/~jsiek Ph.D. Student, Indiana Univ. B'ton email: jsiek@osl.iu.edu C++ Booster (http://www.boost.org) office phone: (812) 855-3608 Yahoo! Groups Sponsor Info: http://www.boost.org Wiki: http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl Unsubscribe: mailto:boost-users-unsubscribe@yahoogroups.com Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service. [Non-text portions of this message have been removed]