Re: [Boost-users] [tokenizer] boost::tokenizer doesn't handle '/' properly?

On 3/23/2010 11:28 AM, Naked Short Selling wrote:

Greetings,

I'm trying to parse a csv file using boost tokenizer. But I see weird result when there are cells like "IG9/10/7-10:USCEFLCORR0001", tokenizer returns tokens like "IG9/,0/7-,0:USCEFLCORR000". I am assuming it's because of slash '/'.

Here's my code snippet: " typedef boost::tokenizer tokenizer; char delim = ','; boost::char_separator<char> sep(&delim); ... tokenizer tokens(str, sep); tokenizer::iterator iter = tokens.begin(); tokenizer::iterator end = tokens.end(); "

Please advise.

Thanks!

I think you actually want: boost::char_separator<char> sep(",", boost::keep_empty_tokens); You can use boost::drop_empty_tokens if you want, but since you're parsing CSV I doubt you want that. Brett