
20 Feb
2008
20 Feb
'08
3:01 a.m.
Is there any input to boost::tokenizer that will produce exactly one empty token? Consider the following code: typedef boost::char_separator<char> Separator; typedef boost::tokenizer<Separator> Tokenizer; Separator sep("|" , "", keep_empty_tokens); Tokenizer tokenizer1("", sep); Tokenizer tokenizer2("|", sep); The first case, where an empty string is tokenized, produces no tokens. The second case, where a string containing just a separator character is split, produces two empty tokens. It seems odd to me that there’s no input that will produce just one empty token. Joe Gottman