RE: [Boost-users] Tokenizer Question

for tokenizing on whitespace, simple stream input (>>) to a std::string suffices.

My own tokenizer does just that--and puts the tokens into a deque.

IMO, it's hardly worth troubling yourself with a tokenizer for whitespace.

Well, not really. When parsing line-oriented output and semi-known structured lines it's handy to be able to sometimes work with a line's tokens as if they were in a vector or deque. In fact, I was going to add a suggestion that the tokenizer also have the [] operator so that the individual tokens could be addressed as tok[1], etc. -Tom