Re: [Boost-users] Tokenizer Question

My own tokenizer does just that--and puts the tokens into a deque.

Have you checked the string_algo library, specially the split function. string str1("hello abc-*-ABC-*-aBc goodbye"); // ranges with iterators in original string vector<iterator_range<string::iterator> > splitref; split(splitref, str1, is_any_of("*") ); // splitref == { "hello abc-","-ABC- ","-aBc goodbye" } // copies of found tokens list<string> splitcopy; split(splitcopy, str1, is_any_of("-*"), token_compress_on ); // splitcopy == { "hello abc","ABC","aBc goodbye" }