At 10:52 2005-06-11, Tom Browder wrote:
I have used my own C++ tokenizer in the past, but I would like to use Boost's instead.
The predominant use of tokenizing for me is to split on white space,
for tokenizing on whitespace, simple stream input (>>) to a std::string suffices. IMO, it's hardly worth troubling yourself with a tokenizer for whitespace.
but Boost's default is to use white space AND punctuation. Is there any possibility to have either the default changed, or another TokenizerFunction added such as ws_separator, or something similar?
I know I can use
boost::char_separator<char> sep(" \n\t");
(but do I need to add "\v" to the char set?)
but I would rather have something like
boost::ws_separator sep;
and, better, make the ws_separator be the default TokenizerFunction for tokenizer.
Thanks for listening.
Tom Browder
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"