But what I want to get is "my", "birthday is 1976", "15" and "17". In other words, I want the tokenizer either can give us the remaining of
the
string or can change the delimiters dynamically. Is there a way to do this?
You need to use two tokenizers, just like in the example you provided. The
only thing you need to do differently is retrieve an iterator to the current
position in the input string after the first tokenizer iterator (beg) has
been dereferenced, and pass the iterator range [current,end) to the
constructor of the tokenizer next rather than the whole string (s). You can
retrieve the current iterator from a token_iterator by calling the
iterator's member function base(). Here's your example again, modified to do
just that:
std::string s = "my birthday is 1976, 15/17";
boost::tokenizer<> tok(s);
boost::tokenizer<>::iterator beg=tok.begin();
// here *beg will give us "my"
typedef boost::tokenizer