Also, I had plans to do a multi-character token tokenizer, but I never got around to it. If someone would like to write one up, that would be great! Cheers, Jeremy On Tue, 3 Sep 2002, Jim.Hyslop wrote: jim.hy> Vijay (macrodisk@yahoo.com) wrote: jim.hy> > Is there anyway I can combine two tokens and check it as single one? jim.hy> > jim.hy> > I want to divide my string when both "-" and ">" occur together: jim.hy> > jim.hy> > Cell 12-1 -> xyz... jim.hy> > jim.hy> > I want to get the first token as Cell 12-1, how can I achive this via jim.hy> > tokenizer? jim.hy> I haven't used the tokenizer, but it seems to me that your TokenizerFunction jim.hy> can peek at the character after the '-' to see if it is a '>', to determine jim.hy> whether it should treat the '-' as a regular character or part of a jim.hy> separator. jim.hy> jim.hy> -- jim.hy> Jim jim.hy> jim.hy> jim.hy> Info: http://www.boost.org jim.hy> Wiki: http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl jim.hy> Unsubscribe: mailto:boost-users-unsubscribe@yahoogroups.com jim.hy> jim.hy> jim.hy> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ jim.hy> jim.hy> ---------------------------------------------------------------------- Jeremy Siek http://php.indiana.edu/~jsiek/ Ph.D. Student, Indiana Univ. B'ton email: jsiek@osl.iu.edu C++ Booster (http://www.boost.org) office phone: (812) 855-3608 ----------------------------------------------------------------------