Hi All, This is what I've done to fix the confusing constructor of char_delimiters_separator. I basically wanted to add another less-confusing constructor to the class, and document the old constructor as deprecated (don't want to break existing code). However, because of the default arguments in the original constructor, there would be bad interactions between the new constructor and the old. Therefore, I instead just created a new class named char_separator and put a note in the documention for char_delimiters_separator that the whole class is deprecated. I've checked in the code, tests, examples, and documentation for char_separator. Let me know what you think. Cheers, Jeremy P.S. It is still a bit annoying that it takes several lines to create a tokenizer. When I get the time I'll had type generators and helper functions (a la the iterator adaptor library) to make it easier to create tokenizers (or if anyone else has the time, go for it!). On Mon, 4 Feb 2002, Jeremy Siek wrote: jsiek> Hi Edward, jsiek> jsiek> Several others have also found the return_delims parameter confusing, and jsiek> I've proposed that we remove that parameter. jsiek> jsiek> John Bandela, would you mind if I go ahead and make that change? jsiek> jsiek> On 2/4/02 2:03 PM, "Edward Diener" <eddielee@tropicsoft.com> wrote: jsiek> > Two tokenizer questions jsiek> > jsiek> > 1) I do not understand the constructor parameters to the jsiek> > char_delimits_separator class which is the default TokenizerFunction jsiek> > model. I assume the constructor would enable one to specify the jsiek> > delimiters for each token. I assume that the "returnable" parameter is a jsiek> > string literal of delimiters which are also returned as tokens and that jsiek> > "nonreturnable" is a string literal of delimiters which are not returned jsiek> > as tokens. I assume that the intersection of these string literals jsiek> > consists of all the delimiters. Given these assumptions, what is the jsiek> > purpose of the bool "return_delims" first parameter ? If "returnable" jsiek> > specifies a non-empty string literal, then why shouldn't these be the jsiek> > delimiters returned as tokens irregardless of the setting of jsiek> > "return_delims". If "returnable" specifies an empty string literal, then jsiek> > no delimiters will be returned as tokens once again irregardless of the jsiek> > setting of "return_delims". Would someone please explain why jsiek> > "return_delims" exists as a parameter in conjunction with the meaning of jsiek> > the "returnable" parameter ? It seems utterly redundant and unnecessary. jsiek> > jsiek> > 2) How does one create one own's TokenizerFunction to be plugged into jsiek> > the tokenizer class template as the first template type ? The jsiek> > documentation on TokenizerFunction concept leaves me utterly confused on jsiek> > a practical level. I assume I am creating a template class here despite jsiek> > the name which suggests it is a function, but the documentation never jsiek> > explains what it is. Maybe someone from Boost or John Bandela should jsiek> > consider updating it. jsiek> jsiek> John, will you address this? jsiek> jsiek> Cheers, jsiek> Jeremy jsiek> jsiek> -- jsiek> Jeremy Siek http://www.osl.iu.edu/~jsiek jsiek> Ph.D. Student, Indiana Univ. B'ton email: jsiek@osl.iu.edu jsiek> C++ Booster (http://www.boost.org) office phone: (812) 855-3608 jsiek> jsiek> jsiek> jsiek> Info: <http://www.boost.org> jsiek> Wiki: <http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl> jsiek> Unsubscribe: <mailto:boost-users-unsubscribe@yahoogroups.com> jsiek> jsiek> jsiek> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ jsiek> jsiek> ---------------------------------------------------------------------- Jeremy Siek http://php.indiana.edu/~jsiek/ Ph.D. Student, Indiana Univ. B'ton email: jsiek@osl.iu.edu C++ Booster (http://www.boost.org) office phone: (812) 855-3608 ----------------------------------------------------------------------