Hi Edward, Several others have also found the return_delims parameter confusing, and I've proposed that we remove that parameter. John Bandela, would you mind if I go ahead and make that change? On 2/4/02 2:03 PM, "Edward Diener" <eddielee@tropicsoft.com> wrote:
Two tokenizer questions
1) I do not understand the constructor parameters to the char_delimits_separator class which is the default TokenizerFunction model. I assume the constructor would enable one to specify the delimiters for each token. I assume that the "returnable" parameter is a string literal of delimiters which are also returned as tokens and that "nonreturnable" is a string literal of delimiters which are not returned as tokens. I assume that the intersection of these string literals consists of all the delimiters. Given these assumptions, what is the purpose of the bool "return_delims" first parameter ? If "returnable" specifies a non-empty string literal, then why shouldn't these be the delimiters returned as tokens irregardless of the setting of "return_delims". If "returnable" specifies an empty string literal, then no delimiters will be returned as tokens once again irregardless of the setting of "return_delims". Would someone please explain why "return_delims" exists as a parameter in conjunction with the meaning of the "returnable" parameter ? It seems utterly redundant and unnecessary.
2) How does one create one own's TokenizerFunction to be plugged into the tokenizer class template as the first template type ? The documentation on TokenizerFunction concept leaves me utterly confused on a practical level. I assume I am creating a template class here despite the name which suggests it is a function, but the documentation never explains what it is. Maybe someone from Boost or John Bandela should consider updating it.
John, will you address this? Cheers, Jeremy -- Jeremy Siek http://www.osl.iu.edu/~jsiek Ph.D. Student, Indiana Univ. B'ton email: jsiek@osl.iu.edu C++ Booster (http://www.boost.org) office phone: (812) 855-3608