Re: [Boost-users] Tokenizer: how to eliminate unwanted null tokens?

On Tue, 1 Nov 2005, CN wrote:

Hi!

The following code uses '\0' and ';' as seperators and produces the result:

<X><Y><><Z><>

string str=string("X;Y\0\0Z\0",7); typedef tokenizer Tok; escaped_list_separator<char> sep(string(),string("\0;",2),string()); Tok tokens(str, sep); for(Tok::iterator i = tokens.begin();i != tokens.end();++i) cout << "<" << *i << ">";

How to revise it in order to get this result?

<X><Y><Z>

The escaped_list_separator is designed to seperated fields for CSV-like inputs, which allow empty fields. You cannot use this type of separator here. The default char_separator would do the trick, but the problem is that it does not allow the null character ('\0') as a separator because of the way the string of separators is passed to its constructor (that is, a const Char*, null-terminated strings). But since the internal representation is a ::std::basic_string<>, then I wonder why the parameter type for the constructor is not the same. It would then solve the problem. As a work around, maybe you can just test if the tokens are empty strings before processing them? -- Fran�ois Duranleau LIGUM, Universit� de Montr�al "Voyager, c'est essayer de revenir avec quelques pr�jug�s en moins." - Jean-Daniel Lafond _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users