This may not matter for the CSV file you're parsing, but at least for a more general solution for CSV processing, you'd also have to handle fields that are surrounded by quotes and may even contain embedded commas. I don't know if split or tokenizer can handle that. -- Bill -- _____ From: Larry [mailto:lknain@nc.rr.com] Sent: Thursday, December 13, 2007 8:08 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [boost-users] tokenizer vs string algorithm split. My limited experience is that tokenizer is faster. I have tried it several times in different schemes but the tokenizer always seems to come out faster by more than a little. I would prefer the split() scheme but I haven't found the way to make it go faster. Larry ----- Original Message ----- From: chun ping wang <mailto:cablepuff@gmail.com> Newsgroups: gmane.comp.lib.boost.user To: boost-users@lists.boost.org Sent: Wednesday, December 12, 2007 10:56 PM Subject: [boost-users] tokenizer vs string algorithm split. Hi I was wondering which one is better and faster to split a file of csv value of number and put it into container of double. 1.) Which option is better. // method 1. std::vector<std::string> split_string; boost::algorithm::trim(flist); boost::algorithm::split(split_string, flist, boost::algorithm::is_any_of(",")); std::vector<double> elements; BOOST_FOREACH(std::string s, split_string) { elements += boost::lexical_cast<double>(s); } // method 2. boost::char_separator<char> sep(","); boost::tokenizer<boost::char_separator<char> > tokens(flist, sep); std::vector<double> elements; BOOST_FOREACH(std::string token, tokens) { elements += boost::lexical_cast<double>(token); } 2.) When is it better to use string algorithm split instead of tokenizer and vice versa. _____ _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users