If your CSV has empty fields (e.g., data,data,,data.....) the only way I found to handle the empty field was to handle the separators yourself with the tokenizer otherwise the tokenizer would skip the field (a la strtok()). For CSVs I tried Spirit and came up with a scheme (with lots of help I would add) that seemed to work. Not many lines of code. It takes more time than I was interested in spending to figure it out. Larry ----- Original Message ----- From: "Edward Diener" <eldiener@tropicsoft.com> Newsgroups: gmane.comp.lib.boost.user To: <boost-users@lists.boost.org> Sent: Saturday, December 15, 2007 9:44 AM Subject: Re: [boost-users] tokenizer vs string algorithm split. Bill Buklis wrote:
This may not matter for the CSV file you’re parsing, but at least for a more general solution for CSV processing, you’d also have to handle fields that are surrounded by quotes and may even contain embedded commas. I don’t know if split or tokenizer can handle that.
Tokenizer's escaped_list_separator handles quotes and embedded commas properly.