I'm having a problem with tokenizer::iterator crashing the program when I increment it.

tokens.assign(string("1234#4567#1##"), separator("#"));

The tokenizer does not copy the sequence (the string) but only remembers its begin and end iterators. You are constructing a temporary string object which will be destroyed immediately at the end of the line, thus rendering the iterators invalid. Make sure that the string exists while you are using the tokenizer and its iterators on it, e.g. by making it a class member. Benjamin