Returning a tokenizer by value

15 Oct 2002

      Is there a problem with returning a tokenizer from a function?

Here's what I've got:

typedef boost::tokenizer< boost::char_separator<char> > tokenizer;

tokenizer getTokens( istream& is ) {
    boost::char_separator<char> sep(",","", boost::keep_empty_tokens);
    std::string str;
    getline(is,str,'\n');
    tokenizer toks( str, sep );
    return toks;
}

..
tokenizer tokens = getTokens(input);

tokenizer::iterator it = tokens.begin();

(*it) now contains garbage, interesting enough, iterating through the
rest of the container works.

when I move the processing of getTokens into the main code body the
first token (*it) is then correct.

Am I missing something fundamental here?

Returning a tokenizer by value

Brian_X7