On Sun, 22 Feb 2004 21:50:34 -0500, Beman Dawes wrote:
Ouch! That seems an unfortunate design decision. Perhaps two forms of constructors could have been provided - one form that takes two iterators and doesn't store a copy of the contents, and another form like the current constructors which takes a container reference but (unlike the current constructor) makes a copy.
Actually, as I can see, both forms are currently provided, and both work the same way (store begin and end iterators). I think that proposed change will not break tokenizer. Few usage scenarios comes to my mind: 1. tokenizer initilized with temporary - currently undefined behaviour, will be well defined; 2. tokenizer initialized with const reference - will make extra copy of input data; 3. like 2, afterwards this reference is modified (ouside tokenizer, of course) - undefined behaviour or well defined behaviour, depending if iterators taken by tokenizer constructor are still valid. Here most important change of behaviour will be seen (collection used by tokenizer won't see changes of input data). 4. tokenizer initialized with iterators - will work exactly the same way as currently. 5. assign from iterators - will work exactly the same way as currently 6. assign from const reference - will make extra copy of input data. Again changed of class behaviour - any change on input data won't be seen by tokenizer. 7. assignment from temporary - currently undefined behaviour, will be well defined. Iterator-based constructor is clearly expressing idea that tokenizer is *iterating* over given collection (input string). Also (IMVHO) expected behaviour when initializing from const-reference is to make a copy, as shown by recent posts on boost-users list. I think that proposed change will make tokenizer behaviour closer to user expectations. Another option (not really better one) is to add extra bool parameter to constructor, default = false, to express if extra copy of data should be made.
I wonder if that was discussed at the time? Or is there a better way to prevent the problem?
I remember thread "Perfect Forwarding" where David presented clever way to detect copy-from--temporary-rvalue, but I do not know if any of his ideas are applicable here. It would be sooo much cleaner just to add constructor like: template <typename Container> tokenizer(const Container&& c, const TokenizerFunc& f); // make a copy unfortunatelly proposal to add "&&" syntax (N1377) is still under Committee consideration ... B.