Hi Saturday, October 16, 2004, 3:53:37 PM, you wrote:

To come back to the initial problem of string tokenizing. I am trying to implement the finder in terms of ForwardIteratorT. Here is what I think I am expecting to do - also actually I am not very sure about it. I justed started to read M. Austern's book and I am quite new to concepts, requirements and so on.

There is nothing realy magical about the concepts. They just define how the type that should be subsituted as a template argument should look like. As for the finder, any function object, that takes begin, end and returns iterator_range will work. There is a requirement, that it must be able to work with the iterators of the sequence that it will be used to search.

So here is what I did so far:

[snip]

Is this the way I have to implement the finder or can it be implemented using some facilities in your library?

Finder is a lower concept. But there is nothing, that would forbid you to use other finders withing the current one. So you can for example use a find_first algorithm for search for '//' and then the same one (or just std::find) to search for following \n'

This implementation is quite low-level, so I am wondering if this is correct. Secondly, for the case this is correct, how should I return an empty range or what shall I return if didn't find a matching range?

Any empty range will do. The best way is to use make_range(End,End) So I would rewrite your finder like this: struct comment_finder { template<typename ForwardIteratorT> boost::iterator_range<ForwardIteratorT> operator()( ForwardIteratorT Begin, ForwardIteratorT End ) { // For convenience typedef iterator_range<ForwardIteratorT> TRange; // Look for comment start TRange rnComment= boost::find_first(boost::make_range(Begin,End), "//"); if(!rcComment.empty()) { // Look for '/n' ForwardIterator It=std::find(rnComment.end(), End, '\n'); if(It==End) { // FAILURE return boost::make_range(End,End) } // Increment It ++It; // SUCCESS return boost::make_range(rcComment.begin(), It); } // FAILURE return boost::make_range(End, End); } }; (Please note, that I have not compiled the code)

What are the following steps? Do I have to split two time times or how do I erase the comments out of the string?

I'm not sure exactly what you want to get. This finder will find all comments. You can easily erase them using boost::find_format_all( Cont, comment_finder(), boost::empty_formatter(Cont)); You may either do splitting in two steps, or to make another finder, that will look for separators, while it will be skipping comments.

Will the final token iterator be only of ForwardIteratorT or will be of the type of the conatiner that is holding the tokens. ( e.g. for a vector it would be a randow access iterator? )

All algorithms in the StringAlgo lib are templated. Generaly if you make alse the finder templated, you will get exatly the same iterator as the sequence that will go in. So the answer is yes. If you iterate over vector, you will get vector::iterator Regards, Pavol