Phil Endecott wrote:
Is it true to say that stacked iterators implement a "data pull" style, while dataflow implements "data push"?
I also note that Arno wants to use stacked iterators because this alternative:
result = fn1( fn2( fn3( fn4( huge_document ) ) ) );
creates large intermediates and requires dynamic allocation.
Personally I like to think of iterator adaptors as a form of lazy evaluation, while algorithms are eager evaluation.
Again, a framework that allowed buffering of "sensible size" chunks and potentially distributed the work between threads could be a good solution.
If you perform n transformations, adaptors will give you loop fusion for free. That kind of optimization seems more interesting to me than work distribution and buffering.