yield statement proposal

Hi all, As of version 2.0 C# will have yield statement. It intended for simplifying iterator development. Yield suspends execution of iterator code and resumes it on next increment. Thus complex traversal iterators are much easier to write in natural loop-like form instead of unroll them. Consider following example (C#): public IEnumerator< int > GetEnumerator() { for ( int i = 1; i < 4; ++i ) for ( int j = 1; j < 4; ++i ) yield return i * j; } Such iterator will generate following sequence: 1 2 3 2 4 6 3 6 9. The same code in C++ (using my library) will look like: class simple_generator { public: YIELD_BEGIN_WITH_CTOR( simple_generator, int ) YIELD_FOR_0( i = 1, i < 4, ++i ) YIELD_FOR_1( j = 1, j < 4, ++j ) YIELD_RETURN( i * j ) YIELD_END_FOR_1 YIELD_END_FOR_0 YIELD_END private: int i; int j; }; int main( int, char ** ) { for ( simple_generator g; !g.at_end(); ++g ) cout << *g << " "; return 0; } Have a look at few more examples: Example 1: YIELD_BEGIN_WITH_CTOR( simple_generator, string ) YIELD_RETURN( "Hello," ) YIELD_RETURN( "world!" ) YIELD_END Will print "Hello, world!" in two iterations with the above main() function. Example 2: YIELD_BEGIN_WITH_CTOR( simple_generator, int ) YIELD_FOR( i = 0, i < 10, ++i ) YIELD_IF( i % 2 == 0 ) YIELD_CONTINUE YIELD_END_IF YIELD_RETURN( i ) YIELD_END_FOR YIELD_END Will print "1 3 5 7 9 ". If anyone interested in such thing? Sincerely yours, Maksym Motornyy.

Maksym Motornyy wrote:
Hi all,
As of version 2.0 C# will have yield statement. It intended for simplifying iterator development. Yield suspends execution of iterator code and resumes it on next increment. Thus complex traversal iterators are much easier to write in natural loop-like form instead of unroll them.
Nicely stolen from Python <g>.

Edward Diener wrote:
Maksym Motornyy wrote: [...]
As of version 2.0 C# will have yield statement. It intended for simplifying iterator development. Yield suspends execution of iterator code and resumes it on next increment. Thus complex traversal iterators are much easier to write in natural loop-like form instead of unroll them.
Nicely stolen from Python <g>.
I think that yield is part of the general concept of coroutines and has not really been invented by Python nor C# (Knuth if I remember correctly). Then again, I did not really read anything about the C# yield, so I'm not sure it is really about coroutines (btw, why the "yield return" syntax and not just "yield"?). Speaking of more Boost centric stuff? There has been any proposal of a coroutine library? I think it should definitely be useful and not really that hard to implement (fibers on windows and makecontext or the setjump/sigaltstack trick on unices). Anyone interested? Regards, ---- Giovanni P. Deretta

Giovanni P. Deretta wrote:
Speaking of more Boost centric stuff? There has been any proposal of a coroutine library? I think it should definitely be useful and not really that hard to implement (fibers on windows and makecontext or the setjump/sigaltstack trick on unices). Anyone interested?
Can I have it yesterday? I'm serious.. The closest I've seen to something I could use is: http://state-threads.sourceforge.net/ -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org

Rene Rivera wrote:
Giovanni P. Deretta wrote:
Speaking of more Boost centric stuff? There has been any proposal of a coroutine library? I think it should definitely be useful and not really that hard to implement (fibers on windows and makecontext or the setjump/sigaltstack trick on unices). Anyone interested?
Can I have it yesterday?
I'm serious.. The closest I've seen to something I could use is: http://state-threads.sourceforge.net/
I remember Daniel James had written a coroutine library. Here's the message: http://lists.boost.org/boost/2005/01/18687.php. The code is available here, but doesn't appear to be well documented: ttp://myweb.tiscali.co.uk/calamity/code/coroutines.tar.gz. Jonathan

Jonathan Turkanis wrote:
message: http://lists.boost.org/boost/2005/01/18687.php. The code is available here, but doesn't appear to be well documented: ttp://myweb.tiscali.co.uk/calamity/code/coroutines.tar.gz.
^ h Jonathan

Jonathan Turkanis wrote:
I remember Daniel James had written a coroutine library. Here's the message: http://lists.boost.org/boost/2005/01/18687.php. The code is available here, but doesn't appear to be well documented: ttp://myweb.tiscali.co.uk/calamity/code/coroutines.tar.gz.
It's also a mess and very, very ugly :(. But it does work. I was really just experimenting with how far I could push the idea. I was thinking of trying to simplify it a bit but never got round to it as I didn't have any need for this myself. One thing I was considering was making use of phoenix's closures, although that might have ended up being quite verbose. But it wouldn't be to hard to add recursion to something like Maksym's version which would make it a lot more powerful. And I'm not sure why he needs 'YIELD_FOR' and 'YIELD_IF'. I think it could quite easily be written so that his example would be: class simple_generator { public: YIELD_BEGIN_WITH_CTOR( simple_generator, int ) for( i = 1, i < 4, ++i ) { for( j = 1, j < 4, ++j ) { YIELD_RETURN( i * j ) } } YIELD_END private: int i; int j; };

Speaking of more Boost centric stuff? There has been any proposal of a coroutine library? I think it should definitely be useful and not really that hard to implement (fibers on windows and makecontext or the setjump/sigaltstack trick on unices). Anyone interested?
I'm interested in implement and have it. Thanks for ideas and links. I want to try fibers approach. Sincerely, Maksym.

I'm interested in implement and have it.
I have to prototypes now. First uses Simon Tatham's trick with case statement. Generator code looks as follow: class simple_generator: public coroutine { public: simple_generator( int j ) : i( j ) { } BEGIN_COROUTINE( int ) for ( ; i < 10; ++i ) { YIELD( i ) } YIELD( -1 ) END_COROUTINE private: int i; }; Second uses Win32 fibers: class simple_generator : public coroutine< simple_generator, int > { public: simple_generator( int j ) : j( j ) { } void execute() { for ( int i = j; i < 10; ++i ) yield( i ); yield( -1 ); } private: int j; }; Both being used in this code: void main() { simple_generator g( 8 ); while ( true ) { int result = g(); if ( result < 0 ) break; cout << result << " "; } } produce output "8 9 ". Is anyone interested in details and further discussion? Sincerely, Maksym.

Giovanni P. Deretta wrote:
Speaking of more Boost centric stuff? There has been any proposal of a coroutine library? I think it should definitely be useful and not really that hard to implement (fibers on windows and makecontext or the setjump/sigaltstack trick on unices). Anyone interested?
IIRC there was an article in C++ user's journal which implemented coroutines using fibres on windows and threads on unix. I'll have a look for it later. Also there are these article on fibres (these are the 2nd and 3rd parts, the first doesn't seem to be available): http://blogs.msdn.com/oldnewthing/archive/2004/12/30/344281.aspx http://blogs.msdn.com/oldnewthing/archive/2004/12/31/344799.aspx Make sure you read the third part - it demonstrates that fibre-safety is very different to thread safety (similar warning apply to coroutines in general). I think setjump would also have some serious problems - you could only have a single coroutine at a time and it could lead to undefined behaviour in lots of nasty ways. Daniel

Daniel James wrote:
Giovanni P. Deretta wrote:
IIRC there was an article in C++ user's journal which implemented coroutines using fibres on windows and threads on unix. I'll have a look for it later. Also there are these article on fibres (these are the 2nd and 3rd parts, the first doesn't seem to be available):
http://blogs.msdn.com/oldnewthing/archive/2004/12/30/344281.aspx http://blogs.msdn.com/oldnewthing/archive/2004/12/31/344799.aspx
Interesting, I'll need to look at the Win32 fiber api more throughly.
Make sure you read the third part - it demonstrates that fibre-safety is very different to thread safety (similar warning apply to coroutines in general).
From a quick look at the article seems that many of these fibre pitfalls are just an artifact of the win32 continuation implementation. Anyway, i think that it is *much* easier to write fibre-safe code than thread-safe code (yay, no locks!).
I think setjump would also have some serious problems - you could only have a single coroutine at a time and it could lead to undefined behaviour in lots of nasty ways.
Well setjump by itself let you only unwind the stack, but used with the sigaltstack trick (see the paper at the Pth site http://www.gnu.org/software/pth/rse-pmt.ps ) you can implement full coroutines. It is portable to all POSIX systems and is probably even more portable than the POSIX makecontext API itself, that not only is slower (it requires a system call to save/restore the sigmask), but it has also been marked as obsolescent in the lastest Single Unix Specification because of ISO C changes). Regards, PS Btw, after reading some articles about the C# yield it seems it is *not* a true form of coroutine but it is limited to iterators (the compiler just rewrites the code inside out). ---- Giovanni P. Deretta

Maksym Motornyy <mmotorny <at> yandex.ru> writes:
The same code in C++ (using my library) will look like:
class simple_generator { public: YIELD_BEGIN_WITH_CTOR( simple_generator, int ) YIELD_FOR_0( i = 1, i < 4, ++i ) YIELD_FOR_1( j = 1, j < 4, ++j ) YIELD_RETURN( i * j ) YIELD_END_FOR_1 YIELD_END_FOR_0 YIELD_END
private: int i;
int j; };
Nice idea - but a tad ugly. Have you seen http://www.chiark.greenend.org.uk/~sgtatham/coroutines.html which deascribes a similar approach in C? I don't actually find this terribly compelling - and it looks like the C# implementation is just as limited, but at least has nicer syntax than a macro hack can manage. I would be far more interested in real coroutines. Regards Darryl.

Darryl Green wrote:
Nice idea - but a tad ugly.
Have you seen http://www.chiark.greenend.org.uk/~sgtatham/coroutines.html which deascribes a similar approach in C?
Daniel said his library was inspired by this approach. Jonathan
participants (7)
-
Daniel James
-
Darryl Green
-
Edward Diener
-
Giovanni P. Deretta
-
Jonathan Turkanis
-
Maksym Motornyy
-
Rene Rivera