[coroutine] interface suggestion
I forgot a suggestion made by Vicente: 5.a coroutine<>::caller_t provides function bind() which must be called after coroutine::caller_t::yield() returned in order to get the parameters int f( caller_t & c, strg & s, int x) { c.yield( 7); // alternative c( 7) c.bind( s, x); // s and x will contain new values given by caller } alternative: because the user could forget to invoke bind(): int f( caller_t & c, strg & s, int x) { c.yield( 7, s, x); // alternative c( 7, s ,x) } Oliver
I forgot a suggestion made by Vicente:
5.a coroutine<>::caller_t provides function bind() which must be called after coroutine::caller_t::yield() returned in order to get the parameters
int f( caller_t & c, strg & s, int x) { c.yield( 7); // alternative c( 7) c.bind( s, x); // s and x will contain new values given by caller } My suggestion was to call bind just once at the beginning of function and before any call to yield. If you want to ensure that the bind has been done, you could request
Le 19/09/12 14:23, Oliver Kowalke a écrit : that yield is provided by the result of the bind operation, as e.g. int f( caller_t & c, strg & s, int x) { auto b = c.bind( s, x); // s and x are now used to store the next calls. b.yield( 7); // alternative b( 7) // here s and x have been reassigned } Best, Vicente
Am 19.09.2012 20:12, schrieb Vicente J. Botet Escriba:
I forgot a suggestion made by Vicente:
5.a coroutine<>::caller_t provides function bind() which must be called after coroutine::caller_t::yield() returned in order to get the parameters
int f( caller_t & c, strg & s, int x) { c.yield( 7); // alternative c( 7) c.bind( s, x); // s and x will contain new values given by caller } My suggestion was to call bind just once at the beginning of function and before any call to yield. If you want to ensure that the bind has been done, you could request
Le 19/09/12 14:23, Oliver Kowalke a écrit : that yield is provided by the result of the bind operation, as e.g.
int f( caller_t & c, strg & s, int x) { auto b = c.bind( s, x); // s and x are now used to store the next calls. b.yield( 7); // alternative b( 7) // here s and x have been reassigned } OK - what if a user forgets to bind ? Maybe we can force the user to bind:
int f( caller_t & c, strg & s, int x) { caller_t::yield_t b = c.bind( s, x); // s and x are now used to store the next calls. b.yield( 7); // alternative b( 7) // here s and x have been reassigned } c.bind() retrieves the addresses of the parameters used to store the values (for each entering of f). only yield_t will provide yield() function. what do you think? Oliver
Le 19/09/12 21:05, Oliver Kowalke a écrit :
Am 19.09.2012 20:12, schrieb Vicente J. Botet Escriba:
I forgot a suggestion made by Vicente:
5.a coroutine<>::caller_t provides function bind() which must be called after coroutine::caller_t::yield() returned in order to get the parameters
int f( caller_t & c, strg & s, int x) { c.yield( 7); // alternative c( 7) c.bind( s, x); // s and x will contain new values given by caller } My suggestion was to call bind just once at the beginning of function and before any call to yield. If you want to ensure that the bind has been done, you could request
Le 19/09/12 14:23, Oliver Kowalke a écrit : that yield is provided by the result of the bind operation, as e.g.
int f( caller_t & c, strg & s, int x) { auto b = c.bind( s, x); // s and x are now used to store the next calls. b.yield( 7); // alternative b( 7) // here s and x have been reassigned } OK - what if a user forgets to bind ? Maybe we can force the user to bind:
int f( caller_t & c, strg & s, int x) { caller_t::yield_t b = c.bind( s, x); // s and x are now used to store the next calls. b.yield( 7); // alternative b( 7) // here s and x have been reassigned }
c.bind() retrieves the addresses of the parameters used to store the values (for each entering of f). only yield_t will provide yield() function.
what do you think?
This is exactly what I proposed above :) and for the time been this is the best interface I have found. Note that as I commented in some of my first post related to bind, you must take care of const and reference parameters in some tricky ways, as in order to be able to reassign them you should do some casts. I hope this will not cache some undefined behavior. Note also that the access to the parameters need to be done using the get function on functions called by the coroutine function such as the below function 'g'. int f( caller_t & c, strg & s, int x) { caller_t::yield_t b = c.bind( s, x); // s and x are now used to store the next calls. b.yield( s.length() + x ); // here s and x have been reassigned g(b); } void g( caller_t::yield_t & b) { b.yield( b.get<0>.length() + 2*b.get<1> ); // here s and x have been reassigned } As you can see, the function called *could* not have the s and x parameters and could not return int, that is, needs just the yield context. Best, Vicente
Note also that the access to the parameters need to be done using the get function on functions called by the coroutine function such as the below function 'g'.
int f( caller_t & c, strg & s, int x) { caller_t::yield_t b = c.bind( s, x); // s and x are now used to store the next calls.
b.yield( s.length() + x ); // here s and x have been reassigned g(b);
}
void g( caller_t::yield_t & b) { b.yield( b.get<0>.length() + 2*b.get<1> ); // here s and x have been reassigned }
As you can see, the function called *could* not have the s and x parameters and could not return int, that is, needs just the yield context.
OK - it should not be to hard to implement this. I agree that generator<> could get an iterator interface (output-iterator fro caller generator, input-iterator in generator<>-fn) But I'm not convinced that the coroutine<> template should have an iterator interface. How does the syntax look like for coroutine-fn with multiple parameters (for instance f() in the example above)? Oliver
Le 20/09/12 08:19, Oliver Kowalke a écrit :
Note also that the access to the parameters need to be done using the get function on functions called by the coroutine function such as the below function 'g'.
int f( caller_t & c, strg & s, int x) { caller_t::yield_t b = c.bind( s, x); // s and x are now used to store the next calls.
b.yield( s.length() + x ); // here s and x have been reassigned g(b);
}
void g( caller_t::yield_t & b) { b.yield( b.get<0>.length() + 2*b.get<1> ); // here s and x have been reassigned }
As you can see, the function called *could* not have the s and x parameters and could not return int, that is, needs just the yield context. OK - it should not be to hard to implement this. See more on the replay to Giovanni. I agree that generator<> could get an iterator interface (output-iterator fro caller generator, input-iterator in generator<>-fn) But I'm not convinced that the coroutine<> template should have an iterator interface. How does the syntax look like for coroutine-fn with multiple parameters (for instance f() in the example above)?
I don't think a generic coroutine should have an iterator interface. But coroutine <T<()> and coroutine <void(T)> could as Giovanni posted already. Best, Vicente
On Thu, Sep 20, 2012 at 7:19 AM, Oliver Kowalke <oliver.kowalke@gmx.de>wrote:
I agree that generator<> could get an iterator interface (output-iterator fro caller generator, input-iterator in generator<>-fn) But I'm not convinced that the coroutine<> template should have an iterator interface. How does the syntax look like for coroutine-fn with multiple parameters (for instance f() in the example above)?
you either enable the iterator interface for only coroutine<T()> and coroutine<void(T)>, or, for muliple parameters, you make the value type a tuple: static_assert(is_same<range_value<coroutine<void(int, double, float)> >, std::tuple<int, double, float>); The only coroutines left out from the range interface are general bidirecitonal coroutines of the form coroutine<T...>(T2...)>. -- gpd
int f( caller_t & c, strg & s, int x) { caller_t::yield_t b = c.bind( s, x); // s and x are now used to store the next calls.
b.yield( s.length() + x ); // here s and x have been reassigned g(b);
}
void g( caller_t::yield_t & b) { b.yield( b.get<0>.length() + 2*b.get<1> ); // here s and x have been reassigned }
do we really need the selft_t::get<>()? The signature of function g() says that it is not interested in variables s and x. Oliver
On Thu, Sep 20, 2012 at 3:51 AM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 19/09/12 21:05, Oliver Kowalke a écrit :
int f( caller_t & c, strg & s, int x) {
caller_t::yield_t b = c.bind( s, x); // s and x are now used to store the next calls. b.yield( 7); // alternative b( 7) // here s and x have been reassigned }
c.bind() retrieves the addresses of the parameters used to store the values (for each entering of f). only yield_t will provide yield() function.
what do you think?
This is exactly what I proposed above :) and for the time been this is
the best interface I have found.
I have issues with this interface, though: - Why do you need to explicitly bind? If you are required to bind the arguments of the coroutine-fn anyway, the trampoline might as well do it for you. - As you described, you need to provide an alternate interface (i.e. get()) to access the arguments from a nested function anyway. You might as well make this the only interface. - Tying the caller_t to an object in the coroutine stack (the parameters), makes it very hard to move the caller object to another coroutine. This is important to implement pipelines, where yield does not return to the caller but the next coroutine in the pipeline. - Also see below:
Note that as I commented in some of my first post related to bind, you must take care of const and reference parameters in some tricky ways, as in order to be able to reassign them you should do some casts. I hope this will not cache some undefined behavior.
How would you exactly rebind references? I can't see any sane way to do it. I would expect the following code to work: int i = 0, j = 0; coroutine<void(int&)> coro([&](caller_t& caller, int& x) { caller.bind(x); assert(&x == &i); // ok caller.yield(); assert(&x == &y); // ????? }); coro(i); coro(j); tl;dr: have get() as the only way to access parameters and have the parameters themselves local variables (i.e. stack allocated) of the calling context, not of the callee.
- Why do you need to explicitly bind? If you are required to bind the arguments of the coroutine-fn anyway, the trampoline might as well do it for you.
bind() in the sense to get the address. with the address I can store new values in the variable.
- As you described, you need to provide an alternate interface (i.e. get())
get() should not be provided
- Tying the caller_t to an object in the coroutine stack (the parameters), makes it very hard to move the caller object to another coroutine. This is important to implement pipelines, where yield does not return to the caller but the next coroutine in the pipeline.
let us keep the library simple - chains of coroutiens will causes many problems
How would you exactly rebind references?
references are pointers == addresses - in the case I get the address of the pointer and store tat this place the address of the new value
int i = 0, j = 0; coroutine<void(int&)> coro([&](caller_t& caller, int& x) { caller.bind(x); assert(&x == &i); // ok caller.yield(); assert(&x == &y); // ????? });
coro(i); coro(j);
caller.bind(x); // get address of the reference of x which contains the address of i caller.yield(); // in this call I store the address of j at the address of reference of x. Oliver
On Thu, Sep 20, 2012 at 12:05 PM, Oliver Kowalke <oliver.kowalke@gmx.de>wrote:
- Why do you need to explicitly bind? If you are required to bind the arguments of the coroutine-fn anyway, the trampoline might as well do it for you.
bind() in the sense to get the address. with the address I can store new values in the variable.
I understand that. What I'm saying is that bind can't rebind a reference parameter.
- As you described, you need to provide an alternate interface (i.e. get())
get() should not be provided
how would a nested fetch the passed parameters then? Being able to yield from nested calls is the whole point of stackful coroutines.
- Tying the caller_t to an object in the coroutine stack (the parameters), makes it very hard to move the caller object to another coroutine. This is important to implement pipelines, where yield does not return to the caller but the next coroutine in the pipeline.
let us keep the library simple - chains of coroutiens will causes many problems
pipelines seems a killer feature of coroutine library, preventing a future addition seems a bad choice.
How would you exactly rebind references?
references are pointers == addresses - in the case I get the address of the pointer and store tat this place the address of the new value
references are not pointers the c++ . They are aliases; there might or might not be an address somewhere in memory representing the reference, depending on compiler version, optimization level and calling convention. In the sniped below, for example, on a modern calling convention, x will be passed via a register. You can't maningfully get the address of a register. The register might be spilled on the stack when needed, but the stack slot location need not be stable and the compiler may assume that the content won't be changed by the user code.
int i = 0, j = 0; coroutine<void(int&)> coro([&](caller_t& caller, int& x) { caller.bind(x); assert(&x == &i); // ok caller.yield(); assert(&x == &y); // ????? });
coro(i); coro(j);
caller.bind(x); // get address of the reference of x which contains the address of i
references are not objects, you cannot get a pointer to a reference.
caller.yield(); // in this call I store the address of j at the address of reference of x.
there is no such a thing :( -- gpd
references are not pointers the c++ . They are aliases; there might or might not be an address somewhere in memory representing the reference, depending on compiler version, optimization level and calling convention. In the sniped below, for example, on a modern calling convention, x will be passed via a register. You can't maningfully get the address of a register. The register might be spilled on the stack when needed, but the stack slot location need not be stable and the compiler may assume that the content won't be changed by the user code.
my information was that references are pointers but because they are not changeable after initialization the compiler can do arbitrary things with it (especially the optimizer). OK - the code could cause undefined behaviour for references. Should bind() deny to bind to references and the coroutine-fn parameters must be pointers in this case? Should yield() return a tuple<> instead (as in the current implementation) and bind() is not provided?
On Thu, Sep 20, 2012 at 12:45 PM, Oliver Kowalke <oliver.kowalke@gmx.de>wrote:
references are not pointers the c++ . They are aliases; there might or might not be an address somewhere in memory representing the reference, depending on compiler version, optimization level and calling convention. In the sniped below, for example, on a modern calling convention, x will be passed via a register. You can't maningfully get the address of a register. The register might be spilled on the stack when needed, but the stack slot location need not be stable and the compiler may assume that the content won't be changed by the user code.
my information was that references are pointers but because they are not changeable after initialization the compiler can do arbitrary things with it (especially the optimizer). OK - the code could cause undefined behaviour for references.
Should bind() deny to bind to references and the coroutine-fn parameters must be pointers in this case?
no please!
Should yield() return a tuple<> instead (as in the current implementation) and bind() is not provided?
Again, I would prefer that a coroutine would allow later retrieval of the result value, but I think such a yield (or caller_t::operator()) would be fine. You should still consier having yield should return a reference to a tuple. The tuple itself should be stored on the calling coroutine operator(). And viceversa of course. Depending on the implementation, it might (or not) be more efficient. -- gpd
Le 20/09/12 13:05, Oliver Kowalke a écrit :
- Why do you need to explicitly bind? If you are required to bind the arguments of the coroutine-fn anyway, the trampoline might as well do it for you. bind() in the sense to get the address. with the address I can store new values in the variable.
- As you described, you need to provide an alternate interface (i.e. get()) get() should not be provided
- Tying the caller_t to an object in the coroutine stack (the parameters), makes it very hard to move the caller object to another coroutine. This is important to implement pipelines, where yield does not return to the caller but the next coroutine in the pipeline. let us keep the library simple - chains of coroutiens will causes many problems
How would you exactly rebind references? references are pointers == addresses - in the case I get the address of the pointer and store tat this place the address of the new value
int i = 0, j = 0; coroutine<void(int&)> coro([&](caller_t& caller, int& x) { caller.bind(x); assert(&x == &i); // ok caller.yield(); assert(&x == &y); // ????? });
coro(i); coro(j); caller.bind(x); // get address of the reference of x which contains the address of i caller.yield(); // in this call I store the address of j at the address of reference of x.
Unfortunately, this shouldn't work. You can change the contents of a reference but not the reference itself. But maybe you have found how to workaround the compiler semantics. Best, Vicente
Le 20/09/12 12:18, Giovanni Piero Deretta a écrit :
On Thu, Sep 20, 2012 at 3:51 AM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 19/09/12 21:05, Oliver Kowalke a écrit :
int f( caller_t & c, strg & s, int x) {
caller_t::yield_t b = c.bind( s, x); // s and x are now used to store the next calls. b.yield( 7); // alternative b( 7) // here s and x have been reassigned }
c.bind() retrieves the addresses of the parameters used to store the values (for each entering of f). only yield_t will provide yield() function.
what do you think?
This is exactly what I proposed above :) and for the time been this is
the best interface I have found.
I have issues with this interface, though: - Why do you need to explicitly bind? If you are required to bind the arguments of the coroutine-fn anyway, the trampoline might as well do it for you. I don't need to if the library hides the bind. - As you described, you need to provide an alternate interface (i.e. get()) to access the arguments from a nested function anyway. You might as well make this the only interface. Yes. - Tying the caller_t to an object in the coroutine stack (the parameters), makes it very hard to move the caller object to another coroutine. This is important to implement pipelines, where yield does not return to the caller but the next coroutine in the pipeline. You are surely right. I have not considered yet symmetric coroutine. Let me think a little bit more about how this interact. - Also see below:
Note that as I commented in some of my first post related to bind, you must take care of const and reference parameters in some tricky ways, as in order to be able to reassign them you should do some casts. I hope this will not cache some undefined behavior.
How would you exactly rebind references? I can't see any sane way to do it. I would expect the following code to work:
int i = 0, j = 0; coroutine<void(int&)> coro([&](caller_t& caller, int& x) { caller.bind(x); assert(&x == &i); // ok caller.yield(); assert(&x == &y); // ????? });
coro(i); coro(j);
Yes, references could not be rebound. We could rebind types, pointers but not references. I guess the coroutine library could store a pointer on the caller object, and allow to obtain them using get<>. But in this case the coroutine function could not follow the coroutine signature. int i = 0, j = 0; coroutine<void(int&)> coro([&](caller_t& caller) { int& x = caller.get<0>; assert(&caller.get<0> == &i); // ok caller.yield(); assert(&caller.get<0> == &j); //ok }); coro(i); coro(j); Best, Vicente
tl;dr: have get() as the only way to access parameters and have the parameters themselves local variables (i.e. stack allocated) of the calling context, not of the callee.
I think you are right. Best, Vicente
On Thu, Sep 20, 2012 at 12:17 PM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 20/09/12 12:18, Giovanni Piero Deretta a écrit :
On Thu, Sep 20, 2012 at 3:51 AM, Vicente J. Botet Escriba <
vicente.botet@wanadoo.fr> wrote:
Note that as I commented in some of my first post related to bind, you must take care of const and reference parameters in some tricky ways, as in order to be able to reassign them you should do some casts. I hope this will not cache some undefined behavior.
How would you exactly rebind references? I can't see any sane way to do it. I would expect the following code to work:
int i = 0, j = 0; coroutine<void(int&)> coro([&](caller_t& caller, int& x) { caller.bind(x); assert(&x == &i); // ok caller.yield(); assert(&x == &y); // ????? });
coro(i); coro(j);
Yes, references could not be rebound. We could rebind types, pointers but not references. I guess the coroutine library could store a pointer on the caller object, and allow to obtain them using get<>. But in this case the coroutine function could not follow the coroutine signature.
you mean that caller_t::yield does not really model "int&()"? Well, this is an issue. As far I understand, currently Oliver is leaning on passing the parameters to the coroutine-fn, having yield() return a tuple, and having the coroutine-fn return the final result. This means that yield() will closely follow the signature and everything is fine. If instead he decides to go with the get<> interface, then maybe it is time to abandon the coroutine as a function model. -- gpd
Le 20/09/12 15:41, Giovanni Piero Deretta a écrit :
On Thu, Sep 20, 2012 at 12:17 PM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 20/09/12 12:18, Giovanni Piero Deretta a écrit :
On Thu, Sep 20, 2012 at 3:51 AM, Vicente J. Botet Escriba <
vicente.botet@wanadoo.fr> wrote:
Note that as I commented in some of my first post related to bind, you must take care of const and reference parameters in some tricky ways, as in order to be able to reassign them you should do some casts. I hope this will not cache some undefined behavior.
How would you exactly rebind references? I can't see any sane way to do it.
Yes, references could not be rebound. We could rebind types, pointers but not references. I guess the coroutine library could store a pointer on the caller object, and allow to obtain them using get<>. But in this case the coroutine function could not follow the coroutine signature.
you mean that caller_t::yield does not really model "int&()"? Well, this is an issue. As far I understand, currently Oliver is leaning on passing the parameters to the coroutine-fn, having yield() return a tuple, and having the coroutine-fn return the final result. This means that yield() will closely follow the signature and everything is fine. Well, this is the initial design that was the source of all the alternative proposals. I have voted YES for the inclusion of the library without conditions. So if at the end this the original interface is retained it will be Ok for me.
After a more deep analysis of the alternatives I have proposed it seems that none make easier to write used code: * bind can not be used with references, limiting the coroutine interface to don't use references is not a solution. * access to the coroutine parameter using get, while it ensures a uniform and safe way to obtain them, is not easier to read, and in this case the coroutine function shouldn't follow the coroutine signature, which for some of you make the interface type unsafe.
If instead he decides to go with the get<> interface, then maybe it is time to abandon the coroutine as a function model.
Could you elaborate more on the alternative model you are suggesting? Best, Vicente
* bind can not be used with references, limiting the coroutine interface to don't use references is not a solution.
yes, I hoped to use bind to store new values in the parameters of coroutine-fn so that the user could still used the fn parameters as in a usual function,. unfortunately it does not work with references.
* access to the coroutine parameter using get, while it ensures a uniform and safe way to obtain them, is not easier to read, and in this case the coroutine function shouldn't follow the coroutine signature, which for some of you make the interface type unsafe.
seams to be a decision between 'tuple<> self_t::yield()' and void 'self_t::yield()' + 'T self_t::get<>()'. The design with get<>() is not hard to implement because I store already the tuple of fn parameters inside. Oliver
On Fri, Sep 21, 2012 at 7:02 AM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 20/09/12 15:41, Giovanni Piero Deretta a écrit :
On Thu, Sep 20, 2012 at 12:17 PM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 20/09/12 12:18, Giovanni Piero Deretta a écrit :
On Thu, Sep 20, 2012 at 3:51 AM, Vicente J. Botet Escriba <
vicente.botet@wanadoo.fr> wrote:
Note that as I commented in some of my first post related to bind, you
must take care of const and reference parameters in some tricky ways, as in order to be able to reassign them you should do some casts. I hope this will not cache some undefined behavior.
How would you exactly rebind references? I can't see any sane way to do
it.
Yes, references could not be rebound. We could rebind types, pointers but not references. I guess the coroutine library could store a pointer on the caller object, and allow to obtain them using get<>. But in this case the coroutine function could not follow the coroutine signature.
you mean that caller_t::yield does not really model "int&()"? Well, this is an issue. As far I understand, currently Oliver is leaning on passing the parameters to the coroutine-fn, having yield() return a tuple, and having the coroutine-fn return the final result. This means that yield() will closely follow the signature and everything is fine.
Well, this is the initial design that was the source of all the alternative proposals. I have voted YES for the inclusion of the library without conditions. So if at the end this the original interface is retained it will be Ok for me.
Pretty much everybody (including me) unconditionally voted yes for the original interface. I guess we are having this discussion for two reasons: 1) getting the first inputs as arguments (and returning the last result as return) breaks a bit the symmetry of yield. Works very well for some scenarios but makes some other a bit awkward 2) more importantly, initial parameters might be left dangling if they are references to objects that go out of scope in subsequent calls. Probably 1) is just a matter of getting used to it, while 2) will need a big warning in the documentation. Another solution is to disallow implicit references (like std::thread) and require an explicit reference_wrapper, but I'm not sure it is the best solution. BTW, if Oliver goes with the original interface, then I agree that coroutines and generators should be separated, but the missing additional *output* generator should be added to the library. It will also be fine if only generators will have range semantics. I expect that for most use cases, people will only use generators.
After a more deep analysis of the alternatives I have proposed it seems that none make easier to write used code: * bind can not be used with references, limiting the coroutine interface to don't use references is not a solution. * access to the coroutine parameter using get, while it ensures a uniform and safe way to obtain them, is not easier to read, and in this case the coroutine function shouldn't follow the coroutine signature, which for some of you make the interface type unsafe.
Wait, of which type unsafety are you talking about here?
If instead he decides to go with the get<> interface, then maybe it is time to abandon the coroutine as a function model.
Could you elaborate more on the alternative model you are suggesting?
not much really, just that a coroutine with get() shouldn't be instantiated as coroutine<In(Out)> as that would lead the user to believe that it is similar to a function. Something like coroutine<In, Out> might be more appropriate. -- gpd
participants (3)
-
Giovanni Piero Deretta -
Oliver Kowalke -
Vicente J. Botet Escriba