[thread, async] Mini design review requested for a generic extensible Boost monadic continuations framework
Dear Boost, I've been working on a Boost generalisation framework for N3857 "Improvements to std::future<T> and Related APIs" (Jan 2014). It's also an extension of Vicente's N3865 "More Improvements to std::future<T>" (Jan 2014), though slightly incompatible with it. I would appreciate the Boost community's feedback on what is essentially pure exploratory prototype code before I commit any major design flaws in a proper implementation. The proposed design is specifically intended to allow proposed Boost.AFIO's async_io_op future-like object to seamlessly interoperate with any other future-like object e.g. std::future<T> or boost::future<T>. The design is intended to be extensible to any other third party future-like object, even including ones whose implementation is completely external (e.g. Qt). I would like to thank some people whose off-list discussions helped inform this design: to Bjorn Reese who has done so much to help me knock the kinks out of AFIO; to Vicente J. Botet Escriba for engaging with me pitching this proposal to him (he disagrees with it by the way); to Artur Laksberg and Niklas Gustafsson for continuing to reply to my criticisms and thoughts of the papers which precede N3857. What N3857 proposes for reference: * future<T> and shared_future<T> gain a .then(R(prec_future_type<T>)) method. Callable will be called with the signalling future after the future signals. Return type is a future<R>. * A new when_all(...) can be used to compose a future which signals when all input futures are signalled. Inputs can be futures of heterogeneous types, in which case a future<tuple<...>> is returned. * A new when_any(...) composes a future which signals when any of its input futures signals. Inputs can be futures of heterogeneous types, in which case a future<tuple<...>> is returned. To avoid the linear scan on return, for a homogeneous types a when_any_swapped(...) swaps the last item in the return value with the ready item. Note there is no way in N3857 to avoid a when_any() linear scan when input types are heterogenous (something I feel is a deficiency, all you need to return is an index!). * A new executor base class, with subclass thread_pool are added. Executors allow adding a std::function<void()> for asynchronous execution. These are detailed by N3785 "Executors and schedulers, revision 3" (Oct 2013). I haven't mentioned in detail here, but N3857 specifies overloads for .then(executor, ...) to send some continuation to some executor. You will note the below framework makes it possible to use tag dispatch instead which looks much cleaner and is easier to write, nevertheless this explicit form is intended to be preserved. I won't mention executor support again for brevity. Where I find issue with a naïve implementation of N3857 (and Vicente's N3865 mentions some of these same issues): * N3857 only understands std::future<> and std::shared_future<>. That means the following code will not compile: std::future<int> &a; boost::future<int> &b; boost::afio::async_io_op &c; std::future<std::tuple<int, int, std::shared_ptr<async_io_handle>>> f=when_all(a, b, c); This is unhelpful for interoperation. It doesn't just affect Boost: imagine mashing up Qt async objects and C++17 objects for example, or even WinRT Task<T> objects and C++17 objects. Most C++ (and C) libraries of any complexity supply their own async notification objects, and this problem of writing never ending boilerplate to get some async object from library X to work with library Y is very tedious considering the compiler can do it for you. * As I mentioned earlier, when_any() is suboptimal when called with heterogenous types. I don't particularly like the when_any_swapped() hack either - a better solution would solve when_any() of all kinds optimally. * The .then(R(prec_future_type<T>)) is not tremendously useful in practice because it's too limited. Witness the very common use pattern of this form: auto ret=openedfile .then(write({ "He", "ll", "o ", "Wo", "rl", "d\n" }, 0)) .then(sync()); .then(write({ "He", "ll", "o ", "Wo", "rl", "d\n" }, 12)) .then(sync()); This sequence of code is clearly a *transaction* i.e. the programmer intends the set of continuations as a contained group of operations, and indeed some async i/o providers may guarantee atomic rollback (hint: forthcoming TripleGit does exactly this). A big additional problem is the lack of error handling logic: if say the second write() fails, you may wish to undo the first write, or do something custom. The problem with N3857 is that only the state of the first sync() is passed into the second write, and it may be the case that in generic code sync() needs to always sync irrespective of its preceding operation and therefore might pass through a garbage state. N3857 provides no easy method for a later .then() to inspect preceding .then()'s and do something about them (e.g. rewinding until it finds a valid open file handle rather than error states), other than by declaring special wrapper lambdas to pass through state. I personally think that is ugly. * Vicente mirrors my point above in N3865 by pointing out that having .then(R(prec_future_type<T>)) push the error handling into the callable makes your callables unnecessarily boilerplate. N3865 proposes adding .next(R(prec_future_type<T>)) for when a preceding operation succeeds and .recover(R(exception_ptr)) for when a preceding operation fails, with .fallback_to(value) as a way of default setting the return from an errored future. I think Vicente's argument here is a good one: but it's not generic enough in my opinion. I think continuations should be able to _filter_ outputs from preceding continuations in a really generic and customisable way. What I'd really like is this: continuations::thenable_get_placeholder<-1> last; auto ret=openedfile .then(write({ "He", "ll", "o ", "Wo", "rl", "d\n" }, 0)) .then(if_error(last, [](future<...> f){ do something with last; })) .then(sync()); .then(write({ "He", "ll", "o ", "Wo", "rl", "d\n" }, 12)) .then(if_error(last, [](thenable<...> t){ do something more with everything; })) .then(sync()); Which treats the continuation as a functional transformation. In other words, Vicente's .recover(R(exception_ptr)) becomes tag dispatched instead of explicitly specified, so it's very easy for the programmer to transform state according to state. What I am proposing instead: a generic extensible Boost monadic continuations framework which via metaprogramming lets the compiler assemble the right code from N3857 like syntax. A quick summary of the proposed generic extensible Boost monadic continuations framework follows: * Future-like behaviours such as .valid(), .then(), .get(), .wait() and .share() have functional mixin classes of the form (simplified for brevity, but you get the picture): namespace boost { namespace continuations { // Indicates the derived type provides a future compatible valid() template<class I> struct validable : public I { bool valid() const; }; // Indicates the derived type provides a future compatible then() template<class I, class... Types> struct thenable : public I { template<class F> thenable<decltype(F(...))>, Types..., I> then(F &&f); }; // Indicates the derived type provides a future compatible get() template<class T, class I> struct gettable: public I { T get(); }; // Indicates the derived type provides a future compatible wait() template<class I> struct waitable: public I { void wait() const; template<class Rep, class Period> std::future_status wait_for(const std::chrono::duration<Rep,Period> &timeout_duration) const; template<class Clock, class Duration> std::future_status wait_until(const std::chrono::time_point<Clock,Duration> &timeout_time) const; }; } } To mark up a type as being future-like, one might use: namespace boost { namespace continuations { template<class T> using monad = thenable<waitable<gettable<T, validable<std::future<T>>>>>; // Do all the partial specialisations of the internal machinery // for this type to hook it into the metaprogramming } } // Voila, boost::continuations::monad<T> is now an interoperable // form of std::future<T> * .then() can now take callables of the following forms: 1. R() and R(prec_future_type<T>) for compatibility with N3857. 2. R(..., continuations::thenable_get_placeholder<idx>, ...) inserts an item from the continuation chain into the call. thenable_get_placeholder can be fed a negative number to wrap from the bottom, so thenable_get_placeholder<-1> is the most recently preceding operation. Therefore R(prec_future_type<T>) from above is actually implemented as R(continuations::thenable_get_placeholder<-1>). This can be used to insert error processing filters as illustrated earlier which can view a preceding chunk of chain at once, and act. 3. R(thenable<...> &&) which gives a continuation access to the entire continuation chain, just for those people who really need it. You saw this in the example above. 4. Any specially treated prototype depending on what the metaprogramming has extended for some future_type. For example, if one does async_io_op.then(void(const boost::system::error_code& error, size_t bytes_transferred)) - which is the ASIO callback spec - then AFIO would emulate the ASIO callback calling convention for the completion of some i/o. This only happens if and only if the item being .then() is an async_io_op i.e. such custom extensions are type specific. * future_type<T>.then(R()) now returns a thenable<future_type<R>, future_type<T>> which auto decays to a future_type<R> on demand for compatibility with N3857. Similarly a future_type<T>.then(R()).then(S()) would return a thenable<future_type<S>, future_type<T>, future_type<R>> and so on. You can convert a thenable<T, ...> into a std::tuple<..., T> using an overload of the make_tuple() function. * when_all() and when_any() gain overloads for thenable<...>. These simply convert the tuple taken out of a thenable chain into a future with the results - AFIO will implement this generically using its closure engine, but a more intelligent solution might decompose all inputs into HANDLEs and do an asynchronous WaitForMultipleObjects() for example. Note this is a BREAKING CHANGE from N3857: If you do a when_all() under N3857 on a .then() chain, you will get a future to the last then() return, whereas under this proposal you will get a future to all the returns from all the then()s. This breaking change may still actually compile and work as expected - however when_any() actually now has completely different behaviour: when_any() under N3857 will be the same effect as when_all() on a .then() chain under N3857, whereas under this proposal when_any() will signal when any of the then() chain signals (I would assume the first item). I personally doubt this breaking change would affect any real world code, but it's worth noting. Note the following: 1. The proposed continuations framework, despite being monadic, is separate from the mooted Boost.Functional/Monads framework. This continuations framework could be implemented using such a Boost.Functional/Monads framework, but I suspect there wouldn't be much gain except interoperability with other monadic patterns. Continuation monads are a very limited specialisation of general monads, especially under the tight constraints of the N3857 requirements, so reuse of any Boost.Functional/Monads I suspect would be minimal. 2. The proposed framework is 100% compatible with N3857 apart from the when_all()/when_any() breakages which I would assume shouldn't turn up in real world code, and it should be viewed as a Boost-specific extension which may get standardised later if it proves popular. Here is the very rough prototype code: https://github.com/BoostGSoC/boost.afio/blob/monadic_continuations/lib s/afio/test/tests/monadic_continuations_test.cpp. It compiles using VS2013 only. It probably doesn't work, but it's simply there to demonstrate the viability of the concept and to help those confused by my unclear explanation above. Thoughts regarding the design much appreciated! My thanks in advance. Niall -- Currently unemployed and looking for work in Ireland. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/
Le 03/03/14 15:16, Niall Douglas a écrit :
Dear Boost,
I've been working on a Boost generalisation framework for N3857 "Improvements to std::future<T> and Related APIs" (Jan 2014). It's also an extension of Vicente's N3865 "More Improvements to std::future<T>" (Jan 2014), though slightly incompatible with it. I would appreciate the Boost community's feedback on what is essentially pure exploratory prototype code before I commit any major design flaws in a proper implementation.
The proposed design is specifically intended to allow proposed Boost.AFIO's async_io_op future-like object to seamlessly interoperate with any other future-like object e.g. std::future<T> or boost::future<T>. The design is intended to be extensible to any other third party future-like object, even including ones whose implementation is completely external (e.g. Qt).
I would like to thank some people whose off-list discussions helped inform this design: to Bjorn Reese who has done so much to help me knock the kinks out of AFIO; to Vicente J. Botet Escriba for engaging with me pitching this proposal to him (he disagrees with it by the way); to Artur Laksberg and Niklas Gustafsson for continuing to reply to my criticisms and thoughts of the papers which precede N3857. Niall, thanks for this more explanatory description of what you have in mind. I think that our positions are closer than I believed after our private exchanges.
What N3857 proposes for reference:
* future<T> and shared_future<T> gain a .then(R(prec_future_type<T>)) method. Callable will be called with the signalling future after the future signals. Return type is a future<R>.
* A new when_all(...) can be used to compose a future which signals when all input futures are signalled. Inputs can be futures of heterogeneous types, in which case a future<tuple<...>> is returned.
* A new when_any(...) composes a future which signals when any of its input futures signals. Inputs can be futures of heterogeneous types, in which case a future<tuple<...>> is returned. To avoid the linear scan on return, for a homogeneous types a when_any_swapped(...) swaps the last item in the return value with the ready item. Note there is no way in N3857 to avoid a when_any() linear scan when input types are heterogenous (something I feel is a deficiency, all you need to return is an index!). I guess that this point could be discuss in std-proposals ML. <snip>
Where I find issue with a naïve implementation of N3857 (and Vicente's N3865 mentions some of these same issues):
* N3857 only understands std::future<> and std::shared_future<>. That means the following code will not compile:
std::future<int> &a; boost::future<int> &b; boost::afio::async_io_op &c; std::future<std::tuple<int, int, std::shared_ptr<async_io_handle>>> f=when_all(a, b, c);
This is unhelpful for interoperation. It doesn't just affect Boost: imagine mashing up Qt async objects and C++17 objects for example, or even WinRT Task<T> objects and C++17 objects. Most C++ (and C) libraries of any complexity supply their own async notification objects, and this problem of writing never ending boilerplate to get some async object from library X to work with library Y is very tedious considering the compiler can do it for you. I have some doubts on who we can decide that the result of such an operation would be
std::future<std::tuple<int, int, std::shared_ptr<async_io_handle>>>
* As I mentioned earlier, when_any() is suboptimal when called with heterogenous types. I don't particularly like the when_any_swapped() hack either - a better solution would solve when_any() of all kinds optimally.
Make a proposal on std-proposals then.
* The .then(R(prec_future_type<T>)) is not tremendously useful in practice because it's too limited. Witness the very common use pattern of this form:
auto ret=openedfile .then(write({ "He", "ll", "o ", "Wo", "rl", "d\n" }, 0)) .then(sync()); .then(write({ "He", "ll", "o ", "Wo", "rl", "d\n" }, 12)) .then(sync());
This sequence of code is clearly a *transaction* i.e. the programmer intends the set of continuations as a contained group of operations, and indeed some async i/o providers may guarantee atomic rollback (hint: forthcoming TripleGit does exactly this). A big additional problem is the lack of error handling logic: if say the second write() fails, you may wish to undo the first write, or do something custom. The problem with N3857 is that only the state of the first sync() is passed into the second write, and it may be the case that in generic code sync() needs to always sync irrespective of its preceding operation and therefore might pass through a garbage state. N3857 provides no easy method for a later .then() to inspect preceding .then()'s and do something about them (e.g. rewinding until it finds a valid open file handle rather than error states), other than by declaring special wrapper lambdas to pass through state. I personally think that is ugly.
Here is when I disagree the most on your proposal. I think that transactions are orthogonal to the future class and the user could return whatever she wants from its continuations. Maybe you need to implement some kind of Monad State.
* Vicente mirrors my point above in N3865 by pointing out that having .then(R(prec_future_type<T>)) push the error handling into the callable makes your callables unnecessarily boilerplate. N3865 proposes adding .next(R(prec_future_type<T>)) for when a preceding operation succeeds and .recover(R(exception_ptr)) for when a preceding operation fails, with .fallback_to(value) as a way of default setting the return from an errored future.
I think Vicente's argument here is a good one: but it's not generic enough in my opinion. I think continuations should be able to _filter_ outputs from preceding continuations in a really generic and customisable way. What I'd really like is this:
continuations::thenable_get_placeholder<-1> last; auto ret=openedfile .then(write({ "He", "ll", "o ", "Wo", "rl", "d\n" }, 0)) .then(if_error(last, [](future<...> f){ do something with last; })) .then(sync()); .then(write({ "He", "ll", "o ", "Wo", "rl", "d\n" }, 12)) .then(if_error(last, [](thenable<...> t){ do something more with everything; })) .then(sync());
Could you clarify what prevents you from doing this with the current proposals?
Which treats the continuation as a functional transformation. In other words, Vicente's .recover(R(exception_ptr)) becomes tag dispatched instead of explicitly specified, so it's very easy for the programmer to transform state according to state. I have been discussing something similar lastly on c++-parallel ML. I was suggesting something as if_valued/if_unexpected. I have implemented it first on Boost.Expected.
What I am proposing instead: a generic extensible Boost monadic continuations framework which via metaprogramming lets the compiler assemble the right code from N3857 like syntax. A quick summary of the proposed generic extensible Boost monadic continuations framework follows:
* Future-like behaviours such as .valid(), .then(), .get(), .wait() and .share() have functional mixin classes of the form (simplified for brevity, but you get the picture):
namespace boost { namespace continuations {
// Indicates the derived type provides a future compatible valid() template<class I> struct validable : public I { bool valid() const; };
// Indicates the derived type provides a future compatible then() template<class I, class... Types> struct thenable : public I { template<class F> thenable<decltype(F(...))>, Types..., I> then(F &&f); };
// Indicates the derived type provides a future compatible get() template<class T, class I> struct gettable: public I { T get(); };
// Indicates the derived type provides a future compatible wait() template<class I> struct waitable: public I { void wait() const; template<class Rep, class Period> std::future_status wait_for(const std::chrono::duration<Rep,Period> &timeout_duration) const; template<class Clock, class Duration> std::future_status wait_until(const std::chrono::time_point<Clock,Duration> &timeout_time) const; };
} }
To mark up a type as being future-like, one might use:
namespace boost { namespace continuations {
template<class T> using monad = thenable<waitable<gettable<T, validable<std::future<T>>>>>; // Do all the partial specialisations of the internal machinery // for this type to hook it into the metaprogramming
} } // Voila, boost::continuations::monad<T> is now an interoperable // form of std::future<T> I have also proposed something like that could be used as
monad_error(f).when_valued(g).when_unexpected(h); or as monad_error(f) & g | h; I have a prototype implementation on one of the examples in Boost.Expected.
* .then() can now take callables of the following forms:
1. R() and R(prec_future_type<T>) for compatibility with N3857.
IMO N3857 doesn't allows callables as R(). Neither Boost.Thread. But Boost.Expected allow it for expected<void> when using .next().
2. R(..., continuations::thenable_get_placeholder<idx>, ...) inserts an item from the continuation chain into the call. thenable_get_placeholder can be fed a negative number to wrap from the bottom, so thenable_get_placeholder<-1> is the most recently preceding operation. Therefore R(prec_future_type<T>) from above is actually implemented as R(continuations::thenable_get_placeholder<-1>). This can be used to insert error processing filters as illustrated earlier which can view a preceding chunk of chain at once, and act.
I don't understand this completely, but I suspect that what you are proposing is some variation of Monad state and an adaptor that extracts part of the state.
3. R(thenable<...> &&) which gives a continuation access to the entire continuation chain, just for those people who really need it. You saw this in the example above.
Again, this is more a kind of monad state that accumulates the results of each continuation.
4. Any specially treated prototype depending on what the metaprogramming has extended for some future_type. For example, if one does async_io_op.then(void(const boost::system::error_code& error, size_t bytes_transferred)) - which is the ASIO callback spec - then AFIO would emulate the ASIO callback calling convention for the completion of some i/o. This only happens if and only if the item being .then() is an async_io_op i.e. such custom extensions are type specific.
You lost me here.
* future_type<T>.then(R()) now returns a thenable<future_type<R>, future_type<T>> which auto decays to a future_type<R> on demand for compatibility with N3857. Similarly a future_type<T>.then(R()).then(S()) would return a thenable<future_type<S>, future_type<T>, future_type<R>> and so on. You can convert a thenable<T, ...> into a std::tuple<..., T> using an overload of the make_tuple() function.
Monad state again.
* when_all() and when_any() gain overloads for thenable<...>. These simply convert the tuple taken out of a thenable chain into a future with the results - AFIO will implement this generically using its closure engine, but a more intelligent solution might decompose all inputs into HANDLEs and do an asynchronous WaitForMultipleObjects() for example.
As I said above, why the result will be a future?
Note this is a BREAKING CHANGE from N3857: If you do a when_all() under N3857 on a .then() chain, you will get a future to the last then() return, whereas under this proposal you will get a future to all the returns from all the then()s. This breaking change may still actually compile and work as expected - however when_any() actually now has completely different behaviour: when_any() under N3857 will be the same effect as when_all() on a .then() chain under N3857, whereas under this proposal when_any() will signal when any of the then() chain signals (I would assume the first item). I personally doubt this breaking change would affect any real world code, but it's worth noting.
Note the following:
1. The proposed continuations framework, despite being monadic, is separate from the mooted Boost.Functional/Monads framework. This continuations framework could be implemented using such a Boost.Functional/Monads framework, but I suspect there wouldn't be much gain except interoperability with other monadic patterns. Continuation monads are a very limited specialisation of general monads, especially under the tight constraints of the N3857 requirements, so reuse of any Boost.Functional/Monads I suspect would be minimal.
I think that the Boost.Functional/Monads framework I had in mind could be used for your use case. All you need to do is to define a specialized Monad, that in your case is close to a Monad State.
2. The proposed framework is 100% compatible with N3857 apart from the when_all()/when_any() breakages which I would assume shouldn't turn up in real world code, and it should be viewed as a Boost-specific extension which may get standardised later if it proves popular.
I have no problem in proposing a different interface as IIUC your proposal, the user would need to wrap the std::future, boost::expected, ... or whatever AFIO monad or use specific monads::when_all/monads::when_any function. I hope that my comments would help you to clarify my discrepancies about your design. Resuming, I think that you must define your own monad state that could be seen as any other monad using the functional/monads framework.
Here is the very rough prototype code: https://github.com/BoostGSoC/boost.afio/blob/monadic_continuations/lib s/afio/test/tests/monadic_continuations_test.cpp. It compiles using VS2013 only. It probably doesn't work, but it's simply there to demonstrate the viability of the concept and to help those confused by my unclear explanation above.
Thoughts regarding the design much appreciated! My thanks in advance.
Niall
I will inspect your code in order to see if I've understood your needs as soon as I have enough time. Best, Vicente
On 5 Mar 2014 at 13:54, Vicente J. Botet Escriba wrote:
Niall, thanks for this more explanatory description of what you have in mind. I think that our positions are closer than I believed after our private exchanges.
I suspected I repeatedly failed to explain myself well to you. I apologise. Before I reply to you, I was just reading through Chris Kohlhoff's N3964 (Library Foundations for Asynchronous Operations, Revision 1) at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3964.pdf. I agree with his argument that a lower level callback API without futures would be wise for those who need such ultra low latency they can't afford malloc, but I am unsure if the C++ standard is the right place to specify such a machinery, even if it standardises Boost.ASIO. There is much of Boost.ASIO which *ought* to be standardised (e.g. I don't think Google's Executors proposal is worth much compared to standardising the core of ASIO as ASIO's core design is very solid, and very popular i.e. perfect for standardisation), but I must confess I don't think that this specific feature of ASIO i.e. the extensible model of asynchronous operations proposed in N3964 - is a good fit for standardisation.
* A new when_any(...) composes a future which signals when any of its input futures signals. Inputs can be futures of heterogeneous types, in which case a future<tuple<...>> is returned. To avoid the linear scan on return, for a homogeneous types a when_any_swapped(...) swaps the last item in the return value with the ready item. Note there is no way in N3857 to avoid a when_any() linear scan when input types are heterogenous (something I feel is a deficiency, all you need to return is an index!). I guess that this point could be discuss in std-proposals ML.
Erm, I think you just need to convince Artur Laksberg there is a better way and you're good here. I know he has personal doubts himself on this design choice. I'm thinking maybe a visitor callable might work, so some callable<T> gets instantiated and called with the ready future.
std::future<int> &a; boost::future<int> &b; boost::afio::async_io_op &c; std::future<std::tuple<int, int, std::shared_ptr<async_io_handle>>> f=when_all(a, b, c);
I have some doubts on who we can decide that the result of such an operation would be
std::future<std::tuple<int, int, std::shared_ptr<async_io_handle>>>
In my proposed generic framework there is some metaprogramming which decides what future-like type decays into. It defaults to the return type of future_type::get(), but it absolutely can be overriden with a partial template specialisation.
* As I mentioned earlier, when_any() is suboptimal when called with heterogenous types. I don't particularly like the when_any_swapped() hack either - a better solution would solve when_any() of all kinds optimally. Make a proposal on std-proposals then.
I think that the small number of actors involved in this tiny niche of C++17 is small enough that personal contact works better. I only posted a request for design review here because what I add to AFIO next is likely very useful to many others using Boost. To be honest, I had hoped for a more vociferous feedback given the number of people writing async object interop boilerplate. I guess they prefer writing boilerplate.
N3857 provides no easy method for a later .then() to inspect preceding .then()'s and do something about them (e.g. rewinding until it finds a valid open file handle rather than error states), other than by declaring special wrapper lambdas to pass through state. I personally think that is ugly. Here is when I disagree the most on your proposal. I think that transactions are orthogonal to the future class and the user could return whatever she wants from its continuations. Maybe you need to implement some kind of Monad State.
continuations::thenable<T, ...> *is* a monadic state. It accepts any set of heterogeneous types so long as they mix in the continuations::thenable<T, ...> type. In other words, I am using the policy class which says "this type is future-like then()able" as the policy implementation class *as well*. I suppose most would consider that bad design and insist on a separate set of concept classes. If that is your criticism, I accept that as probably valid.
continuations::thenable_get_placeholder<-1> last; auto ret=openedfile .then(write({ "He", "ll", "o ", "Wo", "rl", "d\n" }, 0)) .then(if_error(last, [](future<...> f){ do something with last; })) .then(sync()); .then(write({ "He", "ll", "o ", "Wo", "rl", "d\n" }, 12)) .then(if_error(last, [](thenable<...> t){ do something more with everything; })) .then(sync()); Could you clarify what prevents you from doing this with the current proposals?
The first if_error() tag could work with the current proposals. The second if_error() tag can see the entire thenable chain, and therefore could not work with the current proposals. My point actually was that tag dispatched filters of state I think is a better, more generic, more powerful design than hard coded member functions.
Which treats the continuation as a functional transformation. In other words, Vicente's .recover(R(exception_ptr)) becomes tag dispatched instead of explicitly specified, so it's very easy for the programmer to transform state according to state. I have been discussing something similar lastly on c++-parallel ML. I was suggesting something as if_valued/if_unexpected. I have implemented it first on Boost.Expected.
I know. I was trying to mirror your design. No point reinventing the wheel.
I have also proposed something like that could be used as
monad_error(f).when_valued(g).when_unexpected(h);
or as
monad_error(f) & g | h;
I have a prototype implementation on one of the examples in Boost.Expected.
I don't consider myself competent to be hugely useful in designing a generic C++ monadic framework - I don't believe myself good enough at language design.
2. R(..., continuations::thenable_get_placeholder<idx>, ...) inserts an item from the continuation chain into the call. thenable_get_placeholder can be fed a negative number to wrap from the bottom, so thenable_get_placeholder<-1> is the most recently preceding operation. Therefore R(prec_future_type<T>) from above is actually implemented as R(continuations::thenable_get_placeholder<-1>). This can be used to insert error processing filters as illustrated earlier which can view a preceding chunk of chain at once, and act. I don't understand this completely, but I suspect that what you are proposing is some variation of Monad state and an adaptor that extracts part of the state.
Correct.
3. R(thenable<...> &&) which gives a continuation access to the entire continuation chain, just for those people who really need it. You saw this in the example above. Again, this is more a kind of monad state that accumulates the results of each continuation.
Correct. thenable<T, ...> is a monad state as well as a policy class indicating a type is capable of being .then()ed.
* when_all() and when_any() gain overloads for thenable<...>. These simply convert the tuple taken out of a thenable chain into a future with the results - AFIO will implement this generically using its closure engine, but a more intelligent solution might decompose all inputs into HANDLEs and do an asynchronous WaitForMultipleObjects() for example. As I said above, why the result will be a future?
Oh, purely because N3857 says so, and I am not trying to change N3857, simply extend it such that it is much more useful than now. Do remember a future is thenable<T, ...> under this proposal, so under my proposal a when_all() or when_any() simply mean that a given continuation chain is to have its monadic state collapsed and a new monadic state started.
Continuation monads are a very limited specialisation of general monads, especially under the tight constraints of the N3857 requirements, so reuse of any Boost.Functional/Monads I suspect would be minimal. I think that the Boost.Functional/Monads framework I had in mind could be used for your use case. All you need to do is to define a specialized Monad, that in your case is close to a Monad State.
Absolutely correct, and I would expect any code I write for AFIO now will be replaced later with an implementation written using your Boost.Functional/Monads framework. Consider my current proposal as a learning exercise/temporary stopgap code.
I have no problem in proposing a different interface as IIUC your proposal, the user would need to wrap the std::future, boost::expected, ... or whatever AFIO monad or use specific monads::when_all/monads::when_any function.
Correct. I mentioned how one would go about marking up future-like types with metaprogramming in my OP. Anyone can extend the metaprogramming with their own types.
I hope that my comments would help you to clarify my discrepancies about your design. Resuming, I think that you must define your own monad state that could be seen as any other monad using the functional/monads framework.
Once your Boost.Functional/Monads framework is available, more than happy to do so. I hate writing and particularly maintaining boilerplate.
Thoughts regarding the design much appreciated! My thanks in advance.
I will inspect your code in order to see if I've understood your needs as soon as I have enough time.
Thanks Vicente. I deeply appreciate your experience on this. Niall -- Currently unemployed and looking for work in Ireland. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/
Le 05/03/14 18:49, Niall Douglas a écrit :
On 5 Mar 2014 at 13:54, Vicente J. Botet Escriba wrote:
I think that you must define your own monad state that could be seen as any other monad using the functional/monads framework. Once your Boost.Functional/Monads framework is available, more than happy to do so. I hate writing and particularly maintaining boilerplate.
Thoughts regarding the design much appreciated! My thanks in advance.
I will inspect your code in order to see if I've understood your needs as soon as I have enough time.
Hi, I have not taken the time to inspect your code. However I have started my POC of the Boost.Functional/Monads framework in [1]. I have just configured boost::expected<T,E> in an example of use [2] and I will extract the expected configuration soon to a specific file expected_monad.hpp. Please could you take a look at these files and see if you can configure your thenable (Monad state)? I suspect that there will be some of your needs that I have not covered yet. The design is based on [3] and [4]. Sorry for the very draft state of the code :( Best, Vicente [1] https://github.com/ptal/Boost.Expected/blob/master/boost/functional/monad.hp... [2] https://github.com/ptal/Boost.Expected/blob/master/libs/expected/test/safe_d... [3] http://yapb-soc.blogspot.fr/2012/10/monads-in-c.html [4] http://yapb-soc.blogspot.fr/2012/10/fmap-in-c.html
Le 09/03/14 10:36, Vicente J. Botet Escriba a écrit :
I have just configured boost::expected<T,E> in an example of use [2] and I will extract the expected configuration soon to a specific file expected_monad.hpp.
The expected configuration is now at [1] Best, Vicente [5] https://github.com/ptal/Boost.Expected/blob/master/boost/expected/expected_m...
On 9 Mar 2014 at 10:36, Vicente J. Botet Escriba wrote:
Once your Boost.Functional/Monads framework is available, more than happy to do so. I hate writing and particularly maintaining boilerplate.
I have not taken the time to inspect your code. However I have started my POC of the Boost.Functional/Monads framework in [1].
I have just configured boost::expected<T,E> in an example of use [2] and I will extract the expected configuration soon to a specific file expected_monad.hpp.
Please could you take a look at these files and see if you can configure your thenable (Monad state)? I suspect that there will be some of your needs that I have not covered yet.
I'll see what I can do. A bit busy next few days, so give me a while. Thanks Vicente. Niall -- Currently unemployed and looking for work in Ireland. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/
participants (2)
-
Niall Douglas -
Vicente J. Botet Escriba