[thread] boost::future::then
Hi, looking through docs for some other futures implementation [1], I see this: Why not use boost::future? # At the time of writing, 1.53 (the first version with the requisite features) was brand new, not well-tested, and not available to [...] developers. It is still a bit buggy/bleeding-edge They haven't fleshed out the threading model very well yet, e.g. every single then currently spawns a new thread unless you explicitly ask it to work on this thread only, and executor support was nonexistent (and now, is still experimental). Now, 1.53 is ancient. Can anybody comment on these statements regarding boost::future::thread state in 1.59? In particular, suppose I have this: // Thread 1 boost::promise<int> pi; boost::future<int> fi = pi.get_future(); // pass pi to thread 2 somehow fi.then(....); // Thread 2 pi.set_value(100); And I want continuation to be executed in thread 1, assuming that thread 1 runs Qt event loop, allowing me to use QTimer::singleShot, or queued signal, or similar mechanism. How would I go about making boost::future do it? Has anybody done so? Thanks, Volodya
On Fri, Oct 9, 2015 at 9:43 AM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
In particular, suppose I have this:
// Thread 1
boost::promise<int> pi; boost::future<int> fi = pi.get_future();
// pass pi to thread 2 somehow
fi.then(....);
// Thread 2
pi.set_value(100);
And I want continuation to be executed in thread 1, assuming that thread 1 runs Qt event loop, allowing me to use QTimer::singleShot, or queued signal, or similar mechanism. How would I go about making boost::future do it? Has anybody done so?
Thanks, Volodya
I might have misunderstood the question but I would assume a solution would be to build boost.thread with executors enabled[1]. And then wrap the QT event loop in an executor interface[2] and use the boost::future<T>::then(Executor&, ...) overload. I haven't really use this stuff, I have been playing with it in order to find out how to combine boost::asio::io_service and boost::future/async, to no avail sadly. /M [1] it is guarded by a macro which I cannot find in the docs: BOOST_THREAD_PROVIDES_EXECUTORS [2] http://www.boost.org/doc/libs/1_59_0/doc/html/thread/synchronization.html#th... [3] the doc specify an overload then(Scheduler& S) but the code looks like this: #ifdef BOOST_THREAD_PROVIDES_EXECUTORS template<typename Ex, typename F> inline BOOST_THREAD_FUTURE<typename boost::result_of<F(BOOST_THREAD_FUTURE)>::type> then(Ex& ex, BOOST_THREAD_FWD_REF(F) func); // EXTENSION #endif
On 09-Oct-15 4:02 PM, Mikael Olenfalk wrote:
On Fri, Oct 9, 2015 at 9:43 AM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
In particular, suppose I have this:
// Thread 1
boost::promise<int> pi; boost::future<int> fi = pi.get_future();
// pass pi to thread 2 somehow
fi.then(....);
// Thread 2
pi.set_value(100);
And I want continuation to be executed in thread 1, assuming that thread 1 runs Qt event loop, allowing me to use QTimer::singleShot, or queued signal, or similar mechanism. How would I go about making boost::future do it? Has anybody done so?
... I might have misunderstood the question but I would assume a solution would be to build boost.thread with executors enabled[1]. And then wrap the QT event loop in an executor interface[2] and use the boost::future<T>::then(Executor&, ...) overload.
Mikael, thanks for the response. I suppose it would work - although if I need to pass this executor any time I call 'then', it become rather awkward rather quickly. It would be nicer if promise could have an executor, and pass it to future and then to futures returned by 'then', so that I only need to to specify custom behaviour when creating a promise - that can be easily wrapped in a function. Do you have any comments on presence or absence of bugs, and overall API maturity? Having to define macros to get useful functionality is not quite perfect. Thanks, Volodya
On Fri, Oct 9, 2015 at 3:44 PM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
On 09-Oct-15 4:02 PM, Mikael Olenfalk wrote: ...
I might have misunderstood the question but I would assume a solution would be to build boost.thread with executors enabled[1]. And then wrap the QT event loop in an executor interface[2] and use the boost::future<T>::then(Executor&, ...) overload.
Mikael,
thanks for the response. I suppose it would work - although if I need to pass this executor any time I call 'then', it become rather awkward rather quickly. It would be nicer if promise could have an executor, and pass it to future and then to futures returned by 'then', so that I only need to to specify custom behaviour when creating a promise - that can be easily wrapped in a function.
I don't think so, I had a very quick look but didn't see anything.
Do you have any comments on presence or absence of bugs, and overall API maturity? Having to define macros to get useful functionality is not quite perfect.
I have barely played with it, I got little bit discouraged on using it at work as it required enabled undocumented feature macros. Also it seemed to me that it actually didn't compile in VS2015 (which is what I use). My guess is that it is very much still experimental code. If you play with it some more I am very interested in your findings. If you also happen to figure out how to combine it with boost::asio::io_service I would be very interested in hearing your findings, we have a (global) worker-pool-thingie at work which uses io_service and I would like to make boost::async(Executor& ex, C...) work with it so that usage becomes nicer/simpler. I guess the author of Boost.Thread is hanging out on this mailing list, maybe (s)he can comment on status? /M
Le 09/10/15 16:16, Mikael Olenfalk a écrit :
On Fri, Oct 9, 2015 at 3:44 PM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
On 09-Oct-15 4:02 PM, Mikael Olenfalk wrote: ...
I might have misunderstood the question but I would assume a solution would be to build boost.thread with executors enabled[1]. And then wrap the QT event loop in an executor interface[2] and use the boost::future<T>::then(Executor&, ...) overload.
Mikael,
thanks for the response. I suppose it would work - although if I need to pass this executor any time I call 'then', it become rather awkward rather quickly. It would be nicer if promise could have an executor, and pass it to future and then to futures returned by 'then', so that I only need to to specify custom behaviour when creating a promise - that can be easily wrapped in a function.
I don't think so, I had a very quick look but didn't see anything. See my replay to Vladimir.
Do you have any comments on presence or absence of bugs, and overall API maturity? Having to define macros to get useful functionality is not quite perfect.
I have barely played with it, I got little bit discouraged on using it at work as it required enabled undocumented feature macros. Please, next time you find things like that please post on this ML or create a ticket. Anyway, I'm sure that there are a lot of error in the documentation. What is not documented for you? Also it seemed to me that it actually didn't compile in VS2015 (which is what I use). You know I started this some time ago and deliver the first version for boost 1.53 (if Vladimir is right). Since then, there have been a lot of compiler versions that have been released. We do as much as possible to write portable code, but no MSVC version is satisfactory, there is always some feature missing or not working completely.
I'm afraid but since more than 3 years I don't have a Windows machine to develop, so I can just try and see what the regression said me. You can see the results on http://www.boost.org/development/tests/develop/developer/thread.html. The results are no so bad for MSVC. There are some lambda issues.
My guess is that it is very much still experimental code. All the executor stuff is really experimental and I have plans to change a little bit the interface to adhere to the Concurrency TS (once they decide the interface).
If you play with it some more I am very interested in your findings. Me too ;-) If you also happen to figure out how to combine it with boost::asio::io_service I would be very interested in hearing your findings, we have a (global) worker-pool-thingie at work which uses io_service and I would like to make boost::async(Executor& ex, C...) work with it so that usage becomes nicer/simpler.
I guess the author of Boost.Thread is hanging out on this mailing list, maybe (s)he can comment on status?
Here I am. Vicente
On Fri, Oct 9, 2015 at 10:01 PM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 09/10/15 16:16, Mikael Olenfalk a écrit :
On Fri, Oct 9, 2015 at 3:44 PM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
be nicer if promise could have an executor, and pass it to future and then
to futures returned by 'then', so that I only need to to specify custom behaviour when creating a promise - that can be easily wrapped in a function.
I don't think so, I had a very quick look but didn't see anything.
See my replay to Vladimir.
Thanks. Do you have any comments on presence or absence of bugs, and overall API
maturity? Having to define macros to get useful functionality is not quite perfect.
I have barely played with it, I got little bit discouraged on using it at work as it required enabled undocumented feature macros.
Please, next time you find things like that please post on this ML or create a ticket.
Absolutely.
Anyway, I'm sure that there are a lot of error in the documentation. What is not documented for you?
It was mostly the _PROVIDES_EXECUTORS macro which I couldn't find. I tried to do the following documentation fix: https://github.com/boostorg/thread/pull/70
Also it seemed to
me that it actually didn't compile in VS2015 (which is what I use).
You know I started this some time ago and deliver the first version for boost 1.53 (if Vladimir is right). Since then, there have been a lot of compiler versions that have been released. We do as much as possible to write portable code, but no MSVC version is satisfactory, there is always some feature missing or not working completely.
I'm afraid but since more than 3 years I don't have a Windows machine to develop, so I can just try and see what the regression said me. You can see the results on http://www.boost.org/development/tests/develop/developer/thread.html. The results are no so bad for MSVC. There are some lambda issues.
I have a VM on Azure with VS2015 which you can use if you want to, ping me privately if so. My
guess is that it is very much still experimental code.
All the executor stuff is really experimental and I have plans to change a little bit the interface to adhere to the Concurrency TS (once they decide the interface).
Ok.
also happen to figure out how to combine it with boost::asio::io_service I would be very interested in hearing your findings, we have a (global) worker-pool-thingie at work which uses io_service and I would like to make boost::async(Executor& ex, C...) work with it so that usage becomes nicer/simpler.
Here I am.
Vicente
Do you have a recommendation on how to combine the Executors and Schedulers with boost::asio::io_service? Wrapping an io_service in an executor is straightforward but I could not really figure out how to do it for scheduler. Regards, /M
On Sat, Oct 10, 2015 at 9:20 AM, Mikael Olenfalk <mikael.olenfalk@gmail.com> wrote:
On Fri, Oct 9, 2015 at 10:01 PM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Anyway, I'm sure that there are a lot of error in the documentation. What is not documented for you?
It was mostly the _PROVIDES_EXECUTORS macro which I couldn't find. I tried to do the following documentation fix: https://github.com/boostorg/thread/pull/70
I had another look and what confuses me is that _PROVIDES_EXECUTORS is never defined anywhere, except in the examples and tests where it is explicitly defined directly before including <boost/thread.hpp>. (That fact makes my PR wrong btw) Is BOOST_THREAD_PROVIDES_EXECUTORS supposed to be defined when using v4? /M
Le 10/10/15 09:38, Mikael Olenfalk a écrit :
On Sat, Oct 10, 2015 at 9:20 AM, Mikael Olenfalk <mikael.olenfalk@gmail.com> wrote:
On Fri, Oct 9, 2015 at 10:01 PM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Anyway, I'm sure that there are a lot of error in the documentation. What is not documented for you?
It was mostly the _PROVIDES_EXECUTORS macro which I couldn't find. I tried to do the following documentation fix: https://github.com/boostorg/thread/pull/70
I had another look and what confuses me is that _PROVIDES_EXECUTORS is never defined anywhere, except in the examples and tests where it is explicitly defined directly before including <boost/thread.hpp>.
(That fact makes my PR wrong btw)
Is BOOST_THREAD_PROVIDES_EXECUTORS supposed to be defined when using v4?
Hrr, Executors are newer than version 4 and even if they can be considered as an extension, the async()/then() interface change and some compilers don't support the Executor interface. This is why I didn't defined the macro for version 4. Sorry for the confusion. I'll add it as soon as the interface is stable (maybe in BOOST_THREAD_VERSION 5) . Unfortunately it is not yet the case. Don't worry for the PR. I will fix it myself. Thanks for all. Vicente
Le 10/10/15 09:20, Mikael Olenfalk a écrit :
On Fri, Oct 9, 2015 at 10:01 PM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 09/10/15 16:16, Mikael Olenfalk a écrit :
On Fri, Oct 9, 2015 at 3:44 PM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
be nicer if promise could have an executor, and pass it to future and then
to futures returned by 'then', so that I only need to to specify custom behaviour when creating a promise - that can be easily wrapped in a function.
I don't think so, I had a very quick look but didn't see anything. See my replay to Vladimir.
Thanks.
Do you have any comments on presence or absence of bugs, and overall API
maturity? Having to define macros to get useful functionality is not quite perfect.
I have barely played with it, I got little bit discouraged on using it at work as it required enabled undocumented feature macros. Please, next time you find things like that please post on this ML or create a ticket.
Absolutely.
Anyway, I'm sure that there are a lot of error in the documentation. What is not documented for you?
It was mostly the _PROVIDES_EXECUTORS macro which I couldn't find. I tried to do the following documentation fix: https://github.com/boostorg/thread/pull/70 I like this kind of quick contributions.
Also it seemed to
me that it actually didn't compile in VS2015 (which is what I use).
You know I started this some time ago and deliver the first version for boost 1.53 (if Vladimir is right). Since then, there have been a lot of compiler versions that have been released. We do as much as possible to write portable code, but no MSVC version is satisfactory, there is always some feature missing or not working completely.
I'm afraid but since more than 3 years I don't have a Windows machine to develop, so I can just try and see what the regression said me. You can see the results on http://www.boost.org/development/tests/develop/developer/thread.html. The results are no so bad for MSVC. There are some lambda issues.
I have a VM on Azure with VS2015 which you can use if you want to, ping me privately if so. Thanks for the offer.
My
guess is that it is very much still experimental code.
All the executor stuff is really experimental and I have plans to change a little bit the interface to adhere to the Concurrency TS (once they decide the interface).
Ok.
also happen to figure out how to combine it with boost::asio::io_service I would be very interested in hearing your findings, we have a (global) worker-pool-thingie at work which uses io_service and I would like to make boost::async(Executor& ex, C...) work with it so that usage becomes nicer/simpler.
Here I am. Vicente
Do you have a recommendation on how to combine the Executors and Schedulers with boost::asio::io_service? Wrapping an io_service in an executor is straightforward but I could not really figure out how to do it for scheduler.
One you have adapted io_service to conform to the Executor requirements I would expect that scheduler<> sch; sch.on(ios).after(seconds(i)).submit(f); I have never used asio, so maybe I don't see the constraints. Vicenre
Le 09/10/15 15:44, Vladimir Prus a écrit :
On 09-Oct-15 4:02 PM, Mikael Olenfalk wrote:
On Fri, Oct 9, 2015 at 9:43 AM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
In particular, suppose I have this:
// Thread 1
boost::promise<int> pi; boost::future<int> fi = pi.get_future();
// pass pi to thread 2 somehow
fi.then(....);
// Thread 2
pi.set_value(100);
And I want continuation to be executed in thread 1, assuming that thread 1 runs Qt event loop, allowing me to use QTimer::singleShot, or queued signal, or similar mechanism. How would I go about making boost::future do it? Has anybody done so?
... I might have misunderstood the question but I would assume a solution would be to build boost.thread with executors enabled[1]. And then wrap the QT event loop in an executor interface[2] and use the boost::future<T>::then(Executor&, ...) overload.
Mikael,
thanks for the response. I suppose it would work - although if I need to pass this executor any time I call 'then', it become rather awkward rather quickly. Well this is the current interface :) It would be nicer if promise could have an executor, and pass it to future and then to futures returned by 'then', so that I only need to to specify custom behaviour when creating a promise - that can be easily wrapped in a function. If your future is created with auto f = async(ex, fct)
then the call to f.then(fct2); would use the same executor ex. "When the scheduler or launch policy is not provided the continuation inherits the parent's launch policy or scheduler. "
Do you have any comments on presence or absence of bugs, and overall API maturity? Having to define macros to get useful functionality is not quite perfect.
I agree, but that is life. I have blocked the interface since version 3. You must request any additional features explicitly if you want them, or just say you want to use a higher version BOOST_THREAD_VERSION 4 Vicente Vicente
On 09-Oct-15 10:49 PM, Vicente J. Botet Escriba wrote:
Le 09/10/15 15:44, Vladimir Prus a écrit :
On 09-Oct-15 4:02 PM, Mikael Olenfalk wrote:
On Fri, Oct 9, 2015 at 9:43 AM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
In particular, suppose I have this:
// Thread 1
boost::promise<int> pi; boost::future<int> fi = pi.get_future();
// pass pi to thread 2 somehow
fi.then(....);
// Thread 2
pi.set_value(100);
And I want continuation to be executed in thread 1, assuming that thread 1 runs Qt event loop, allowing me to use QTimer::singleShot, or queued signal, or similar mechanism. How would I go about making boost::future do it? Has anybody done so?
... I might have misunderstood the question but I would assume a solution would be to build boost.thread with executors enabled[1]. And then wrap the QT event loop in an executor interface[2] and use the boost::future<T>::then(Executor&, ...) overload.
Mikael,
thanks for the response. I suppose it would work - although if I need to pass this executor any time I call 'then', it become rather awkward rather quickly. Well this is the current interface :) It would be nicer if promise could have an executor, and pass it to future and then to futures returned by 'then', so that I only need to to specify custom behaviour when creating a promise - that can be easily wrapped in a function.
If your future is created with auto f = async(ex, fct)
then the call to
f.then(fct2);
would use the same executor ex.
Vicente, thanks, that's good to know. What if I don't want to use async, but rather create promise/future directly? Can I specified executor when creating future? Suppose I'm in UI thread, and I ask backend thread to do something. I would like for the work to be done in backend thread, and then for all continuations to be executed in UI thread. Or, more generically, in whatever thread has initialized the operation. So, if I implement a "run in UI thread" executor, I don't want async(ex, fct) to execute fct in UI thread. So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work? - Volodya
Le 10/10/15 07:57, Vladimir Prus a écrit :
On 09-Oct-15 10:49 PM, Vicente J. Botet Escriba wrote:
Le 09/10/15 15:44, Vladimir Prus a écrit :
On 09-Oct-15 4:02 PM, Mikael Olenfalk wrote:
On Fri, Oct 9, 2015 at 9:43 AM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
In particular, suppose I have this:
// Thread 1
boost::promise<int> pi; boost::future<int> fi = pi.get_future();
// pass pi to thread 2 somehow
fi.then(....);
// Thread 2
pi.set_value(100);
And I want continuation to be executed in thread 1, assuming that thread 1 runs Qt event loop, allowing me to use QTimer::singleShot, or queued signal, or similar mechanism. How would I go about making boost::future do it? Has anybody done so?
... I might have misunderstood the question but I would assume a solution would be to build boost.thread with executors enabled[1]. And then wrap the QT event loop in an executor interface[2] and use the boost::future<T>::then(Executor&, ...) overload.
Mikael,
thanks for the response. I suppose it would work - although if I need to pass this executor any time I call 'then', it become rather awkward rather quickly. Well this is the current interface :) It would be nicer if promise could have an executor, and pass it to future and then to futures returned by 'then', so that I only need to to specify custom behaviour when creating a promise - that can be easily wrapped in a function.
If your future is created with auto f = async(ex, fct)
then the call to
f.then(fct2);
would use the same executor ex.
Vicente,
thanks, that's good to know. What if I don't want to use async, but rather create promise/future directly? Can I specified executor when creating future? Not now. The executors interface is experimental, so we can consider adding a way to associate a future to an executor. The sources of a future are promise, packaged task, async and make_ready_future/make_exceptional_future.
I've not though too much on it but I believe that it is better to associate the Executor interface to the source instead of changing it directly on the future even if this multiplies the interfaces. This is because I would need to associate the Executor at creation time. I'm not for providing a set_executor function, as this would imply that the shared state must always be aware of a possible executor and so the Executor should be type-erased which is more costly than needed. We could add a set_executor function, but this function will work only if the shared state is allocated lazily, which I(m not sure it is mandated by the standard.
Suppose I'm in UI thread, and I ask backend thread to do something. I would like for the work to be done in backend thread, and then for all continuations to be executed in UI thread. Or, more generically, in whatever thread has initialized the operation. I understand your use case, but what is wrong wrapping the back-end task with the executor that initialized it and use this executor every time you need it.
So, if I implement a "run in UI thread" executor, I don't want async(ex, fct) to execute fct in UI thread.
async(ex, fct) will be executed on the thread(s) associated to ex, not the current thread. What am I missing.
So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work?
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case. Best, Vicente P.S. Maybe Harmut, Thomas Heller or Agustín Bergé have already some experience with his HPC library.
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit :
On 09-Oct-15 10:49 PM, Vicente J. Botet Escriba wrote:
Le 09/10/15 15:44, Vladimir Prus a écrit :
On 09-Oct-15 4:02 PM, Mikael Olenfalk wrote:
On Fri, Oct 9, 2015 at 9:43 AM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
In particular, suppose I have this:
// Thread 1
boost::promise<int> pi; boost::future<int> fi = pi.get_future();
// pass pi to thread 2 somehow
fi.then(....);
// Thread 2
pi.set_value(100);
And I want continuation to be executed in thread 1, assuming that thread 1 runs Qt event loop, allowing me to use QTimer::singleShot, or queued signal, or similar mechanism. How would I go about making boost::future do it? Has anybody done so?
... I might have misunderstood the question but I would assume a solution would be to build boost.thread with executors enabled[1]. And then wrap the QT event loop in an executor interface[2] and use the boost::future<T>::then(Executor&, ...) overload.
Mikael,
thanks for the response. I suppose it would work - although if I need to pass this executor any time I call 'then', it become rather awkward rather quickly. Well this is the current interface :) It would be nicer if promise could have an executor, and pass it to future and then to futures returned by 'then', so that I only need to to specify custom behaviour when creating a promise - that can be easily wrapped in a function.
If your future is created with auto f = async(ex, fct)
then the call to
f.then(fct2);
would use the same executor ex.
Vicente,
thanks, that's good to know. What if I don't want to use async, but rather create promise/future directly? Can I specified executor when creating future? Not now. The executors interface is experimental, so we can consider adding a way to associate a future to an executor. The sources of a future are promise, packaged task, async and make_ready_future/make_exceptional_future.
I've not though too much on it but I believe that it is better to associate the Executor interface to the source instead of changing it directly on the future even if this multiplies the interfaces. This is because I would need to associate the Executor at creation time.
I'm not for providing a set_executor function, as this would imply that the shared state must always be aware of a possible executor and so the Executor should be type-erased which is more costly than needed.
We could add a set_executor function, but this function will work only if the shared state is allocated lazily, which I(m not sure it is mandated by the standard.
Suppose I'm in UI thread, and I ask backend thread to do something. I would like for the work to be done in backend thread, and then for all continuations to be executed in UI thread. Or, more generically, in whatever thread has initialized the operation. I understand your use case, but what is wrong wrapping the back-end task with the executor that initialized it and use this executor every time you need it.
So, if I implement a "run in UI thread" executor, I don't want async(ex, fct) to execute fct in UI thread.
async(ex, fct) will be executed on the thread(s) associated to ex, not the current thread. What am I missing.
So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work?
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request Vicente
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit :
So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work?
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter. https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca... any feedback is welcome, Vicente
On Sun, Oct 11, 2015 at 8:30 PM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit :
So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work?
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
any feedback is welcome,
Awesome, just a few minutes ago I got bit by the fact that .then() doesn't inherit the executor (disclaimer: haven't used it though yet). /M
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit :
So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work?
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente, thanks for the addition. I'll take a look at implementing Qt-friendly future on top of this, and report how it goes. Thanks, Volodya
Le 13/10/15 10:34, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit :
So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work?
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente,
thanks for the addition. I'll take a look at implementing Qt-friendly future on top of this, and report how it goes.
Thanks, Volodya
You are welcome. Note that I've not implemented the constructors as it need to change the hierarchy of classes, but promise::set_executor and packaged_task::set_executor which is more expensive as I need to type-erase the executor. Any feedback would be much appreciated. Vicente
Le 13/10/15 19:33, Vicente J. Botet Escriba a écrit :
Le 13/10/15 10:34, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit :
So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work?
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente,
thanks for the addition. I'll take a look at implementing Qt-friendly future on top of this, and report how it goes.
Thanks, Volodya
You are welcome.
Note that I've not implemented the constructors as it need to change the hierarchy of classes, but promise::set_executor and packaged_task::set_executor which is more expensive as I need to type-erase the executor.
Any feedback would be much appreciated.
After looking at the Concurrency TS, I'm not sure if it is not a good idea to store the executor so that f.then(cont) launch cont on the executor associated to f. In the Concurrent TS, ::then() can execute the continuation on any thread. " * When the object's shared state is ready, the continuation |/INVOKE/(/DECAY_COPY/(std::forward<F>(func)), std::move(*this))| is called on an unspecified thread of execution with the call to |/DECAY_COPY/()| being evaluated in the thread that called |then|. " What do you think? Best, Vicente
Le 16/10/15 01:19, Vicente J. Botet Escriba a écrit :
Le 13/10/15 19:33, Vicente J. Botet Escriba a écrit :
Le 13/10/15 10:34, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit : > > So, the function must be executed in different thread from all > the continuations, > and it would seem I'd need to something set executor on promise > for that to work? >
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente,
thanks for the addition. I'll take a look at implementing Qt-friendly future on top of this, and report how it goes.
Thanks, Volodya
You are welcome.
Note that I've not implemented the constructors as it need to change the hierarchy of classes, but promise::set_executor and packaged_task::set_executor which is more expensive as I need to type-erase the executor.
Any feedback would be much appreciated.
After looking at the Concurrency TS, I'm not sure if it is not a good idea to store the executor so that
f.then(cont)
launch cont on the executor associated to f.
In the Concurrent TS, ::then() can execute the continuation on any thread.
"
* When the object's shared state is ready, the continuation |/INVOKE/(/DECAY_COPY/(std::forward<F>(func)), std::move(*this))| is called on an unspecified thread of execution with the call to |/DECAY_COPY/()| being evaluated in the thread that called |then|.
"
What do you think?
Hi again, I believe that it would be better if the use states explicitly that she wants to inherit the policy unsing a specific policy f.then(launch::inherit, cont); I have not implemented this already, but I have already the possibility to use the undocumented executor policy f.then(launch::executor, cont); This would launch cont using the inherited executor. Best, Vicente
On 17-Oct-15 7:41 PM, Vicente J. Botet Escriba wrote:
Le 16/10/15 01:19, Vicente J. Botet Escriba a écrit :
Le 13/10/15 19:33, Vicente J. Botet Escriba a écrit :
Le 13/10/15 10:34, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit : > Le 10/10/15 07:57, Vladimir Prus a écrit : >> >> So, the function must be executed in different thread from all the continuations, >> and it would seem I'd need to something set executor on promise for that to work? >> > > I will see how adding an Executor parameter to promise, packaged_task constructors and > make_ready_future/make_exceptional_future could be implemented if this will solve your use case. > I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente,
thanks for the addition. I'll take a look at implementing Qt-friendly future on top of this, and report how it goes.
Thanks, Volodya
You are welcome.
Note that I've not implemented the constructors as it need to change the hierarchy of classes, but promise::set_executor and packaged_task::set_executor which is more expensive as I need to type-erase the executor.
Any feedback would be much appreciated.
After looking at the Concurrency TS, I'm not sure if it is not a good idea to store the executor so that
f.then(cont)
launch cont on the executor associated to f.
In the Concurrent TS, ::then() can execute the continuation on any thread.
"
* When the object's shared state is ready, the continuation |/INVOKE/(/DECAY_COPY/(std::forward<F>(func)), std::move(*this))| is called on an unspecified thread of execution with the call to |/DECAY_COPY/()| being evaluated in the thread that called |then|.
"
What do you think?
Hi again,
I believe that it would be better if the use states explicitly that she wants to inherit the policy unsing a specific policy
f.then(launch::inherit, cont);
I have not implemented this already, but I have already the possibility to use the undocumented executor policy
f.then(launch::executor, cont);
This would launch cont using the inherited executor.
I suppose I can implement QtFuture that: - Holds boost::future and a pointer to my executor - Has .then method (or better yet, overrides the >> operator) - Calls boost::future::then with whatever parameters From the code above I'm not sure what f.then(launch::inherit, cont); would do? Launch using the executor that was specified in promise? It would seem, really, that if an executor was specified in promise, then using that executor should be default? - Volodya -- Vladimir Prus http://vladimirprus.com
Le 29/10/15 10:14, Vladimir Prus a écrit :
On 17-Oct-15 7:41 PM, Vicente J. Botet Escriba wrote:
Le 16/10/15 01:19, Vicente J. Botet Escriba a écrit :
Le 13/10/15 19:33, Vicente J. Botet Escriba a écrit :
Le 13/10/15 10:34, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit : > Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit : >> Le 10/10/15 07:57, Vladimir Prus a écrit : >>> >>> So, the function must be executed in different thread from all >>> the continuations, >>> and it would seem I'd need to something set executor on >>> promise for that to work? >>> >> >> I will see how adding an Executor parameter to promise, >> packaged_task constructors and >> make_ready_future/make_exceptional_future could be implemented >> if this will solve your use case. >> > I've create https://svn.boost.org/trac/boost/ticket/11717 to > track this feature request > > This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente,
thanks for the addition. I'll take a look at implementing Qt-friendly future on top of this, and report how it goes.
Thanks, Volodya
You are welcome.
Note that I've not implemented the constructors as it need to change the hierarchy of classes, but promise::set_executor and packaged_task::set_executor which is more expensive as I need to type-erase the executor.
Any feedback would be much appreciated.
After looking at the Concurrency TS, I'm not sure if it is not a good idea to store the executor so that
f.then(cont)
launch cont on the executor associated to f.
In the Concurrent TS, ::then() can execute the continuation on any thread.
"
* When the object's shared state is ready, the continuation |/INVOKE/(/DECAY_COPY/(std::forward<F>(func)), std::move(*this))| is called on an unspecified thread of execution with the call to |/DECAY_COPY/()| being evaluated in the thread that called |then|.
"
What do you think?
Hi again,
I believe that it would be better if the use states explicitly that she wants to inherit the policy unsing a specific policy
f.then(launch::inherit, cont);
I have not implemented this already, but I have already the possibility to use the undocumented executor policy
f.then(launch::executor, cont);
This would launch cont using the inherited executor.
I suppose I can implement QtFuture that:
- Holds boost::future and a pointer to my executor - Has .then method (or better yet, overrides the >> operator) - Calls boost::future::then with whatever parameters
From the code above I'm not sure what
f.then(launch::inherit, cont);
would do? Launch using the executor that was specified in promise? Yes. It would seem, really, that if an executor was specified in promise, then using that executor should be default? Well, this is now the default behavior as it corresponds to what was documented.
However I believe that the default behavior shouldn't be to inherit. I want to be able to execute the continuation synchronously by the thread that made ready the promise. I have added an experimental flag BOOST_THREAD_CONTINUATION_SYNC that switch the default behavior. Alternatively I could add a launch::sync policy (and I will do it). So the single thing that will remain is to decide what is the default behavior. Volodya, if you use this in production code, please, be ready to have changes in the next release. There will be no promise::set_executor. You will need to give the executor at the creation of the promise, as it was planned from the beginning (See that the set_executor has not been documented and that the ticket is still open. It is just that this need to refactor the core more deeply. The advantage is that you will not need anymore to allocate the future (no type erasure needed). Vicente
On 29-Oct-15 1:01 PM, Vicente J. Botet Escriba wrote:
Le 29/10/15 10:14, Vladimir Prus a écrit :
On 17-Oct-15 7:41 PM, Vicente J. Botet Escriba wrote:
Le 16/10/15 01:19, Vicente J. Botet Escriba a écrit :
Le 13/10/15 19:33, Vicente J. Botet Escriba a écrit :
Le 13/10/15 10:34, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote: > Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit : >> Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit : >>> Le 10/10/15 07:57, Vladimir Prus a écrit : >>>> >>>> So, the function must be executed in different thread from all the continuations, >>>> and it would seem I'd need to something set executor on promise for that to work? >>>> >>> >>> I will see how adding an Executor parameter to promise, packaged_task constructors and >>> make_ready_future/make_exceptional_future could be implemented if this will solve your use case. >>> >> I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request >> >> > This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and > packaged_task::set_executor. These should be replaced by constructors having an executor as parameter. > > https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente,
thanks for the addition. I'll take a look at implementing Qt-friendly future on top of this, and report how it goes.
Thanks, Volodya
You are welcome.
Note that I've not implemented the constructors as it need to change the hierarchy of classes, but promise::set_executor and packaged_task::set_executor which is more expensive as I need to type-erase the executor.
Any feedback would be much appreciated.
After looking at the Concurrency TS, I'm not sure if it is not a good idea to store the executor so that
f.then(cont)
launch cont on the executor associated to f.
In the Concurrent TS, ::then() can execute the continuation on any thread.
"
* When the object's shared state is ready, the continuation |/INVOKE/(/DECAY_COPY/(std::forward<F>(func)), std::move(*this))| is called on an unspecified thread of execution with the call to |/DECAY_COPY/()| being evaluated in the thread that called |then|.
"
What do you think?
Hi again,
I believe that it would be better if the use states explicitly that she wants to inherit the policy unsing a specific policy
f.then(launch::inherit, cont);
I have not implemented this already, but I have already the possibility to use the undocumented executor policy
f.then(launch::executor, cont);
This would launch cont using the inherited executor.
I suppose I can implement QtFuture that:
- Holds boost::future and a pointer to my executor - Has .then method (or better yet, overrides the >> operator) - Calls boost::future::then with whatever parameters
From the code above I'm not sure what
f.then(launch::inherit, cont);
would do? Launch using the executor that was specified in promise? Yes. It would seem, really, that if an executor was specified in promise, then using that executor should be default? Well, this is now the default behavior as it corresponds to what was documented.
However I believe that the default behavior shouldn't be to inherit. I want to be able to execute the continuation synchronously by the thread that made ready the promise. I have added an experimental flag BOOST_THREAD_CONTINUATION_SYNC that switch the default behavior. Alternatively I could add a launch::sync policy (and I will do it). So the single thing that will remain is to decide what is the default behavior.
I see. I've only started to play with this code, so can't yet voice a strong opinion either way.
Volodya, if you use this in production code, please, be ready to have changes in the next release. There will be no promise::set_executor. You will need to give the executor at the creation of the promise, as it was planned from the beginning (See that the set_executor has not been documented and that the ticket is still open. It is just that this need to refactor the core more deeply. The advantage is that you will not need anymore to allocate the future (no type erasure needed).
Thanks for the warning - I don't use it in production at present. Passing executor when creating a promise is also fine with me. Will it still be possible to pass an explicit executor to ::then? -- Vladimir Prus http://vladimirprus.com
Le 29/10/15 11:21, Vladimir Prus a écrit :
On 29-Oct-15 1:01 PM, Vicente J. Botet Escriba wrote:
Le 29/10/15 10:14, Vladimir Prus a écrit :
On 17-Oct-15 7:41 PM, Vicente J. Botet Escriba wrote:
Hi again,
I believe that it would be better if the use states explicitly that she wants to inherit the policy unsing a specific policy
f.then(launch::inherit, cont);
I have not implemented this already, but I have already the possibility to use the undocumented executor policy
f.then(launch::executor, cont);
This would launch cont using the inherited executor.
I suppose I can implement QtFuture that:
- Holds boost::future and a pointer to my executor - Has .then method (or better yet, overrides the >> operator) - Calls boost::future::then with whatever parameters
From the code above I'm not sure what
f.then(launch::inherit, cont);
would do? Launch using the executor that was specified in promise? Yes. It would seem, really, that if an executor was specified in promise, then using that executor should be default? Well, this is now the default behavior as it corresponds to what was documented.
However I believe that the default behavior shouldn't be to inherit. I want to be able to execute the continuation synchronously by the thread that made ready the promise. I have added an experimental flag BOOST_THREAD_CONTINUATION_SYNC that switch the default behavior. Alternatively I could add a launch::sync policy (and I will do it). So the single thing that will remain is to decide what is the default behavior.
I see. I've only started to play with this code, so can't yet voice a strong opinion either way.
Volodya, if you use this in production code, please, be ready to have changes in the next release. There will be no promise::set_executor. You will need to give the executor at the creation of the promise, as it was planned from the beginning (See that the set_executor has not been documented and that the ticket is still open. It is just that this need to refactor the core more deeply. The advantage is that you will not need anymore to allocate the future (no type erasure needed).
Thanks for the warning - I don't use it in production at present. Passing executor when creating a promise is also fine with me. Will it still be possible to pass an explicit executor to ::then?
Of course. Vicente
On 29/10/2015 23:01, Vicente J. Botet Escriba wrote:
Le 29/10/15 10:14, Vladimir Prus a écrit :
It would seem, really, that if an executor was specified in promise, then using that executor should be default? Well, this is now the default behavior as it corresponds to what was documented.
However I believe that the default behavior shouldn't be to inherit. I want to be able to execute the continuation synchronously by the thread that made ready the promise. I have added an experimental flag BOOST_THREAD_CONTINUATION_SYNC that switch the default behavior. Alternatively I could add a launch::sync policy (and I will do it). So the single thing that will remain is to decide what is the default behavior.
Admittedly I'm not very familiar with the Concurrency TS, but I think the reason that continuations are supposed to be executed via the inherited executor by default is for the marshal-back-to-main/UI-thread case when handing off background work. In the case where background work is itself registering further continuations, then the background thread is supposed to be running a more generalised executor which can allow synchronous execution (eg. the executor's policy says "must run on one of these threadpool threads", and you happen to already be running on one of those threads, or "can run on any thread"). This is similar behaviour to ASIO's io_service::dispatch. I don't think trying to decide this at the time the promise is made ready is the right choice. Unless I'm just misunderstanding what you're referring to here.
Le 06/11/15 08:35, Gavin Lambert a écrit :
On 29/10/2015 23:01, Vicente J. Botet Escriba wrote:
Le 29/10/15 10:14, Vladimir Prus a écrit :
It would seem, really, that if an executor was specified in promise, then using that executor should be default? Well, this is now the default behavior as it corresponds to what was documented.
However I believe that the default behavior shouldn't be to inherit. I want to be able to execute the continuation synchronously by the thread that made ready the promise. I have added an experimental flag BOOST_THREAD_CONTINUATION_SYNC that switch the default behavior. Alternatively I could add a launch::sync policy (and I will do it). So the single thing that will remain is to decide what is the default behavior.
Admittedly I'm not very familiar with the Concurrency TS, but I think the reason that continuations are supposed to be executed via the inherited executor by default is for the marshal-back-to-main/UI-thread case when handing off background work.
In the case where background work is itself registering further continuations, then the background thread is supposed to be running a more generalised executor which can allow synchronous execution (eg. the executor's policy says "must run on one of these threadpool threads", and you happen to already be running on one of those threads, or "can run on any thread").
This is similar behaviour to ASIO's io_service::dispatch.
I don't think trying to decide this at the time the promise is made ready is the right choice. Unless I'm just misunderstanding what you're referring to here.
You misunderstood it. The launch::sync policy is usable with future::then() f.then launch::sync, next); Here next continuation will be executed on the same thread that will make ready the future. Vicente
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit :
So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work?
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente, I've created a very early experiment, here: https://gist.github.com/vprus/3a7bea25e4c0a0e4222d Does it look generally right for you? I'm worried that Executor interface requires methods like close/closed, try_executing_once and reschedule_until that don't seem documented in detail. Is it fine to leave them undocumented in my case? -- Vladimir Prus http://vladimirprus.com
Le 29/10/15 10:08, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit :
So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work?
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente,
I've created a very early experiment, here:
https://gist.github.com/vprus/3a7bea25e4c0a0e4222d
Does it look generally right for you? I don't know Qt. It seems ok beside tht fact that you create an Executor for each Task. Maybe this is not important for you as your Executor class is adapting Qt. I'm worried that Executor interface requires methods like close/closed, This is used to shutdown smoothly the worker threads. try_executing_once and reschedule_until that don't seem documented in detail. These should be put in another concept refining executor. I will try to do it for 1.61. Is it fine to leave them undocumented in my case?
Well, I don't know what do you mean by undocumented and in your case. Document for what? Vicente
On 29-Oct-15 12:49 PM, Vicente J. Botet Escriba wrote:
Le 29/10/15 10:08, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit :
So, the function must be executed in different thread from all the continuations, and it would seem I'd need to something set executor on promise for that to work?
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente,
I've created a very early experiment, here:
https://gist.github.com/vprus/3a7bea25e4c0a0e4222d
Does it look generally right for you? I don't know Qt. It seems ok beside tht fact that you create an Executor for each Task. Maybe this is not important for you as your Executor class is adapting Qt.
It's not hugely important, since in practice the work performance is orders of magnitude longer than QObject creation. Anyway, I possibly can create an executor per thread on demand, and reuse them.
I'm worried that Executor interface requires methods like close/closed, This is used to shutdown smoothly the worker threads.
So where will ::close be called? Is it ever called from boost::thread? That seems unlikely, since boost::thread does not know at which point I no longer need an executor. And if boost::thread does not call this, why am I required to implement this method at all?
try_executing_once and reschedule_until that don't seem documented in detail. These should be put in another concept refining executor. I will try to do it for 1.61. Is it fine to leave them undocumented in my case?
Well, I don't know what do you mean by undocumented and in your case. Document for what?
I meant 'unimplemented' - will try_executing_once and reschedule_until be ever called in my case? If not, which you seem to indicate above, then everything is fine. - Volodya -- Vladimir Prus http://vladimirprus.com
On 29-Oct-15 12:49 PM, Vicente J. Botet Escriba wrote:
Le 29/10/15 10:08, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit :
Le 10/10/15 07:57, Vladimir Prus a écrit : > > So, the function must be executed in different thread from all > the continuations, > and it would seem I'd need to something set executor on promise > for that to work? >
I will see how adding an Executor parameter to promise, packaged_task constructors and make_ready_future/make_exceptional_future could be implemented if this will solve your use case.
I've create https://svn.boost.org/trac/boost/ticket/11717 to track this feature request
This commit contains a fix for this issue as well as the addition of the VERY-EXPERIMENTAL promise::set_executor and packaged_task::set_executor. These should be replaced by constructors having an executor as parameter.
https://github.com/boostorg/thread/commit/b8db8fef8b28414d16c66761badc1c6fca...
Vicente,
I've created a very early experiment, here:
https://gist.github.com/vprus/3a7bea25e4c0a0e4222d
Does it look generally right for you? I don't know Qt. It seems ok beside tht fact that you create an Executor for each Task. Maybe this is not important for you as your Executor class is adapting Qt.
It's not hugely important, since in practice the work performance is orders of magnitude longer than QObject creation. Anyway, I possibly can create an executor per thread on demand, and reuse them. Don't worry with the new version where you will give the Executor at
Le 29/10/15 11:13, Vladimir Prus a écrit : promise creation the cost will be minimal in your case.
I'm worried that Executor interface requires methods like close/closed, This is used to shutdown smoothly the worker threads.
So where will ::close be called? Is it ever called from boost::thread? That seems unlikely, since boost::thread does not know at which point I no longer need an executor. And if boost::thread does not call this, why am I required to implement this method at all?
Some executors call close in the destructor before joining all the threads. In addition the use can request to close the submission of new work. If needed we can move this also to another Executor layer, so that Executors can be created without close()/closed().
try_executing_once and reschedule_until that don't seem documented in detail. These should be put in another concept refining executor. I will try to do it for 1.61. Is it fine to leave them undocumented in my case?
Well, I don't know what do you mean by undocumented and in your case. Document for what?
I meant 'unimplemented' - will try_executing_once and reschedule_until be ever called in my case? If not, which you seem to indicate above, then everything is fine.
Right, these functions are not called of you don't call them. So for the time been and as the executor interface requires them, you can just implement them as you did. I will create 3 tickets: * Add launch:sync policy * Extract close/closed to a more specific shutdonw-executor interface. * Extract try_executing_one, reschedule_until to a more specific reentrant executor interface. Glad to have a user that push me to improve the library :) Well, at least this is what I believe I'm doing. Best, Vicente
On 29-Oct-15 4:09 PM, Vicente J. Botet Escriba wrote:
Does it look generally right for you? I don't know Qt. It seems ok beside tht fact that you create an Executor for each Task. Maybe this is not important for you as your Executor class is adapting Qt.
It's not hugely important, since in practice the work performance is orders of magnitude longer than QObject creation. Anyway, I possibly can create an executor per thread on demand, and reuse them. Don't worry with the new version where you will give the Executor at promise creation the cost will be minimal in your case.
I'm worried that Executor interface requires methods like close/closed, This is used to shutdown smoothly the worker threads.
So where will ::close be called? Is it ever called from boost::thread? That seems unlikely, since boost::thread does not know at which point I no longer need an executor. And if boost::thread does not call this, why am I required to implement this method at all? Some executors call close in the destructor before joining all the threads. In addition the use can request to close the submission of new work.
If needed we can move this also to another Executor layer, so that Executors can be created without close()/closed().
Yes, that would make sense. I found it a tad confusing that a concept requires some methods that I'd have to call myself. Usually, a concept requires only the method that the library will actually call.
try_executing_once and reschedule_until that don't seem documented in detail. These should be put in another concept refining executor. I will try to do it for 1.61. Is it fine to leave them undocumented in my case?
Well, I don't know what do you mean by undocumented and in your case. Document for what?
I meant 'unimplemented' - will try_executing_once and reschedule_until be ever called in my case? If not, which you seem to indicate above, then everything is fine.
Right, these functions are not called of you don't call them. So for the time been and as the executor interface requires them, you can just implement them as you did.
I will create 3 tickets: * Add launch:sync policy * Extract close/closed to a more specific shutdonw-executor interface. * Extract try_executing_one, reschedule_until to a more specific reentrant executor interface.
Thanks.
Glad to have a user that push me to improve the library :) Well, at least this is what I believe I'm doing.
You're welcome. In the meantime, I'll try to make my example into more reusable code. While Qt has QFuture, it has no continuation, which is sad. -- Vladimir Prus http://vladimirprus.com
On 29-Oct-15 4:09 PM, Vicente J. Botet Escriba wrote:
I'm worried that Executor interface requires methods like
close/closed, This is used to shutdown smoothly the worker threads.
So where will ::close be called? Is it ever called from boost::thread? That seems unlikely, since boost::thread does not know at which point I no longer need an executor. And if boost::thread does not call this, why am I required to implement this method at all? Some executors call close in the destructor before joining all the threads. In addition the use can request to close the submission of new work.
If needed we can move this also to another Executor layer, so that Executors can be created without close()/closed().
Yes, that would make sense. I found it a tad confusing that a concept requires some methods that I'd have to call myself. Usually, a concept requires only the method that the library will actually call. You are right. I'll check where these are close/closed()/try_executing_one/ are needed for the concept and the
Le 29/10/15 14:34, Vladimir Prus a écrit : polymorphic interface. Vicente
Le 29/10/15 14:09, Vicente J. Botet Escriba a écrit :
On 29-Oct-15 12:49 PM, Vicente J. Botet Escriba wrote:
Le 29/10/15 10:08, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit :
Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit : > Le 10/10/15 07:57, Vladimir Prus a écrit : >> Vicente,
I've created a very early experiment, here:
https://gist.github.com/vprus/3a7bea25e4c0a0e4222d
Does it look generally right for you? I don't know Qt. It seems ok beside tht fact that you create an Executor for each Task. Maybe this is not important for you as your Executor class is adapting Qt.
It's not hugely important, since in practice the work performance is orders of magnitude longer than QObject creation. Anyway, I possibly can create an executor per thread on demand, and reuse them. Don't worry with the new version where you will give the Executor at
Le 29/10/15 11:13, Vladimir Prus a écrit : promise creation the cost will be minimal in your case.
Hrr, I'm not able to remove completely the Executor type erasure as the type of f2 would depend on the Executor associated to the shared state of f1, that is unknown.
MyExecutor ex; auto f1 = async(ex, cont); auto f2 = f1.then(launch::inherit, cont); So, f1 could have a shared_state that depends on MyExecutor, however f2 would have an shared state that depends on the type erased boost::executor. That means that it is better for the user to store the specific executor in some way and use it as follow auto f2 = f1.then(ex, cont); Anyway, I would try implement as at least I will able to optimize auto f1 = async(ex, cont); and promise<int> p(ex); Vicente
On 29-Oct-15 11:00 PM, Vicente J. Botet Escriba wrote:
Le 29/10/15 14:09, Vicente J. Botet Escriba a écrit :
Le 29/10/15 11:13, Vladimir Prus a écrit :
On 29-Oct-15 12:49 PM, Vicente J. Botet Escriba wrote:
Le 29/10/15 10:08, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote:
Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit : > Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit : >> Le 10/10/15 07:57, Vladimir Prus a écrit : >>> Vicente,
I've created a very early experiment, here:
https://gist.github.com/vprus/3a7bea25e4c0a0e4222d
Does it look generally right for you? I don't know Qt. It seems ok beside tht fact that you create an Executor for each Task. Maybe this is not important for you as your Executor class is adapting Qt.
It's not hugely important, since in practice the work performance is orders of magnitude longer than QObject creation. Anyway, I possibly can create an executor per thread on demand, and reuse them. Don't worry with the new version where you will give the Executor at promise creation the cost will be minimal in your case.
Hrr, I'm not able to remove completely the Executor type erasure as the type of f2 would depend on the Executor associated to the shared state of f1, that is unknown.
MyExecutor ex; auto f1 = async(ex, cont); auto f2 = f1.then(launch::inherit, cont);
So, f1 could have a shared_state that depends on MyExecutor, however f2 would have an shared state that depends on the type erased boost::executor.
Okay, but does it matter much for performance?
That means that it is better for the user to store the specific executor in some way and use it as follow
auto f2 = f1.then(ex, cont);
I've tried to do so, at: https://gist.github.com/vprus/7bb55cc6c6a1d4bbd3c4 This initial version was straight-forward, though it will probably need further tweaks for production use. What does boost::future::then do right now if the continuation returns boost::future, is it unwrapped automatically? -- Vladimir Prus http://vladimirprus.com
On 30-Oct-15 10:08 AM, Vladimir Prus wrote:
I've tried to do so, at:
https://gist.github.com/vprus/7bb55cc6c6a1d4bbd3c4
This initial version was straight-forward, though it will probably need further tweaks for production use.
What does boost::future::then do right now if the continuation returns boost::future, is it unwrapped automatically?
BTW, I've noticed that if I call boost::future::get() twice in my continuation, on the same future object, the second call throws, saying: Operation not permitted on an object without an associated state you can find the example at: https://gist.github.com/vprus/d53ef0860608b856aa60 Am I missing something? Looking at both std::future and boost::future documentation, I see no indication that calling get clears the state. Thanks, Volodya -- Vladimir Prus http://vladimirprus.com
Le 30/10/15 13:42, Vladimir Prus a écrit :
On 30-Oct-15 10:08 AM, Vladimir Prus wrote:
BTW, I've noticed that if I call boost::future::get() twice in my continuation, on the same future object, the second call throws, saying:
Operation not permitted on an object without an associated state
you can find the example at:
https://gist.github.com/vprus/d53ef0860608b856aa60
Am I missing something? Looking at both std::future and boost::future documentation, I see no indication that calling get clears the state.
No. a future moves itself to the continuation so you can not associate twice a continuation. You need to use shared_future. Please, create a ticket. I will look for where this is said, and try to be more explicit so that the next one doesn't miss the specificity. The major difference between future and shared_future is that, you can get only once the value. Vicente
On 30-Oct-15 8:28 PM, Vicente J. Botet Escriba wrote:
Le 30/10/15 13:42, Vladimir Prus a écrit :
On 30-Oct-15 10:08 AM, Vladimir Prus wrote:
BTW, I've noticed that if I call boost::future::get() twice in my continuation, on the same future object, the second call throws, saying:
Operation not permitted on an object without an associated state
you can find the example at:
https://gist.github.com/vprus/d53ef0860608b856aa60
Am I missing something? Looking at both std::future and boost::future documentation, I see no indication that calling get clears the state.
No. a future moves itself to the continuation so you can not associate twice a continuation. You need to use shared_future.
Please, create a ticket.
Trac or GitHub? -- Vladimir Prus http://vladimirprus.com
Le 30/10/15 20:24, Vladimir Prus a écrit :
On 30-Oct-15 8:28 PM, Vicente J. Botet Escriba wrote:
Le 30/10/15 13:42, Vladimir Prus a écrit :
On 30-Oct-15 10:08 AM, Vladimir Prus wrote:
BTW, I've noticed that if I call boost::future::get() twice in my continuation, on the same future object, the second call throws, saying:
Operation not permitted on an object without an associated state
you can find the example at:
https://gist.github.com/vprus/d53ef0860608b856aa60
Am I missing something? Looking at both std::future and boost::future documentation, I see no indication that calling get clears the state.
No. a future moves itself to the continuation so you can not associate twice a continuation. You need to use shared_future.
Please, create a ticket.
Trac or GitHub?
Trac :)
Le 30/10/15 08:08, Vladimir Prus a écrit :
On 29-Oct-15 11:00 PM, Vicente J. Botet Escriba wrote:
On 29-Oct-15 12:49 PM, Vicente J. Botet Escriba wrote:
Le 29/10/15 10:08, Vladimir Prus a écrit :
On 11-Oct-15 9:30 PM, Vicente J. Botet Escriba wrote: > Le 10/10/15 16:58, Vicente J. Botet Escriba a écrit : >> Le 10/10/15 15:26, Vicente J. Botet Escriba a écrit : >>> Le 10/10/15 07:57, Vladimir Prus a écrit : >>>> Vicente,
I've created a very early experiment, here:
https://gist.github.com/vprus/3a7bea25e4c0a0e4222d
Does it look generally right for you? I don't know Qt. It seems ok beside tht fact that you create an Executor for each Task. Maybe this is not important for you as your Executor class is adapting Qt.
It's not hugely important, since in practice the work performance is orders of magnitude longer than QObject creation. Anyway, I possibly can create an executor per thread on demand, and reuse them. Don't worry with the new version where you will give the Executor at
Le 29/10/15 11:13, Vladimir Prus a écrit : promise creation the cost will be minimal in your case.
Hrr, I'm not able to remove completely the Executor type erasure as
Le 29/10/15 14:09, Vicente J. Botet Escriba a écrit : the type of f2 would depend on the Executor associated to the shared state of f1, that is unknown.
MyExecutor ex; auto f1 = async(ex, cont); auto f2 = f1.then(launch::inherit, cont);
So, f1 could have a shared_state that depends on MyExecutor, however f2 would have an shared state that depends on the type erased boost::executor.
Okay, but does it matter much for performance?
That means that it is better for the user to store the specific executor in some way and use it as follow
auto f2 = f1.then(ex, cont);
I've tried to do so, at:
https://gist.github.com/vprus/7bb55cc6c6a1d4bbd3c4
This initial version was straight-forward, though it will probably need further tweaks for production use.
What does boost::future::then do right now if the continuation returns boost::future, is it unwrapped automatically?
Unfortunately, no. I based my development in the fist proposals. I have this on to of my to do list already. Vicente
Le 09/10/15 21:49, Vicente J. Botet Escriba a écrit :
Le 09/10/15 15:44, Vladimir Prus a écrit :
thanks for the response. I suppose it would work - although if I need to pass this executor any time I call 'then', it become rather awkward rather quickly.
Well this is the current interface :)
It would be nicer if promise could have an executor, and pass it to future and then to futures returned by 'then', so that I only need to to specify custom behaviour when creating a promise - that can be easily wrapped in a function. If your future is created with auto f = async(ex, fct)
then the call to
f.then(fct2);
would use the same executor ex.
"When the scheduler or launch policy is not provided the continuation inherits the parent's launch policy or scheduler. "
Hrr, I've just checked that I've not implemented this :(. This needs some kind of type-erasure of the Executor as we need to store it in order to use the same executor. Ticket created https://svn.boost.org/trac/boost/ticket/11716 Vicente
Le 09/10/15 15:02, Mikael Olenfalk a écrit :
On Fri, Oct 9, 2015 at 9:43 AM, Vladimir Prus <vladimir.prus@gmail.com> wrote:
In particular, suppose I have this:
// Thread 1
boost::promise<int> pi; boost::future<int> fi = pi.get_future();
// pass pi to thread 2 somehow
fi.then(....);
// Thread 2
pi.set_value(100);
And I want continuation to be executed in thread 1, assuming that thread 1 runs Qt event loop, allowing me to use QTimer::singleShot, or queued signal, or similar mechanism. How would I go about making boost::future do it? Has anybody done so?
Thanks, Volodya
I might have misunderstood the question but I would assume a solution would be to build boost.thread with executors enabled[1]. And then wrap the QT event loop in an executor interface[2] and use the boost::future<T>::then(Executor&, ...) overload. Right.
I haven't really use this stuff, I have been playing with it in order to find out how to combine boost::asio::io_service and boost::future/async, to no avail sadly.
/M
[1] it is guarded by a macro which I cannot find in the docs: BOOST_THREAD_PROVIDES_EXECUTORS Or BOOST_THREAD_VERSION 4
[2] http://www.boost.org/doc/libs/1_59_0/doc/html/thread/synchronization.html#th...
[3] the doc specify an overload then(Scheduler& S) Could you point me where exactly you can fin this?
The doc says template<typename S, typename F> |future| <http://www.boost.org/doc/libs/1_59_0/doc/html/thread/synchronization.html#thread.synchronization.futures.reference.unique_future><typename boost::result_of<F(shared_future)>::type> then(S& scheduler, F&& func) const; // EXTENSION
but the code looks like this:
#ifdef BOOST_THREAD_PROVIDES_EXECUTORS template<typename Ex, typename F> inline BOOST_THREAD_FUTURE<typename boost::result_of<F(BOOST_THREAD_FUTURE)>::type> then(Ex& ex, BOOST_THREAD_FWD_REF(F) func); // EXTENSION #endif
Vicente
On Fri, Oct 9, 2015 at 9:43 PM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 09/10/15 15:02, Mikael Olenfalk a écrit :
[3] the doc specify an overload then(Scheduler& S)
Could you point me where exactly you can fin this?
Vicente
I tried to fix the doc in the following PR: https://github.com/boostorg/thread/pull/69 /M
Le 09/10/15 09:43, Vladimir Prus a écrit :
Hi,
looking through docs for some other futures implementation [1], I see this:
Why not use boost::future? #
At the time of writing, 1.53 (the first version with the requisite features) was brand new, not well-tested, and not available to [...] developers. It is still a bit buggy/bleeding-edge They haven't fleshed out the threading model very well yet, e.g. every single then currently spawns a new thread unless you explicitly ask it to work on this thread only, and executor support was nonexistent (and now, is still experimental).
Now, 1.53 is ancient. Can anybody comment on these statements regarding boost::future::thread state in 1.59?
The comment should be almost right, the first implementations had a lot of issues. Even in 1.59 the feature is broken (at least respect to what the standard TS Concurrency says) as the future resulting from ::then() blocks. If you admit that the resulting futures blocks the develop branch is OK for you (which includes some fixes respect to 1.59). I have a branch (https://github.com/boostorg/thread/tree/feature/non_blocking_futures) that don't blocks on any future, but this will break async. I have another branch that has copyable executors and non-blocking futures (https://github.com/boostorg/thread/tree/feature/make_executors_copyable). I expect to have this branch merged for 1.60.
In particular, suppose I have this:
// Thread 1
boost::promise<int> pi; boost::future<int> fi = pi.get_future();
// pass pi to thread 2 somehow
fi.then(....);
// Thread 2
pi.set_value(100);
And I want continuation to be executed in thread 1, assuming that thread 1 runs Qt event loop, allowing me to use QTimer::singleShot, or queued signal, or similar mechanism. How would I go about making boost::future do it? Has anybody done so?
You need the Executor interface. ::then(ex, f); http://www.boost.org/doc/libs/1_59_0/doc/html/thread/synchronization.html#th... Vicente
participants (4)
-
Gavin Lambert -
Mikael Olenfalk -
Vicente J. Botet Escriba -
Vladimir Prus