Hi, I am not a threading expert, so please be patient when I got everything wrong. I tried to read up on all the proposals mentioned in this thread. What is troubling me, especially with executioners, is that the proposals are not very detailed about their guaranties. Assume the creation of an async work package in an executioner as outlined in p0058R0 std::future<T> fut =std::async(ex, function,arguments); fut.wait();//?? what p0058R0 is not saying anything about is what is happening when the current thread creating fut is a working thread of the executioner ex, for example when ex is a threadpool. In the worst implementation, fut.wait() will keep the working thread waiting and thus ex has one working thread less until fut becomes ready. This behaviour could cause a deadlock, as all working threads might be waiting for a future to become ready thus exhausting the computing capabilities of ex. The desired behaviour would be that ex is rescheduling a new work-package for the current thread until fut is ready (for example the work package of fut in case it is not scheduled on another thread yet). This has to be guarantied. similarly, I miss possibilities to give the scheduler hints on what should be ready in which order, especially in hierarchical models (the current work package depends on all packages it spawns but it might have to wait for a specific subset of them before it can actually compute something), but also in graphs (e.g. computing block A_ij of some matrix requires the results of Blocks A_i-1j and A_ij-1). the default implementation can ignore hints, but i think that advanced executioners will use this information, especially when a small number of worker threads needs to compute a large amount of work packages with complex dependencies. one way to give this information might simply be fut.wait();//i am waiting for this work package so this is a dependency but maybe a simple extension of the future interface to mark critical dependencies might make this even more powerful: fut.mark_as_critical();//if fut is scheduled in an executioner, inform ex to reschedule to the front of the work package list and an extension of boost wait_for_all (no variadic templates for simplicity) could look like template<typename F1,typename F2,typename F3> void wait_for_all(F1& f1,F2& f2,F3& f3) { //inform that these futures are critical for this work package to continue f1.mark_as_critical(); f2.mark_as_critical(); f3.mark_as_critical(); //now just wait (and get new work packages by the executioner of this thread) f1.wait(); f2.wait(); f3.wait(); } As I said, I might have been wrong all along, thanks for reading to here anyways :) On 2015-10-14 08:03, Vicente J. Botet Escriba wrote:
Le 13/10/15 20:34, Mikael Olenfalk a écrit :
On Tue, Oct 13, 2015 at 7:48 PM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 13/10/15 10:39, Mikael Olenfalk a écrit :
However using shared_ptr as copyable ensures the lifetime issue, but I don't see the advantage in the split then.There is a problem with the shared_ptr approach that my current implementation in make_executors_copyable shares. The destructor of the shared state can be called in a thread that is part of the threads of the executor. That mean that the destructor must check if the thread to join is this thread and then not call the join.
I only use the shared_ptr internally in order to detect when the underlying executor is gone. In our code base we only use it to ensure that nobody posts to an executor after it has been destroyed (during shutdown). The split is "necessary" to hide the weak_ptr (because it is ugly) and in order to ensure that nobody accidentally uses a raw reference (the submit() function is gone from the executor). Oh, I missed the sing uses weak_ptr. This justify the split. I'll experiment on the make_executor_copyable_branch.
I hadn't even thought of the problem where the shared-state is destroyed in the wrong thread but you are obviously correct. Is it possible to come up with a design which does not have this problem? No that I know.
So, do we want a design that force the user to ensure that the executor (execution_context) outlive the executor sinks (executor_type?
Please no.
Or, just a copyable executor?
How does that work when the actual underlying thingie (e.g. boost::asio::io_service) is non-copyable?
The boost::asio::io_service could be stored on the shared state that is not copyable nor movable.
Best, Vicente
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost