Futures Join the review and discussion starting January 5, 2009. I need commitments for reviews from threading experts on the two Futures library candidates. Braddock Gaskill - http://braddock.com/~braddock/future Anthony Williams - http://www.justsoftwaresolutions.co.uk/files/n2561_future.hpp Early comments are welcome.
Tom Brinkman wrote:
Futures
Join the review and discussion starting January 5, 2009.
I need commitments for reviews from threading experts on the two Futures library candidates.
Braddock Gaskill - http://braddock.com/~braddock/future Anthony Williams - http://www.justsoftwaresolutions.co.uk/files/n2561_future.hpp
Early comments are welcome.
Is the expected result of the review to choose a library (or none) or to create a third library that is a mixture of the two? -- Michael Marcin
----- Original Message ----- From: "Tom Brinkman" <reportbase@gmail.com> To: <boost-announce@lists.boost.org>; <boost@lists.boost.org> Sent: Tuesday, December 30, 2008 8:15 PM Subject: [boost] Futures - Reviews Needed (January 5, 2009)
Futures
Join the review and discussion starting January 5, 2009.
I need commitments for reviews from threading experts on the two Futures library candidates.
Braddock Gaskill - http://braddock.com/~braddock/future Anthony Williams - http://www.justsoftwaresolutions.co.uk/files/n2561_future.hpp
Early comments are welcome.
Hi, This will be a particular review for everal reasons: * there are two libraries for review the 'same' concept * this is the first time we review a library that has a corresponding accepted standard for C++0x (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2800.pdf based on http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2627.html) * one of them is presented by one of the authors of the C++0x proposal. I want to do a review, but IMO any Future Boost library should at least provide the current C++0x Future accepted standard and use the Boost.Exception library as mechanism to transport the exceptions. Other libraries that should be used are Boost.Chrono and Boost.Move, but these are not yet even in the schedule queue. Once we have an implementation of the standard interface we can see how other features can be implemented on top of this interface. Neither Braddock nor Anthony implements the C++0x accepted interface even if the Anthony is much closer. But Anthony Williams has already an implementation of the C++0x Future on his comercial just::thread library (http://www.stdthread.co.uk/forum/index.php?topic=64.0). The link on the Review Scheduled of the Anthony library has no other documentation than http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2561.html which seems not enough to start a review. Anthony has posted on this list an update of its library (http://www.justsoftwaresolutions.co.uk/threading/free-implementation-of-c++-...). Tom, It is not clear from your mail which library from Anthony will be reviewed. Is it http://www.justsoftwaresolutions.co.uk/files/n2561_future.hpp, http://www.justsoftwaresolutions.co.uk/threading/free-implementation-of-c++-... or http://www.stdthread.co.uk/forum/index.php?topic=64.0? Thanks in advance, Vicente
Vicente Botet Escriba wrote:
Tom, It is not clear from your mail which library from Anthony will be reviewed. Is it http://www.justsoftwaresolutions.co.uk/files/n2561_future.hpp, http://www.justsoftwaresolutions.co.uk/threading/free-implementation-of-c++-... or http://www.stdthread.co.uk/forum/index.php?topic=64.0?
http://www.justsoftwaresolutions.co.uk/files/n2561_future.hpp does not contain any wait_for_any or wait_for_all functionality and it includes packaged_task. Is this really the version that is being reviewed? Best Regards, Johan Torp www.johantorp.com -- View this message in context: http://www.nabble.com/Futures---Reviews-Needed-%28January-5%2C-2009%29-tp212... Sent from the Boost - Dev mailing list archive at Nabble.com.
I have some comments on the design of Braddock Gaskill's Boost.Future library. I haven't really looked at the other one because I don't want to reverse engineer the code. I would not accept that library without documentation comparable to Gaskill's. I have a fair amount of experience with using threads from C++. While I've seen the idea of futures before I haven't used them. I really like the documentation. It is a major help for evaluating the design decisions. I have not yet used the library, at this point I'm just evaluating the design and interface in the docs. Since I haven't acutally used it yet I'm not yet prepared to say accept or reject on this library. It appears to be well thought through. I like the split between promise and future to separate the put and get interfaces. There is one point where I question the design decision: reference semantics. There are certainly times when that is useful, but I can also see times when it is not. My preference would be for a exposed concrete class. That could then be used as a building block for a reference semantic class. Allowing users to minimize interaction with memory management seems like a potential win. This doesn't seem like a huge issue, but I thought I would raise it for discussion. A second issue has to do with capturing and rethrowing exceptions. This is a very cool capability but the implementation is limited because it can't detect and use user derived subtypes of the system exception types. This can be done with external polymorphism and would be a significant improvement. Instead of storing a runtime_error store a pointer to an internal abstract base class such as untyped_exception_wrapper. Derive from that a templated class exception_wrapper that stores the user's type and can rethrow that type. Then the set_exception member can be template and the exception type, wraps it with exception_wrapper and stores the pointer to the abstract base. Then when the user goes to get the value from the future and it finds that there was an exception, it calls the rethrow method on the abstract base, the templated derived class does the concrete rethrow of the user's type and the user can then catch that type. This way the future library can catch and rethrow exceptions of types that it knows nothing about. Below is some demonstration code for this pattern. -Steve Karmesin Here is exception_wrapper,h: #ifndef _exception_wrapper_h_ #define _exception_wrapper_h_ class exception_wrapper { public: virtual void rethrow() const = 0; }; template<typename E> class typed_exception_wrapper : public exception_wrapper { public: typed_exception_wrapper(const E& e) : exception_(e) {} void rethrow() const { throw exception_; } private: E exception_; }; template<typename E> exception_wrapper* wrap_exception(const E& e) { return new typed_exception_wrapper<E>(e); } #endif And a simple usage of it: #include "exception_wrapper.h" #include <iostream> using namespace std; exception_wrapper* caught; void bar() { throw "This is an example of an exception of arbitrary class"; } void foo() { try { bar(); } catch (const char *p) { caught = wrap_exception(p); } } void baz() { if ( caught ) caught->rethrow(); } int main() { // Call foo, which will throw and save the arbitrary type. foo(); // Call baz, which will rethrow the arbitrary type. try { baz(); cout << "Huh, we should have thrown!" << endl; } catch (const char *p) { cout << "Caught: " << p << endl; } return 0; }
AMDG Steve Karmesin wrote:
A second issue has to do with capturing and rethrowing exceptions. This is a very cool capability but the implementation is limited because it can't detect and use user derived subtypes of the system exception types. This can be done with external polymorphism and would be a significant improvement.
Instead of storing a runtime_error store a pointer to an internal abstract base class such as untyped_exception_wrapper. Derive from that a templated class exception_wrapper that stores the user's type and can rethrow that type.
Then the set_exception member can be template and the exception type, wraps it with exception_wrapper and stores the pointer to the abstract base.
Then when the user goes to get the value from the future and it finds that there was an exception, it calls the rethrow method on the abstract base, the templated derived class does the concrete rethrow of the user's type and the user can then catch that type. This way the future library can catch and rethrow exceptions of types that it knows nothing about.
Below is some demonstration code for this pattern.
This functionality is already in Boost. http://www.boost.org/libs/exception/doc/tutorial_exception_ptr.html In Christ, Steven Watanabe
On Wed, Dec 31, 2008 at 12:53 PM, Steven Watanabe <watanabesj@gmail.com>wrote:
This functionality is already in Boost. http://www.boost.org/libs/exception/doc/tutorial_exception_ptr.html
Cool. Then the Future library should use it. -Steve
A couple of points that I would want to see clarified on the Boost.Futures proposal to make sure the major design decisions are vetted: The first is on callbacks. The description does not make clear what thread the callback is required to be done in. I would guess that the easiest implementation is to put it be in the thread in which the promise is fulfilled, but I would want that to be specified in the interface since it would affect how I would use it. A second question on callbacks: Would it not make sense to use the boost signals package for this? I think that provides a very rich interface for binding values to the callbacks, and that is very useful. Next, I'm wondering about the reference semantics. Is that there because it is not sensible to copy them? If that is the case then the alternative would be to prohibit copy and not require memory management under the hood. The downside there is that the user would usually have to use pointer to futures/promises in order to build containers of them. On the whole I think I come down on the side of liking the reference semantics, but I don't think it is a no brainer and would want to see some commentary on it. In general I like this package a lot and wish I could use it on my current project :-). -Steve Karmesin
"Tom Brinkman" <reportbase@gmail.com> writes:
Futures
Join the review and discussion starting January 5, 2009.
I need commitments for reviews from threading experts on the two Futures library candidates.
Braddock Gaskill - http://braddock.com/~braddock/future Anthony Williams - http://www.justsoftwaresolutions.co.uk/files/n2561_future.hpp
The latest version of that library is at: Docs: http://www.justsoftwaresolutions.co.uk/files/futures_documentation.html Code+Docs: http://www.justsoftwaresolutions.co.uk/files/n2561_futures_revised_20080530.... Anthony -- Anthony Williams Author of C++ Concurrency in Action | http://www.manning.com/williams Custom Software Development | http://www.justsoftwaresolutions.co.uk Just Software Solutions Ltd, Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL, UK
Here is my review. Quite long so please bear with me :) * What is your evaluation of the design? The split promise/future design, exception propagation mechanism and broken promise detection are well thought through. I have full confidence that both authors are quite capable to nail down any details remaining regarding these. I think we should have support for movable types which motivates the distinction between shared_future and unique_future (as in William's version). I am however, very concerned about the waiting mechanisms and callback hooks of both libraries. See motivation below. * What is your evaluation of the implementation? Both implementations seem to be of good quality. * What is your evaluation of the documentation? I haven't spent much time reviewing them, at a first glance they both seem fine but William's might add some flesh. I personally wouldn't mind allowing a lot of documentation to be added after a formal review acceptance. * What is your evaluation of the potential usefulness of the library? It depends. It has the potential to become an extremely useful library if we can find a powerful enough interface, otherwise I'm afraid it might cause more problems than it solves. See motivation below. * Did you try to use the library? With what compiler? Did you have any problems? I've used a slimmed down version of Gaskill's implementation in production code. I have not written any toy programs using either of the two libraries * How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I've spent a lot of hours thinking about the waiting and callback problems, 50h++ during the last year. I’ve focused the review solely on these matters. * Are you knowledgeable about the problem domain? Quite. I've have many years of experience in writing commercial C++ multithreaded applications and have recently written a five months master's thesis on the shift to parallel processors, thread based programming and memory consistency. * Do you think the library should be accepted as a Boost library? Unfortunately, no. Some fundamental and very difficult design decision remains. If, however, we can find consensus on these matters, I'd gladly change my vote to yes. --- MOTIVATION--- My hope is that we can find a minimal acceptable library which support whatever use cases we deem important. I am worried that the C++ committee is too focused on using futures as a part of a thread pool interface. Gaskill has identified a number of use cases (guards, scheduling, lazy futures etc) which OTOH might be too elaborate. But if the interface isn't powerful enough we risk getting a wide variety of non-compatible future implementations out there. I'll give a brief introduction for those who haven't followed my earlier future-related posts. I think that "choice" or waiting hasn't been thought through properly. In non-trivial use cases you probably have multiple futures and would like to schedule executions once they are completed. Gaskill's implementation allows schedulers and multiple waiting to be implemented by allowing arbitrary callbacks to be executed in the promise fulfilling thread (set in future::add_callback, executed in promise::set). This is powerful but quite dangerous, for instance if that callback throws an exception, it will propagate to the promise-fulfilling thread. Another problem is that you cannot implement future::remove_callback in a dead-lock free way so that the callback is guaranteed to not take place as soon as future::remove_callback returns. Alternatively, the callback can be executed lazily in future::wait and future::get once the future-listening thread calls them. This removes some problems but introduces some others. For instance a scheduler cannot know when a promise is fulfilled until a client somehow waits for them. And there might be several future-listening threads which want to execute different callbacks, etc. A third option is that the user starts an extra fire-and-forget thread (or run it in a thread pool or has an idle thread waiting around) which waits for the future. However, threads are rather heavy weight in contemporary desktop operating systems (default stack space 1mb in windows and 8mb in linux) so this would, IMHO, severely limit the usefulness of futures. William's version provides the free functions wait_for_any and wait_for_all. Using these, a scheduler could schedule an arbitrary amount of futures using just one additional future waiting thread. However, the current implementation of wait_for_any suffers some serious problems: - It takes O(N) time to wait for one of N futures. If you want to schedule M callbacks after M futures are ready you get O(M^2) waiting time. - wait_for_any does not allow arbitrary code to be executed upon readiness. This forces the user to implement a runtime mapping after wait_for_any has returned which determines which future was ready and takes the right action. This is both cumbersome and less efficient than if the callback had been bound before initiating waiting. Perhaps more importantly than these shortcomings, William's version does not support "future composition" without using a third thread. This is not necessarily a flaw but rather a design decision. Consider implementing addition of two future integers: future<int> add(future<int> lhs, future<int> rhs); - Using Gaskill's version the actual addition would be performed by lhs's or rhs's promise-fulfilling thread. - Using the lazy approach I suggested above, the addition is carried out in the returned future's wait() or get() method. - William's version requires a third thread which waits for lhs and rhs and then carries out the addition. This makes future composition way too heavyweight for ubiquitous simple logic. For instance, the proposed future operators would require a thread per operator (f1 || f2 && f3 || f4 would require three threads!) and Gaskill's guarded schedulers would be quite unusable. OTOH, not supporting "threadless future composition" makes futures much simpler and more lightweight (which might be good for threadpool). IMO, this is the most important and difficult design decision. Unfortunately, I think it will take quite some time to find a good interface for "threadless future composition". If we had fibers or some other lightweight execution mechanism (like Erlang’s processes) it would be a non-problem. --- DISCUSSION --- IMHO, we need to: #0 Decide if we want to support "threadless future composition" and if so sketch on some implementation #1 Come up with a better wait_for_any mechanism (probably some kind of future container, another idea is exposing the internal condition variable) #2 Discuss if we want Gaskill's promise::is_needed and promise::wait_until_needed functionality #3 Discuss if we want William's promise::set_wait_callback #4 Verify that the proposed future is suitable for all identified use cases or discard them as irrelevant. The use cases I've seen discussed are boost.asio, libpoet, the proposed std.threadpool, future operators, lazy futures, guards and future streams #2 adds some state and complexity to a future/promise pair. How big of a problem is this for thread pools (which I presume want a really lightweight future to allow really fine grain tasks)? My 5 cents on #3 is that it's not really needed to implement efficient thread pools. The user might as well write std::wait_or_work(some_future); which either waits for the future or performs additional work if the executing thread happens to be a worker thread. The task calling std::wait_or_work gets a dependency to std.threadpool but I think it is well worth it considering the added clarity and simplification of a future object. Allowing execution of arbitrary code in future::wait from promise::set_wait_callback (a la William's implementation) is just as bad and hackish as allowing promise::set_value to execute code from future::add_callback (a la Gaskill's implementation). I hope I haven't kept a too negative tone in my review. I think both implementations are very good and I really appreciate the effort you both have spent :) Best Regards, Johan Torp www.johantorp.com -- View this message in context: http://www.nabble.com/Futures---Reviews-Needed-%28January-5%2C-2009%29-tp212... Sent from the Boost - Dev mailing list archive at Nabble.com.
Hi, ----- Original Message ----- From: "Johan Torp" <johan.torp@gmail.com> To: <boost@lists.boost.org> Sent: Monday, January 05, 2009 10:29 PM Subject: Re: [boost] Futures - Reviews Needed (January 5, 2009)
Here is my review. Quite long so please bear with me :)
--- MOTIVATION---
My hope is that we can find a minimal acceptable library which support whatever use cases we deem important. I am worried that the C++ committee is too focused on using futures as a part of a thread pool interface. Gaskill has identified a number of use cases (guards, scheduling, lazy futures etc) which OTOH might be too elaborate. But if the interface isn't powerful enough we risk getting a wide variety of non-compatible future implementations out there.
Yes, IMO the documentation should show at least that the accepted library can manage with these use cases. Why not as examples.
I'll give a brief introduction for those who haven't followed my earlier future-related posts. I think that "choice" or waiting hasn't been thought through properly. In non-trivial use cases you probably have multiple futures and would like to schedule executions once they are completed.
A third option is that the user starts an extra fire-and-forget thread (or run it in a thread pool or has an idle thread waiting around) which waits for the future. However, threads are rather heavy weight in contemporary desktop operating systems (default stack space 1mb in windows and 8mb in linux) so this would, IMHO, severely limit the usefulness of futures.
This is exactly what I have implemented with the fork_after function using wait_for_all. If the asynchronous executor is a thread_pool you will need a task and not a thread, so no heavy at all.
William's version provides the free functions wait_for_any and wait_for_all. Using these, a scheduler could schedule an arbitrary amount of futures using just one additional future waiting thread. However, the current implementation of wait_for_any suffers some serious problems: - It takes O(N) time to wait for one of N futures.
I have proposed an implementation that takes constant time without any sort of polling. But no comments for the moment.
If you want to schedule M callbacks after M futures are ready you get O(M^2) waiting time. With my implementation only O(M)
- wait_for_any does not allow arbitrary code to be executed upon readiness. This forces the user to implement a runtime mapping after wait_for_any has returned which determines which future was ready and takes the right action. This is both cumbersome and less efficient than if the callback had been bound before initiating waiting.
My fork_after function allows to execute asynchronously any function after the completion of N functions.
Perhaps more importantly than these shortcomings, William's version does not support "future composition" without using a third thread. This is not necessarily a flaw but rather a design decision. Consider implementing addition of two future integers:
future<int> add(future<int> lhs, future<int> rhs); - Using Gaskill's version the actual addition would be performed by lhs's or rhs's promise-fulfilling thread. - Using the lazy approach I suggested above, the addition is carried out in the returned future's wait() or get() method. - William's version requires a third thread which waits for lhs and rhs and then carries out the addition.
What about int direct_add(int lhs, int rhs) { return lhs+rhs}; act_adapter<shared_future<int> > add(shared_future<int> lhs, shared_future<int> rhs) { return fork_after(bind(direct_add, lns, rhs), fusion::tuple(lhs, rhs)); }
This makes future composition way too heavyweight for ubiquitous simple logic. For instance, the proposed future operators would require a thread per operator (f1 || f2 && f3 || f4 would require three threads!) and
Three task with a thread_pool. Is this better for you?
Gaskill's guarded schedulers would be quite unusable. OTOH, not supporting "threadless future composition" makes futures much simpler and more lightweight (which might be good for threadpool). IMO, this is the most important and difficult design decision. Unfortunately, I think it will take quite some time to find a good interface for "threadless future composition". If we had fibers or some other lightweight execution mechanism (like Erlang’s processes) it would be a non-problem.
--- DISCUSSION ---
IMHO, we need to: #0 Decide if we want to support "threadless future composition" and if so sketch on some implementation
I haven't found a safe implementation.
#1 Come up with a better wait_for_any mechanism (probably some kind of future container, another idea is exposing the internal condition variable)
Are you requiring some kind of public registration on completion as used by the future_waiters? Anthony has sais that it could be something like that, but no concrete porposal for the the moment.
#2 Discuss if we want Gaskill's promise::is_needed and promise::wait_until_needed functionality #3 Discuss if we want William's promise::set_wait_callback
I would want this but I prefer that if it is done non intrusively, i.e. only if possible in a hierarchycal design.
#4 Verify that the proposed future is suitable for all identified use cases or discard them as irrelevant.
I agree.
The use cases I've seen discussed are boost.asio, libpoet, the proposed std.threadpool, future operators, lazy futures, guards and future streams
#2 adds some state and complexity to a future/promise pair. How big of a problem is this for thread pools (which I presume want a really lightweight future to allow really fine grain tasks)?
My 5 cents on #3 is that it's not really needed to implement efficient thread pools. The user might as well write std::wait_or_work(some_future); which either waits for the future or performs additional work if the executing thread happens to be a worker thread. The task calling std::wait_or_work gets a dependency to std.threadpool but I think it is well worth it considering the added clarity and simplification of a future object. Allowing execution of arbitrary code in future::wait from promise::set_wait_callback (a la William's implementation) is just as bad and hackish as allowing promise::set_value to execute code from future::add_callback (a la Gaskill's implementation).
I have already proposed to Olivier a reschedule_until_ready that schedule other tasks until the asynchronous completion token (a future by example) is ready. Best, Vicente
Vicente Botet Escriba wrote:
A third option is that the user starts an extra fire-and-forget thread (or run it in a thread pool or has an idle thread waiting around) which waits for the future. However, threads are rather heavy weight in contemporary desktop operating systems (default stack space 1mb in windows and 8mb in linux) so this would, IMHO, severely limit the usefulness of futures.
This is exactly what I have implemented with the fork_after function using wait_for_all. If the asynchronous executor is a thread_pool you will need a task and not a thread, so no heavy at all.
However, thread pools do not aim to provide good latency which might make a such future implementation useless for scheduling related usage (which might still be ok since it could be the best design option we have). I'd love to hear the authors views on this. Also we do not have any thread pools in boost today. Vicente Botet Escriba wrote:
William's version provides the free functions wait_for_any and wait_for_all. Using these, a scheduler could schedule an arbitrary amount of futures using just one additional future waiting thread. However, the current implementation of wait_for_any suffers some serious problems: - It takes O(N) time to wait for one of N futures.
I have proposed an implementation that takes constant time without any sort of polling. But no comments for the moment.
Actually, I started writing a reply but I couldn't really understand your implementation. I was hoping one of the authors would reply first and I could join in later. To solve the complexity problem, I agree we need to expose some stateful abstraction and can't do with just free functions. Vicente Botet Escriba wrote:
If you want to schedule M callbacks after M futures are ready you get O(M^2) waiting time. With my implementation only O(M)
- wait_for_any does not allow arbitrary code to be executed upon readiness. This forces the user to implement a runtime mapping after wait_for_any has returned which determines which future was ready and takes the right action. This is both cumbersome and less efficient than if the callback had been bound before initiating waiting.
My fork_after function allows to execute asynchronously any function after the completion of N functions.
Perhaps more importantly than these shortcomings, William's version does not support "future composition" without using a third thread. This is not necessarily a flaw but rather a design decision. Consider implementing addition of two future integers:
future<int> add(future<int> lhs, future<int> rhs); - Using Gaskill's version the actual addition would be performed by lhs's or rhs's promise-fulfilling thread. - Using the lazy approach I suggested above, the addition is carried out in the returned future's wait() or get() method. - William's version requires a third thread which waits for lhs and rhs and then carries out the addition.
What about
int direct_add(int lhs, int rhs) { return lhs+rhs}; act_adapter<shared_future<int> > add(shared_future<int> lhs, shared_future<int> rhs) { return fork_after(bind(direct_add, lns, rhs), fusion::tuple(lhs, rhs)); }
This makes future composition way too heavyweight for ubiquitous simple logic. For instance, the proposed future operators would require a thread per operator (f1 || f2 && f3 || f4 would require three threads!) and
Three task with a thread_pool. Is this better for you?
If we choose require a third thread (or thread pool) for future composition, I'm sure there are lots of nice interfaces like your proposal above. I don't want to spend too much time designing these fancier features now though. Vicente Botet Escriba wrote:
#1 Come up with a better wait_for_any mechanism (probably some kind of future container, another idea is exposing the internal condition variable)
Are you requiring some kind of public registration on completion as used by the future_waiters? Anthony has sais that it could be something like that, but no concrete porposal for the the moment.
At this point I'm not suggesting anything. I'd like the authors acknowledgement that this is a problem first. Who knows, maybe they have some insight which makes it a non-problem. I'd rather build consensus on what the problems are first, then solve the problems. Vicente Botet Escriba wrote:
#2 Discuss if we want Gaskill's promise::is_needed and promise::wait_until_needed functionality #3 Discuss if we want William's promise::set_wait_callback
I would want this but I prefer that if it is done non intrusively, i.e. only if possible in a hierarchycal design.
Personally I really hope the C++ committee turn around on providing promise::set_wait_callback. It seems very hackish and unneccessary. Java.util.concurrent.Future can provide "stack-stealing" transparently because it is an interface but that's a whole different story. Vicente Botet Escriba wrote:
#4 Verify that the proposed future is suitable for all identified use cases or discard them as irrelevant.
I agree.
I would not be surprised if requiring a third thread for future composition would render it useless for many of Gaskill's use cases, libpoet and asio - which means most of the identified use cases. But I'm far from sure. Best Regards, Johan Torp www.johantorp.com -- View this message in context: http://www.nabble.com/Futures---Reviews-Needed-%28January-5%2C-2009%29-tp212... Sent from the Boost - Dev mailing list archive at Nabble.com.
----- Original Message ----- From: "Johan Torp" <johan.torp@gmail.com> To: <boost@lists.boost.org> Sent: Monday, January 19, 2009 6:59 AM Subject: Re: [boost] Futures - Reviews Needed (January 5, 2009)
Also we do not have any thread pools in boost today.
There is at least the ThreadPool proposal from Olivier on the queue Schedule. Vicente
----- Original Message ----- From: "Johan Torp" <johan.torp@gmail.com> To: <boost@lists.boost.org> Sent: Monday, January 19, 2009 6:59 AM Subject: Re: [boost] Futures - Reviews Needed (January 5, 2009)
However, thread pools do not aim to provide good latency which might make a such future implementation useless for scheduling related usage (which might still be ok since it could be the best design option we have). I'd love to hear the authors views on this.
Could you develop the use case you have in mind.
Actually, I started writing a reply but I couldn't really understand your implementation. I was hoping one of the authors would reply first and I could join in later. To solve the complexity problem, I agree we need to expose some stateful abstraction and can't do with just free functions.
Replay to the post. Let me know what you don't understad.
If we choose require a third thread (or thread pool) for future composition, I'm sure there are lots of nice interfaces like your proposal above. I don't want to spend too much time designing these fancier features now though.
Which other features would you want?
Vicente Botet Escriba wrote:
#1 Come up with a better wait_for_any mechanism (probably some kind of future container, another idea is exposing the internal condition variable)
Are you requiring some kind of public registration on completion as used by the future_waiters? Anthony has sais that it could be something like that, but no concrete porposal for the the moment.
At this point I'm not suggesting anything. I'd like the authors acknowledgement that this is a problem first. Who knows, maybe they have some insight which makes it a non-problem. I'd rather build consensus on what the problems are first, then solve the problems.
I understand
I would not be surprised if requiring a third thread for future composition would render it useless for many of Gaskill's use cases, libpoet and asio - which means most of the identified use cases. But I'm far from sure.
Could you or someone develop these needs. Best, Vicente
Vicente Botet Escriba wrote:
However, thread pools do not aim to provide good latency which might make a such future implementation useless for scheduling related usage (which might still be ok since it could be the best design option we have). I'd love to hear the authors views on this.
Could you develop the use case you have in mind.
I'm thinking about libpoet, asio and the scheduling related use cases from Gaskill's documentation. Vicente Botet Escriba wrote:
Actually, I started writing a reply but I couldn't really understand your implementation. I was hoping one of the authors would reply first and I could join in later. To solve the complexity problem, I agree we need to expose some stateful abstraction and can't do with just free functions.
Replay to the post. Let me know what you don't understad.
I'd like acknowledgement from Anthony - that he too thinks the complexity is a real problem - before trying to solve it. Vicente Botet Escriba wrote:
If we choose require a third thread (or thread pool) for future composition, I'm sure there are lots of nice interfaces like your proposal above. I don't want to spend too much time designing these fancier features now though.
Which other features would you want?
As few as possible :) I hope we can accept a minimal library first, then let real users' need dictate the features. Vicente Botet Escriba wrote:
I would not be surprised if requiring a third thread for future composition would render it useless for many of Gaskill's use cases, libpoet and asio - which means most of the identified use cases. But I'm far from sure.
Could you or someone develop these needs.
The authors of asio, libpoet and Gaskill are best suited to answer if Anthony's proposal would be useful to them. Cheers, Johan -- View this message in context: http://www.nabble.com/Futures---Reviews-Needed-%28January-5%2C-2009%29-tp212... Sent from the Boost - Dev mailing list archive at Nabble.com.
On Monday 19 January 2009 13:19, Johan Torp wrote:
Vicente Botet Escriba wrote:
I would not be surprised if requiring a third thread for future composition would render it useless for many of Gaskill's use cases, libpoet and asio - which means most of the identified use cases. But I'm far from sure.
Could you or someone develop these needs.
The authors of asio, libpoet and Gaskill are best suited to answer if Anthony's proposal would be useful to them.
Unfortunately, I haven't had time to follow the future review threads in detail, or any changes that may have been made to the submitted library since I last looked at them many months ago. I can only summarize some things about the future implementation in libpoet in the hope it will be helpful in some way. As far as composing futures, the poet::future_combining_barrier supports obtaining a new future whose value is obtained by applying an arbitrary "combiner" functor to the values from a group of input futures. The combiner is always executed in a future-waiting thread, not the promise-fulfilling thread, or a third thread. The implementation relies on internal signals being emitted immediately when a promise is fulfilled, as well as internal event queues associated with futures. I described it a little in this post: http://lists.boost.org/Archives/boost/2008/06/138422.php With respect to an O(1) future wait_for_any, I added poet::future_selector, which is like a future queue which reorders based on the order that the futures pushed into the queue become ready. I was able to use them to reimplement my schedulers for active objects, without the scheduler using "future is ready" callbacks directly. To be more clear about my use case: I have a thread which executes method requests, and each method request has a group of futures associated with it. The thread's scheduler has a queue of method requests and it selects a method request to run when all the method request's futures are ready.
----- Original Message ----- From: "Frank Mori Hess" <fmhess@speakeasy.net> To: <boost@lists.boost.org> Sent: Monday, January 19, 2009 9:03 PM Subject: Re: [boost] Futures - Reviews Needed (January 5, 2009) On Monday 19 January 2009 13:19, Johan Torp wrote:
Vicente Botet Escriba wrote:
I would not be surprised if requiring a third thread for future composition would render it useless for many of Gaskill's use cases, libpoet and asio - which means most of the identified use cases. But I'm far from sure.
Could you or someone develop these needs.
The authors of asio, libpoet and Gaskill are best suited to answer if Anthony's proposal would be useful to them.
Unfortunately, I haven't had time to follow the future review threads in detail, or any changes that may have been made to the submitted library since I last looked at them many months ago. I can only summarize some things about the future implementation in libpoet in the hope it will be helpful in some way.
As far as composing futures, the poet::future_combining_barrier supports obtaining a new future whose value is obtained by applying an arbitrary "combiner" functor to the values from a group of input futures. The combiner is always executed in a future-waiting thread, not the promise-fulfilling thread, or a third thread. The implementation relies on internal signals being emitted immediately when a promise is fulfilled, as well as internal event queues associated with futures. I described it a little in this post:
http://lists.boost.org/Archives/boost/2008/06/138422.php
With respect to an O(1) future wait_for_any, I added poet::future_selector, which is like a future queue which reorders based on the order that the futures pushed into the queue become ready.
I was able to use them to reimplement my schedulers for active objects, without the scheduler using "future is ready" callbacks directly. To be more clear about my use case: I have a thread which executes method requests, and each method request has a group of futures associated with it. The thread's scheduler has a queue of method requests and it selects a method request to run when all the method request's futures are ready.
Hi, I have 3 questions, Can this poet::future_combining_barrier (function?) , poet::future_selector be implemented with the current proposals? If yes there is no issue. If no, what is missing in the current interface to implement them? I see on the web that libPoet have its onw mutexes, exception_ptr, futures, scheduler and much more ... Is libPoet a candidate to be included in Boost or a stand alone library? Do you plan to propose libPoet or part of it to Boost? What about doing a review? It is not too late. Best, Vicente
participants (9)
-
Anthony Williams -
Frank Mori Hess -
Johan Torp -
Michael Marcin -
Steve Karmesin -
Steven Watanabe -
Tom Brinkman -
Vicente Botet -
vicente.botet