[futures] Future Containers?
Hi Everyone, I've been following the future implementation proposals that are set to be available for review soon for inclusion in Boost. I for one would like to look at the implementations in their "review-ready" forms before doing an actual review of the different (competing?) implementations. However, before it goes out there, I would like to throw something into the mix. Right now, we've been discussing futures that contain individual values -- I know, these values can be containers, or practically anything that can be referred to with a pointer. That being said, I'd like to look at a broader application of future containers in the form of asynchronous random access containers. First, let me set up why I'm interested particularly in this matter. I'm looking at being able to map a function to a range and being able to asynchronously access certain parts of the container from multiple threads and perform operations on them. This marries the functional programming notion of function application, with asynchronous/parallel programming with "lazy values" -- only here, the values are actually computed in parallel or in separate threads (accessed through futures). So given for example: transform( input.begin(), input.end(), back_inserter(output), function_returns_future ); It would make 'output' be a back insertion sequence (or a container that supports .push_back) that contains futures. Now this would be fine, but now the problem becomes this: In the case of an associative container for example, we'd have a pair<key, future<value> > -- since futures cannot be default constructable in either implementation, I don't think code that does something like: map<int, future<int> > container; container[0].wait(); // dunno what happens here Will compile. Right now, having futures non-default constructable makes it hard(er) to put them in standard containers. This now begs the question: Will there be, or should there be, special classes of future containers that would specially be defined to work with futures? For example, in associative containers we should be able to access the elements using the index operator without having to explicitly call 'wait()', or perhaps have special future container iterators that allow for waiting on a future when dereferencing "pointed to elements". Is there a thrust towards developing a future-aware class of containers parallel to the STL containers? I personally think this will be a game changer in helping people transition towards and actually maximizing futures. I'd love to hear your thoughts on this. Have a good day everyone. -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday 18 May 2008 22:40 pm, Dean Michael Berris wrote:
Right now, having futures non-default constructable makes it hard(er) to put them in standard containers.
I believe futures should be default constructible. The William's future already have an "empty" state, which can be reached by moving a future. I made poet::future default constructible so they could be used in containers, and made default constructed futures throw an exception if someone attempts to wait on them.
This now begs the question: Will there be, or should there be, special classes of future containers that would specially be defined to work with futures? For example, in associative containers we should be able to access the elements using the index operator without having to explicitly call 'wait()', or perhaps have special future container iterators that allow for waiting on a future when dereferencing "pointed to elements".
Would this add anything besides automatically waiting for futures to become ready when they are accessed? Automatic waiting is the reason some people don't like implicit conversion of futures to values (I actually don't like it so much myself any more). They want points in the program where a future might block to be as explicit as possible. - -- Frank -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIMYJF5vihyNWuA4URAm1iAKDLGl1BgwrALTtD8i4yx9F6el2YfwCglpFC 3h1Z3fdxDhPoSXGnD2oj1b4= =kwL2 -----END PGP SIGNATURE-----
On Mon, May 19, 2008 at 9:36 PM, Frank Mori Hess <frank.hess@nist.gov> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Sunday 18 May 2008 22:40 pm, Dean Michael Berris wrote:
Right now, having futures non-default constructable makes it hard(er) to put them in standard containers.
I believe futures should be default constructible. The William's future already have an "empty" state, which can be reached by moving a future. I made poet::future default constructible so they could be used in containers, and made default constructed futures throw an exception if someone attempts to wait on them.
This sounds like a good idea, waiting on a future that doesn't have an associated promise should throw.
This now begs the question: Will there be, or should there be, special classes of future containers that would specially be defined to work with futures? For example, in associative containers we should be able to access the elements using the index operator without having to explicitly call 'wait()', or perhaps have special future container iterators that allow for waiting on a future when dereferencing "pointed to elements".
Would this add anything besides automatically waiting for futures to become ready when they are accessed? Automatic waiting is the reason some people don't like implicit conversion of futures to values (I actually don't like it so much myself any more). They want points in the program where a future might block to be as explicit as possible.
For the special container types I've been pondering, aside from automatically waiting on contained futures there's the possibility of making standard algorithms more specialized for these container types (or via tag dispatch). Perhaps having a special std::transform() implementation (the five argument version) that does a 'wait on any' future in the container and in a non-deterministic order apply a function on the contained future value that becomes available, and the result mapped to the appropriate location in the resulting container. -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday 19 May 2008 22:22 pm, Dean Michael Berris wrote:
For the special container types I've been pondering, aside from automatically waiting on contained futures there's the possibility of making standard algorithms more specialized for these container types (or via tag dispatch).
Perhaps having a special std::transform() implementation (the five argument version) that does a 'wait on any' future in the container and in a non-deterministic order apply a function on the contained future value that becomes available, and the result mapped to the appropriate location in the resulting container.
That's interesting. I can do that with ordinary containers/std::transform by using a poet::active_function from libpoet as the binary function passed to the transform algorithm. The active function's scheduler performs each add operation as the futures become ready. Example code attached. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIMtvK5vihyNWuA4URApfIAJ0Uc9x8MJZ/mEubT12aKzmTVE5Y6wCg4Eyu iuYOLdFnfltTe3nnoYWd4m0= =r7As -----END PGP SIGNATURE-----
"Dean Michael Berris" <mikhailberis@gmail.com> writes:
Right now, having futures non-default constructable makes it hard(er) to put them in standard containers.
The futures in my proposal can be default-constructed. unique_future requires a move-aware container, but shared_future should be usable in any container. Anthony -- Anthony Williams | Just Software Solutions Ltd Custom Software Development | http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday 19 May 2008 10:05 am, Anthony Williams wrote:
"Dean Michael Berris" <mikhailberis@gmail.com> writes:
Right now, having futures non-default constructable makes it hard(er) to put them in standard containers.
The futures in my proposal can be default-constructed. unique_future requires a move-aware container, but shared_future should be usable in any container.
Turns out the Gaskill future is also default constructible, according to its docs. Where did the idea they neither were default constructible come from? - -- Frank -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIMYv65vihyNWuA4URAqVoAJwMyYOckvqmtaCbDTIQRXEOuVhRSgCgxuys LzXQ1RulJ7LvlnB5r4B1lmo= =7t3C -----END PGP SIGNATURE-----
On Mon, May 19, 2008 at 10:17 PM, Frank Mori Hess <frank.hess@nist.gov> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Monday 19 May 2008 10:05 am, Anthony Williams wrote:
"Dean Michael Berris" <mikhailberis@gmail.com> writes:
Right now, having futures non-default constructable makes it hard(er) to put them in standard containers.
The futures in my proposal can be default-constructed. unique_future requires a move-aware container, but shared_future should be usable in any container.
Turns out the Gaskill future is also default constructible, according to its docs. Where did the idea they neither were default constructible come from?
My bad. I was thinking more about the "validity" of default constructed futures that don't have an associated promise. I was under the impression that there wasn't a way to create a future without an existing promise object. So something like: future<int>(); Would introduce an either invalid future whose value could not ever be set through an associated promise. -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday 19 May 2008 22:26 pm, Dean Michael Berris wrote:
My bad. I was thinking more about the "validity" of default constructed futures that don't have an associated promise. I was under the impression that there wasn't a way to create a future without an existing promise object. So something like:
future<int>();
Would introduce an either invalid future whose value could not ever be set through an associated promise.
Futures are assignable, so a default constructed future could later be assigned a future that does have an associated promise. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIMtME5vihyNWuA4URAgTkAJ403TL2L+1Lu3tMdWfcpQ5fzt5lOACfc2yK LXZZ2iPdpmDBKeisWaIS7bY= =efEN -----END PGP SIGNATURE-----
Anthony Williams wrote:
"Dean Michael Berris" <mikhailberis@gmail.com> writes:
Right now, having futures non-default constructable makes it hard(er) to put them in standard containers.
The futures in my proposal can be default-constructed. unique_future requires a move-aware container, but shared_future should be usable in any container.
Anthony
Just to add a little bit more information. When move semantics emulation are unified, Interprocess containers (their default allocator is std::allocator<T>) could be used to store unique futures. This would include map/set, family flat_map/set family, list, vector and deque. boost::unordered_map would also be compatible with non-copyable types. Regards, Ion
participants (4)
-
Anthony Williams -
Dean Michael Berris -
Frank Mori Hess -
Ion Gaztañaga