Re: [boost] [gsoc-2013] Boost.Thread/ThreadPool project
I'm not ruling out the merit of a Boost.Thread threadpool. For me personally though, the hurdle is very significantly raised: you need to strongly prove to me why your proposed thread pool is much superior to a Boost.ASIO based threadpool. Otherwise I will zero score the GSoC proposal, because in the end we need other proposed projects more. This is your right, but I don't know how the fact you can write easily the thread pool you need to diminish a generic thread pool working well with futures. [snip] Hopping this helps you to understand the project I'm looking for.
There are many ways systemically of implementing futures. The way you and Anthony have taken to date is a non-intrusive approach such that futures stand alone. This is great for not forcing design onto third parties and is a decentralised POSIX-y approach, but it comes with significant performance and memory costs because it's hard to get standalone, non-intrusive futures to batch themselves without sacrificing performance. An alternative approach is to assume a central dispatcher/event loop which can optimize then() chaining because it knows of all futures and their relationships currently in flight (this is a much more centralized statist NT-y approach, and it you can see that philosophy clearly in Microsoft's N3428). You can see an example of this approach to implementing future chaining with multiple dependencies which extends Boost.ASIO in https://github.com/ned14/TripleGit/blob/master/triplegit/include/async_file_ io.hpp#L785. I also went ahead and implemented N3428's when_all() which returns a future chained to multiple input futures, not that I claim the implementation mature yet. My concern with Boost.ThreadPool right now is that it could get in the way of the latter approach. I have zero objection to a formulation which embraces both approaches, but when you ask why a thread pool isn't in the standard, I think it's because until we've fixed futures implementation and all the composibility and chaining we're sure we need, and made sure that interops smoothly with the forthcoming TR2 networking proposal, then and only then is it wise to proceed with threadpool design. All that said, this is purely a cautious opinion of mine. You're one of the Boost.Thread maintainers, so I happily defer to your judgment on this. If you think we need a threadpool now, you'll find no opposition from me. Niall --- Opinions expressed here are my own and do not necessarily represent those of BlackBerry Inc.
Niall Douglas-2 wrote
I'm not ruling out the merit of a Boost.Thread threadpool. For me personally though, the hurdle is very significantly raised: you need to strongly prove to me why your proposed thread pool is much superior to a Boost.ASIO based threadpool. Otherwise I will zero score the GSoC proposal, because in the end we need other proposed projects more. This is your right, but I don't know how the fact you can write easily the thread pool you need to diminish a generic thread pool working well with futures. [snip] Hopping this helps you to understand the project I'm looking for.
There are many ways systemically of implementing futures. The way you and Anthony have taken to date is a non-intrusive approach such that futures stand alone. This is great for not forcing design onto third parties and is a decentralised POSIX-y approach, but it comes with significant performance and memory costs because it's hard to get standalone, non-intrusive futures to batch themselves without sacrificing performance.
I have no merit here. This is the approach the standard has defined futures.
An alternative approach is to assume a central dispatcher/event loop which can optimize then() chaining because it knows of all futures and their relationships currently in flight (this is a much more centralized statist NT-y approach, and it you can see that philosophy clearly in Microsoft's N3428). You can see an example of this approach to implementing future chaining with multiple dependencies which extends Boost.ASIO in https://github.com/ned14/TripleGit/blob/master/triplegit/include/async_file_ io.hpp#L785. I also went ahead and implemented N3428's when_all() which returns a future chained to multiple input futures, not that I claim the implementation mature yet.
I don't know ASIO enough to understand the advantages or liabilities. While a central dispatcher/event loop could be useful for some applications other need more concurrency. I'll need to take a deep look at ASIO when I would have enough time.
My concern with Boost.ThreadPool right now is that it could get in the way of the latter approach. I have zero objection to a formulation which embraces both approaches, but when you ask why a thread pool isn't in the standard, I think it's because until we've fixed futures implementation and all the composibility and chaining we're sure we need, and made sure that interops smoothly with the forthcoming TR2 networking proposal, then and only then is it wise to proceed with threadpool design.
I'm not aware of the TR2 networking proposal needs respect to futures or thread pool. Again I'll take a look to the current proposals.
All that said, this is purely a cautious opinion of mine. You're one of the Boost.Thread maintainers, so I happily defer to your judgment on this. If you think we need a threadpool now, you'll find no opposition from me.
If IIUC you are not suggesting that the student uses the ASIO approach, but that it is not time to solve the thread_pool now. Could you confirm? If yes, could you tell me what is blocking that we can not try to solve? Best, Vicente -- View this message in context: http://boost.2283326.n4.nabble.com/gsoc-2013-Boost-Thread-ThreadPool-project... Sent from the Boost - Dev mailing list archive at Nabble.com.
Le 29/04/13 17:40, Niall Douglas a écrit :
I'm not ruling out the merit of a Boost.Thread threadpool. For me personally though, the hurdle is very significantly raised: you need to strongly prove to me why your proposed thread pool is much superior to a Boost.ASIO based threadpool. Otherwise I will zero score the GSoC proposal, because in the end we need other proposed projects more. This is your right, but I don't know how the fact you can write easily the thread pool you need to diminish a generic thread pool working well with futures. [snip] Hopping this helps you to understand the project I'm looking for. There are many ways systemically of implementing futures. The way you and Anthony have taken to date is a non-intrusive approach such that futures stand alone. This is great for not forcing design onto third parties and is a decentralised POSIX-y approach, but it comes with significant performance and memory costs because it's hard to get standalone, non-intrusive futures to batch themselves without sacrificing performance.
An alternative approach is to assume a central dispatcher/event loop which can optimize then() chaining because it knows of all futures and their relationships currently in flight (this is a much more centralized statist NT-y approach, and it you can see that philosophy clearly in Microsoft's N3428). You can see an example of this approach to implementing future chaining with multiple dependencies which extends Boost.ASIO in https://github.com/ned14/TripleGit/blob/master/triplegit/include/async_file_ io.hpp#L785. I also went ahead and implemented N3428's when_all() which returns a future chained to multiple input futures, not that I claim the implementation mature yet.
My concern with Boost.ThreadPool right now is that it could get in the way of the latter approach. I have zero objection to a formulation which embraces both approaches, but when you ask why a thread pool isn't in the standard, I think it's because until we've fixed futures implementation and all the composibility and chaining we're sure we need, and made sure that interops smoothly with the forthcoming TR2 networking proposal, then and only then is it wise to proceed with threadpool design.
All that said, this is purely a cautious opinion of mine. You're one of the Boost.Thread maintainers, so I happily defer to your judgment on this. If you think we need a threadpool now, you'll find no opposition from me.
Hi again, I was not aware that there was a AFSIO GSoC project as described here https://svn.boost.org/trac/boost/wiki/SoC2013#Boost.ASIO and that this project propose also thread pool. I understand better now your concerns. I have no problem at all if there is only student that works on a thread pool proposal, but I think the approaches are quite different and so it will be worth to have students fr both projects. Nest, Vicente
participants (3)
-
Niall Douglas -
Vicente Botet -
Vicente J. Botet Escriba