Using boost::asio with thousands of concurrent connections
Hi, I started to use boost::asio some time ago and want to use it for a windows commercial product that does web filtering and handles thousands of connections concurrently (proxy based product) I am wandering if anybody had experience using it at such a large scale, where besides handling many connections you have to handle inside a connection communication with databases, things that potentially incur delaying handling of subsequent requests. What kind of configuration would you advise in such a case (multiple I/O services per n threads, on IO service that handles all incoming connections and a thread pool that handles processing of async handlers - how one would assure in this case a connections is handled in the same thread then ..etc) Thanks for any input -Ghira
On 2/9/2012 1:04 PM, Gheorghe Marinca wrote:
Hi,
I started to use boost::asio some time ago and want to use it for a windows commercial product that does web filtering and handles thousands of connections concurrently (proxy based product)
I am wandering if anybody had experience using it at such a large scale, where besides handling many connections you have to handle inside a connection communication with databases, things that potentially incur delaying handling of subsequent requests. What kind of configuration would you advise in such a case (multiple I/O services per n threads, on IO service that handles all incoming connections and a thread pool that handles processing of async handlers - how one would assure in this case a connections is handled in the same thread then ..etc)
Thanks for any input -Ghira
I haven't used boost::asio yet, so I can't speak for its capabilities, but on Windows I/O completion ports are definitely the way to go for handling large numbers of connections. You basically set up a pool of worker threads that the OS automatically cycles through whenever there is any socket activity waiting to be handled. The pool is shared among all connections and not divided into any specific grouping, such as n connections per thread. -- Bill
On Thu, Feb 9, 2012 at 1:04 PM, Gheorghe Marinca
Hi,
I started to use boost::asio some time ago and want to use it for a windows commercial product that does web filtering and handles thousands of connections concurrently (proxy based product)
I am wandering if anybody had experience using it at such a large scale, where besides handling many connections you have to handle inside a connection communication with databases, things that potentially incur delaying handling of subsequent requests. What kind of configuration would you advise in such a case (multiple I/O services per n threads, on IO service that handles all incoming connections and a thread pool that handles processing of async handlers - how one would assure in this case a connections is handled in the same thread then ..etc)
Asio uses an I/O completion port on Windows, which is designed to scale with multi-threading. This will probably be the simplest option, so try there first. There is some contention and always a possibility that completions will come back on a different thread than what started the request, though, so you may find better performance using one io_service (and thus one I/O completion port) per thread if you've got enough connections that automatic load balancing isn't important. As always you can never really know until you write some code and benchmark, but that has been my experience. Databases are a tricky thing -- none that I've seen have a satisfactory async API, if any at all. I would create additional dedicated threads to handle such things. If you go the one io_service per thread route, using SetThreadIdealProcessor to group the I/O thread and database threads together will help avoid sharing. Good luck! -- Cory Nelson http://int64.org
participants (3)
-
Bill Buklis -
Cory Nelson -
Gheorghe Marinca