[Boost.ASIO] [asio] Scaling TCP Server
I have created a TCP server that can receive two types of message from client. 1. "Heartbeat" : If this is found and "first_fetch" is 1, send some config in Json format else send {"config_changed" : "true"} 2. "Message" : If this is found, send {"success" : "true"} The async server is here => https://gist.github.com/mesarvagya/a73f2428417cb10336ef and the client is given here => https://gist.github.com/mesarvagya/90d764c749d18d9c5184 While both works fine, but the server is slower than expected. If one sees, I have used ~3kb message in client. When the client and server are connected, I did a preliminary benchmarking and the result were as follows: 1. For 50K messages, the time was 6 seconds. 2. For 100K messages, the time was 11 seconds. 3. For 200K messages, the time was 36 seconds. 3. For 500K messages, the time was 82 seconds 3. For 1million messages, the time was 174 seconds. The server is not working as expected. Currently it works at ~6K message/seconds in localhost. I would like to optimize the current server to receive at-least 100K messages per second. How should I begin with optimizing the server code? Is it possible to receive such high message/seconds in boost.ASIO and if so, how should I approach with the server design? -- *Sarvagya Pant* *Kathmandu, Nepal* *+9779803468257*
Hi,
I have created a TCP server that can receive two types of message from client. 1. "Heartbeat" : If this is found and "first_fetch" is 1, send some config in Json format else send {"config_changed" : "true"} 2. "Message" : If this is found, send {"success" : "true"}
The async server is here => https://gist.github.com/mesarvagya/a73f2428417cb10336ef and the client is given here => https://gist.github.com/mesarvagya/90d764c749d18d9c5184
While both works fine, but the server is slower than expected. If one sees, I have used ~3kb message in client. When the client and server are connected, I did a preliminary benchmarking and the result were as follows:
1. For 50K messages, the time was 6 seconds. 2. For 100K messages, the time was 11 seconds. 3. For 200K messages, the time was 36 seconds. 3. For 500K messages, the time was 82 seconds 3. For 1million messages, the time was 174 seconds.
The server is not working as expected. Currently it works at ~6K message/seconds in localhost. I would like to optimize the current server to receive at-least 100K messages per second. How should I begin with optimizing the server code? Is it possible to receive such high message/seconds in boost.ASIO and if so, how should I approach with the server design? What platform are you testing? The localhost adapter implementation might be a limiting factor on some platforms. Can you try to use smaller packet sizes for comparison?
Have you measured pure throughput by not handling the messages but simply receiving them? I would recommend to profile your code for bottlenecks, such as parsing the strings etc. Cheers Sebastian
-- /Sarvagya Pant / /Kathmandu, Nepal/ /+9779803468257/
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Hello Sebastian, Thank you for your response. I have tested the server in Windows (Visual studio 2010 and O2 optimization). I am trying to model the server that will receive a JSON message from client. Upon receiving the message as JSON, I should be adding some fields to it and the message received will be then sent to some upper layer. The maximum size of message received from client could be 4Kb in size. How should I handle the scenario if I want to handle thousands of message/second? Thank you once again. On Tue, Aug 11, 2015 at 6:42 PM, Sebastian Messerschmidt < sebastian.messerschmidt@gmx.de> wrote:
Hi,
I have created a TCP server that can receive two types of message from client. 1. "Heartbeat" : If this is found and "first_fetch" is 1, send some config in Json format else send {"config_changed" : "true"} 2. "Message" : If this is found, send {"success" : "true"}
The async server is here => https://gist.github.com/mesarvagya/a73f2428417cb10336ef https://gist.github.com/mesarvagya/a73f2428417cb10336ef and the client is given here => https://gist.github.com/mesarvagya/90d764c749d18d9c5184 https://gist.github.com/mesarvagya/90d764c749d18d9c5184
While both works fine, but the server is slower than expected. If one sees, I have used ~3kb message in client. When the client and server are connected, I did a preliminary benchmarking and the result were as follows:
1. For 50K messages, the time was 6 seconds. 2. For 100K messages, the time was 11 seconds. 3. For 200K messages, the time was 36 seconds. 3. For 500K messages, the time was 82 seconds 3. For 1million messages, the time was 174 seconds.
The server is not working as expected. Currently it works at ~6K message/seconds in localhost. I would like to optimize the current server to receive at-least 100K messages per second. How should I begin with optimizing the server code? Is it possible to receive such high message/seconds in boost.ASIO and if so, how should I approach with the server design?
What platform are you testing? The localhost adapter implementation might be a limiting factor on some platforms. Can you try to use smaller packet sizes for comparison?
Have you measured pure throughput by not handling the messages but simply receiving them? I would recommend to profile your code for bottlenecks, such as parsing the strings etc.
Cheers Sebastian
--
*Sarvagya Pant * *Kathmandu, Nepal* *+9779803468257 <%2B9779803468257>*
_______________________________________________ Boost-users mailing listBoost-users@lists.boost.orghttp://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-- *Sarvagya Pant* *Kathmandu, Nepal* *+9779803468257*
Hello Sebastian, Thank you for your response. I have tested the server in Windows (Visual studio 2010 and O2 optimization). I am trying to model the server that will receive a JSON message from client. Upon receiving the message as JSON, I should be adding some fields to it and the message received will be then sent to some upper layer. The maximum size of message received from client could be 4Kb in size. How should I handle the scenario if I want to handle thousands of message/second? Now, that totally depends on where your performance bottleneck is ... I've given you some hints regarding message size, handling/parsing which should help to identify your bottleneck. ->small packets to see if this impacts throughput ->remove any handling of the received data to see if this improves
throughput significantly ->check if the client is limiting the throughput by using multiple clients ... Later you could move message decoding/handling to separate worker thread/worker threads with some queue between the receiving and handling parts. But first of all you should measure where your program spends its time before blindly implementing anything. Cheers Sebastian
Thank you once again.
On Tue, Aug 11, 2015 at 6:42 PM, Sebastian Messerschmidt
mailto:sebastian.messerschmidt@gmx.de> wrote: Hi,
I have created a TCP server that can receive two types of message from client. 1. "Heartbeat" : If this is found and "first_fetch" is 1, send some config in Json format else send {"config_changed" : "true"} 2. "Message" : If this is found, send {"success" : "true"}
The async server is here => https://gist.github.com/mesarvagya/a73f2428417cb10336ef and the client is given here => https://gist.github.com/mesarvagya/90d764c749d18d9c5184
While both works fine, but the server is slower than expected. If one sees, I have used ~3kb message in client. When the client and server are connected, I did a preliminary benchmarking and the result were as follows:
1. For 50K messages, the time was 6 seconds. 2. For 100K messages, the time was 11 seconds. 3. For 200K messages, the time was 36 seconds. 3. For 500K messages, the time was 82 seconds 3. For 1million messages, the time was 174 seconds.
The server is not working as expected. Currently it works at ~6K message/seconds in localhost. I would like to optimize the current server to receive at-least 100K messages per second. How should I begin with optimizing the server code? Is it possible to receive such high message/seconds in boost.ASIO and if so, how should I approach with the server design?
What platform are you testing? The localhost adapter implementation might be a limiting factor on some platforms. Can you try to use smaller packet sizes for comparison?
Have you measured pure throughput by not handling the messages but simply receiving them? I would recommend to profile your code for bottlenecks, such as parsing the strings etc.
Cheers Sebastian
-- /Sarvagya Pant / /Kathmandu, Nepal/ /+9779803468257 tel:%2B9779803468257/
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org mailto:Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org mailto:Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-- /Sarvagya Pant / /Kathmandu, Nepal/ /+9779803468257/
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On 08/11/2015 03:27 PM, Sarvagya Pant wrote:
I have created a TCP server that can receive two types of message from client. 1. "Heartbeat" : If this is found and "first_fetch" is 1, send some config in Json format else send {"config_changed" : "true"} 2. "Message" : If this is found, send {"success" : "true"}
The async server is here => https://gist.github.com/mesarvagya/a73f2428417cb10336ef and the client is given here => https://gist.github.com/mesarvagya/90d764c749d18d9c5184
While both works fine, but the server is slower than expected. If one sees, I have used ~3kb message in client. When the client and server are connected, I did a preliminary benchmarking and the result were as follows:
1. For 50K messages, the time was 6 seconds. 2. For 100K messages, the time was 11 seconds. 3. For 200K messages, the time was 36 seconds. 3. For 500K messages, the time was 82 seconds 3. For 1million messages, the time was 174 seconds.
The server is not working as expected. Currently it works at ~6K message/seconds in localhost. I would like to optimize the current server to receive at-least 100K messages per second. How should I begin with optimizing the server code? Is it possible to receive such high message/seconds in boost.ASIO and if so, how should I approach with the server design?
The measurements suggest quadratic behavior. Try plotting it on a graph. I'd guess you have a resource leak, plus you are traversing the leaked resource, which results in both slowness and the slowdown with increased number of messages. Try profiling the server (or the client).
On 11 Aug 2015 at 18:12, Sarvagya Pant wrote:
The server is not working as expected. Currently it works at ~6K message/seconds in localhost. I would like to optimize the current server to receive at-least 100K messages per second. How should I begin with optimizing the server code? Is it possible to receive such high message/seconds in boost.ASIO and if so, how should I approach with the server design?
I assume Nagle's algorithm is turned off? Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
participants (4)
-
Avi Kivity
-
Niall Douglas
-
Sarvagya Pant
-
Sebastian Messerschmidt