Hi Niall The ASIO development started some work on working with 'registered buffers' using lib io_uring (on Linux). Windows RIO support need to ensure some thread semantics that must be checked if they are compatible with ASIO model. For standard IOCP mode, WSASendMsg is analog to sendmsg, and it's already implemented on ASIO. My patch was develop because I need to implement a high-volume UDP service and I reached the limit of sendto/recvfrom calls/CPU core/second on Linux (about 35k calls/datagrams on my box). With sendmmsg / recvmmsg, I easily reaches 200k datagrams packets/second perform about 6k sendmmsg/recvmmsg calls on same hardware). Using SO_REUSEPORT, with 12 sockets, without sendmmsg / recvmmsg, I reached about 190k datagrams/s using 13 cores. Using 1 receive and 5 senders, reached 1 million datagrams / second (I've reached a peak 12.5 gigabit/second transfers, without any packet loss, using 64MB send/receive buffers). To reach this kind of performance, very specialized programs are needed (busy pooling udp sockets, pin each socket process to a cpu core, also pin RSS-queue to same CPU core (you can read some material on that on Cloudfare's blog https://blog.cloudflare.com/how-to-receive-a-million-packets ... it's not a trivial task). The recvmmsg / sendmmsg also suppots TCP on some OS's, so it raised the of I/O performance to another level. On Mon, Jan 16, 2023 at 12:17 PM Niall Douglas via Boost < boost@lists.boost.org> wrote:
On 14/01/2023 18:41, Virgilio Fornazin via Boost wrote:
I finished the first version of the patch. It's able to send and receive 1million packet / second on my box at same time (an intel i9 9900k / ubuntu 22.04 lts low latency kernel) using packet sizes of 64 / 1430 bytes without packet loss (using 64mb udp socket buffers)
A good example of why support for one million scatter-gather buffer lists is a useful thing.
Did you consider implementing the same support for Windows? It's more work than it would be on Linux as you'd need to extend the ASIO IOCP reactor with RIO, but you'd get similar performance.
Niall
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost