Asio and software interrupts
Hello, I’m use the Boost::Asio libraries to read data from multicast sockets. This is a high rate application that needs to listen to data coming on a large number of multicast addresses (up to 500 different addresses). I would like to funnel all the multicast traffic to a single class where the data can get handled very quickly. I have the code working now using the socket->async_receive_from() method but the system is getting bogged down handling a large number of software interrupts. 51% of the CPU time is spent handling software interrupts. I create each multicast socket (up to 500 of them) like so: socket = new udp::socket(io_service); socket->open(listenEndPoint.protocol()); socket->set_option(boost::asio::ip::udp::socket::resuse_address(true)); socket->bind(listenEndPoint); socket->set_option(boost::asio::ip::multicast::join_group(multicastAddress.to_v4())); Slightly after that I register for data like so: socket->async_receive_from(buffer,senderEndPoint,boost::bind(&EthernetCapture::receiveData,this,boost::asio::placeholders::bytes_transferred)); Is there a more efficient way to receive multicast data without incurring so many software interrupts? By the way this is for an application to run on 32 bit Linux for an embedded application. Thanks
Hi, I don't think the softirqs can be avoided, since the network driver dispatches each packet it receives to the network layer (which causes a software interrupt). Now if you listen to 500+ multicast addresses and given the limited resources available on the embedded platform (What platform is it actually? And what performance does it offer?) I would certainly expect a lot of time to be spent simply for handling the network packets in the kernel. How did you determine the 51% spent in softirqs? Did you watch the ksoftirq process? Regards, Andreas On 11/28/2012 10:34 PM, Joseph Sulewski wrote:
Hello,
I’m use the Boost::Asio libraries to read data from multicast sockets. This is a high rate application that needs to listen to data coming on a large number of multicast addresses (up to 500 different addresses). I would like to funnel all the multicast traffic to a single class where the data can get handled very quickly.
I have the code working now using the socket->async_receive_from() method but the system is getting bogged down handling a large number of software interrupts. 51% of the CPU time is spent handling software interrupts.
I create each multicast socket (up to 500 of them) like so:
socket = new udp::socket(io_service);
socket->open(listenEndPoint.protocol());
socket->set_option(boost::asio::ip::udp::socket::resuse_address(true));
socket->bind(listenEndPoint);
socket->set_option(boost::asio::ip::multicast::join_group(multicastAddress.to_v4()));
Slightly after that I register for data like so:
socket->async_receive_from(buffer,senderEndPoint,boost::bind(&EthernetCapture::receiveData,this,boost::asio::placeholders::bytes_transferred));
Is there a more efficient way to receive multicast data without incurring so many software interrupts?
By the way this is for an application to run on 32 bit Linux for an embedded application.
Thanks
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Mit freundlichen Grüßen / Best regards Andreas Wehrmann CENTER COMMUNICATION SYSTEMS GMBH Ein Unternehmen der STRABAG AG Software Development Ignaz-Köck-Str. 19 A-1210 Wien, Österreich Tel.: +43 (0) 190 199 - 3616 Fax: +43 (0) 190 199 - 2110 Mobil: +43 (0) 664 884 75916 a.wehrmann@centersystems.com FN 796 88p, Sitz in Wien Firmenbuchgericht Wien http://www.centersystems.com/ www.centersystems.com Geschäftsführung: Ing. Gerhard Jelinek, Josef-Eduard Burger
Andreas, Thank you for your response and I apologize for the delayed reply. The top application in Linux shows how much time is spent handling software interrupts. Is there a more efficient way of listening to the sockets in boost? Can we create a socket that registers for more than one multicast address? Would that even help? If not, perhaps there is some kernel or driver tuning that can reduce the number of interrupts. Thanks for any help or pointers, On Thu, Nov 29, 2012 at 3:57 AM, Andreas Wehrmann < a.wehrmann@centersystems.com> wrote:
Hi,
I don't think the softirqs can be avoided, since the network driver dispatches each packet it receives to the network layer (which causes a software interrupt). Now if you listen to 500+ multicast addresses and given the limited resources available on the embedded platform (What platform is it actually? And what performance does it offer?) I would certainly expect a lot of time to be spent simply for handling the network packets in the kernel.
How did you determine the 51% spent in softirqs? Did you watch the ksoftirq process?
Regards,
Andreas
On 11/28/2012 10:34 PM, Joseph Sulewski wrote:
Hello,
I’m use the Boost::Asio libraries to read data from multicast sockets. This is a high rate application that needs to listen to data coming on a large number of multicast addresses (up to 500 different addresses). I would like to funnel all the multicast traffic to a single class where the data can get handled very quickly.
I have the code working now using the socket->async_receive_from() method but the system is getting bogged down handling a large number of software interrupts. 51% of the CPU time is spent handling software interrupts.
I create each multicast socket (up to 500 of them) like so:
socket = new udp::socket(io_service);
socket->open(listenEndPoint.**protocol());
socket->set_option(boost::**asio::ip::udp::socket::resuse_** address(true));
socket->bind(listenEndPoint);
socket->set_option(boost::**asio::ip::multicast::join_** group(multicastAddress.to_v4()**));
Slightly after that I register for data like so:
socket->async_receive_from(**buffer,senderEndPoint,boost::** bind(&EthernetCapture::**receiveData,this,boost::asio::** placeholders::bytes_**transferred));
Is there a more efficient way to receive multicast data without incurring so many software interrupts?
By the way this is for an application to run on 32 bit Linux for an embedded application.
Thanks
______________________________**_________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/**mailman/listinfo.cgi/boost-**usershttp://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Mit freundlichen Grüßen / Best regards
Andreas Wehrmann
CENTER COMMUNICATION SYSTEMS GMBH
Ein Unternehmen der STRABAG AG
Software Development
Ignaz-Köck-Str. 19 A-1210 Wien, Österreich
Tel.: +43 (0) 190 199 - 3616 Fax: +43 (0) 190 199 - 2110 Mobil: +43 (0) 664 884 75916
a.wehrmann@centersystems.com
FN 796 88p, Sitz in Wien Firmenbuchgericht Wien
http://www.centersystems.com/** www.centersystems.com
Geschäftsführung: Ing. Gerhard Jelinek, Josef-Eduard Burger ______________________________**_________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/**mailman/listinfo.cgi/boost-**usershttp://lists.boost.org/mailman/listinfo.cgi/boost-users
On 3 December 2012 14:26, Joseph Sulewski
Can we create a socket that registers for more than one multicast address?
Yes, you can join > 1 multicast address on a socket. for (auto& multicast_address : addresses) socket.set_option(boost::asio::ip::multicast::join_group(multicast_address));
On 12/03/2012 05:34 AM, Steve Lorimer wrote:
On 3 December 2012 14:26, Joseph Sulewski
mailto:joemacher@gmail.com> wrote: Can we create a socket that registers for more than one multicast address?
Yes, you can join > 1 multicast address on a socket.
for (auto& multicast_address : addresses)
socket.set_option(boost::asio::ip::multicast::join_group(multicast_address));
@Joseph: If you try this, let us know what difference it made (if any). -- Mit freundlichen Grüßen / Best regards Andreas Wehrmann CENTER COMMUNICATION SYSTEMS GMBH Ein Unternehmen der STRABAG AG Software Development Ignaz-Köck-Str. 19 A-1210 Wien, Österreich Tel.: +43 (0) 190 199 - 3616 Fax: +43 (0) 190 199 - 2110 Mobil: +43 (0) 664 884 75916 a.wehrmann@centersystems.com FN 796 88p, Sitz in Wien Firmenbuchgericht Wien http://www.centersystems.com/ www.centersystems.com Geschäftsführung: Ing. Gerhard Jelinek, Josef-Eduard Burger
Thanks guys I'll give this a try. Two questions.
When I call socket->bind I provide it with an endpoint. When I create the
endpoint I'm currently using the multicast address, what should I use for
the address? If I'm binding to multiple multicast addresses it doesn't
make sense to bind to one of them.
Finally, it appears I can join multiple multicast groups but they need to
be the same port. Is this correct?
Thanks again
On Dec 3, 2012 2:13 AM, "Andreas Wehrmann"
On 12/03/2012 05:34 AM, Steve Lorimer wrote:
On 3 December 2012 14:26, Joseph Sulewski
mailto:joemacher@gmail.com> wrote: Can we create a socket that registers for more than one multicast address?
Yes, you can join > 1 multicast address on a socket.
for (auto& multicast_address : addresses)
socket.set_option(boost::asio:**:ip::multicast::join_group(** multicast_address));
@Joseph: If you try this, let us know what difference it made (if any).
-- Mit freundlichen Grüßen / Best regards
Andreas Wehrmann
CENTER COMMUNICATION SYSTEMS GMBH
Ein Unternehmen der STRABAG AG
Software Development
Ignaz-Köck-Str. 19 A-1210 Wien, Österreich
Tel.: +43 (0) 190 199 - 3616 Fax: +43 (0) 190 199 - 2110 Mobil: +43 (0) 664 884 75916
a.wehrmann@centersystems.com
FN 796 88p, Sitz in Wien Firmenbuchgericht Wien
http://www.centersystems.com/** www.centersystems.com
Geschäftsführung: Ing. Gerhard Jelinek, Josef-Eduard Burger ______________________________**_________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/**mailman/listinfo.cgi/boost-**usershttp://lists.boost.org/mailman/listinfo.cgi/boost-users
On 12/03/2012 03:51 PM, Joseph Sulewski wrote:
Thanks guys I'll give this a try. Two questions.
When I call socket->bind I provide it with an endpoint. When I create the endpoint I'm currently using the multicast address, what should I use for the address? If I'm binding to multiple multicast addresses it doesn't make sense to bind to one of them.
Finally, it appears I can join multiple multicast groups but they need to be the same port. Is this correct?
Thanks again
In socket->bind() you specify the address of the local network device you want to listen on for incoming packets, not the multicast address you want to join. You can also specify "any" address (that is: "all" interfaces) like so: socket->bind( boost::asio::ip::udp::endpoint( boost::asio::ip::udp::v4(), port_number ) ); Or use v6(), if you want an IPv6 socket. Regarding the port number: you can bind to only one port per socket. -- Mit freundlichen Grüßen / Best regards Andreas Wehrmann CENTER COMMUNICATION SYSTEMS GMBH Ein Unternehmen der STRABAG AG Software Development Ignaz-Köck-Str. 19 A-1210 Wien, Österreich Tel.: +43 (0) 190 199 - 3616 Fax: +43 (0) 190 199 - 2110 Mobil: +43 (0) 664 884 75916 a.wehrmann@centersystems.com FN 796 88p, Sitz in Wien Firmenbuchgericht Wien http://www.centersystems.com/ www.centersystems.com Geschäftsführung: Ing. Gerhard Jelinek, Josef-Eduard Burger
Thanks. I'll try to get to this tonight and see if it fixes the si
interrupts.
On Dec 3, 2012 10:04 AM, "Andreas Wehrmann"
On 12/03/2012 03:51 PM, Joseph Sulewski wrote:
Thanks guys I'll give this a try. Two questions.
When I call socket->bind I provide it with an endpoint. When I create the endpoint I'm currently using the multicast address, what should I use for the address? If I'm binding to multiple multicast addresses it doesn't make sense to bind to one of them.
Finally, it appears I can join multiple multicast groups but they need to be the same port. Is this correct?
Thanks again
In socket->bind() you specify the address of the local network device you want to listen on for incoming packets, not the multicast address you want to join.
You can also specify "any" address (that is: "all" interfaces) like so:
socket->bind( boost::asio::ip::udp::**endpoint( boost::asio::ip::udp::v4(), port_number ) );
Or use v6(), if you want an IPv6 socket.
Regarding the port number: you can bind to only one port per socket.
-- Mit freundlichen Grüßen / Best regards
Andreas Wehrmann
CENTER COMMUNICATION SYSTEMS GMBH
Ein Unternehmen der STRABAG AG
Software Development
Ignaz-Köck-Str. 19 A-1210 Wien, Österreich
Tel.: +43 (0) 190 199 - 3616 Fax: +43 (0) 190 199 - 2110 Mobil: +43 (0) 664 884 75916
a.wehrmann@centersystems.com
FN 796 88p, Sitz in Wien Firmenbuchgericht Wien
http://www.centersystems.com/** www.centersystems.com
Geschäftsführung: Ing. Gerhard Jelinek, Josef-Eduard Burger ______________________________**_________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/**mailman/listinfo.cgi/boost-**usershttp://lists.boost.org/mailman/listinfo.cgi/boost-users
On 4 December 2012 01:51, Joseph Sulewski
*When I call socket->bind I provide it with an endpoint. When I create the endpoint I'm currently using the multicast address, what should I use for the address? *
If you bind to INADDR_ANY then the OS will use its ip routing tables to work out which the best local nic to use is. You can force it to use a specific local nic by providing its address. Most applications will use INADDR_ANY *If I'm binding to multiple multicast addresses it doesn't make sense to
bind to one of them.*
Because UDP is connectionless there is no way for the kernel / network stack to know which pid to deliver datagrams to for a particular port unless you bind your socket's file descriptor to that port. (note, if you want to allow other file descriptors to be able to bind to that port you need to set SO_REUSEADDR to 1)
*Finally, it appears I can join multiple multicast groups but they need to be the same port. Is this correct?*
The reason for this is as my previous point above. What you would do is create as many sockets as there are different ports, and have each socket join the multicast addresses which are on that port
participants (3)
-
Andreas Wehrmann
-
Joseph Sulewski
-
Steve Lorimer