Re: async socket i/o [long]
Hi Michel,
Don G wrote:
I have also been putting together some ideas which are similar to those you outlined, though perhaps at a higher level of abstraction. I will try to wrap up what I have and post it in the next day or two. I will also try to look more closely at your code.
It's not that much code more interfaces ;), but implementability on at least linux, solaris, windows is a minimum.
I have implemented these interfaces (for my work) on Linx, Solaris, Windows (all versions) and Mac OS/X (and even OS 9, though we have removed that support at this point). The interfaces I am proposing don't mention socket or sockaddr or select et.al.. As a matter of fact, not all OS's even use sockets (MacOS < X doesn't if that matters). I have even implemented these interfaces as a pseudo-network over HTTP tunnels and reflection servers. About the only portability requirements that I can think of are: threads and a network access API of some kind. Not just any network API will do, but just about any system with threads should have the necessary power. There are implementations that would fit only certain platforms that may be better (epoll vs. kqueue vs. select), but a select() based version would be extremely portable for a first version. Subsequent versions could add an epoll implementation and others. I suppose that if multiple choices exist for a given platform, different concrete classes could provide the user with a choice, but see my comments below. I really do believe that the best choice for a given platform should be made (via profiler as necessary) and that should be that.
From a quick read, it looks similar to the implementation path I had used at my work, but probably more flexible in how event_dispatcher relates to event_handler. I think there is a good likelihood that the two are complementary.
Ok could you expand a bit? What do you mean by complementary?
The code I wrote for work had an implementation-side abstraction with an abstract base like event_handler, though I called it Selectable. It had a signal mask that mapped to your event_handler::event_type_t enum. The thread pool internal to the network object managed calls to select (or whatever) based on the signal mask. At least for Windows, one cannot mix sockets with other types of objects (files or whatnot). And, while I have not tried it, the docs seem to suggest that sockets for IPv4 and IPv6 may not be able to go to a single select call. All this makes matching event_handler instances to an appropriate event_dispatcher tricky and not portable. In other words, something that should be hidden from the user. The real complexity, though, comes when the number of objects gets large, in particular larger than a single call to select() can handle. This necessitated some load balancing logic to shuffle Selectable objects between threads, create new threads, cleanup extra threads, etc.. All of this is stuff that should only be written once! :) As a user of the library (again, at my work), I never have to worry about that mundane stuff. That is the job of the network library. And it scales pretty well. Though I haven't tested N*1000 simultaneous connections using drones across the network to simulate heavy CPU load, I have stress tested the library upto the limit of Windows XP Pro (not Server) and on Linux with N*1000 where N < 3, using lots of threads to generate the load. While all thread pooling and connection aggregation is managed behind the scenes, some of this could be exposed in an abstract manner to allow user code to influence the load balancing algorithm. This, I think, would be for only the very advanced user and not something to tackle initially.
1. Should the network library provide different choices of implementation to the user? For the most part, the platform will dictate the available solutions, so having the user decide would make their code non-portable. [my thought so far is no]
2. Would it be better perhaps to only provide behavior choices (sync, async, non-blocking) as part of the interface? [my thought is yes]
I think two but not providing choices just support them all automatically and efficiently. And give the user a possibilty to plugin own implementations of the concepts.
This is where our goals diverge I guess ;). I don't see any point in the user deciding these details, especially implementing their own concrete classes. And I do see it creating confusion amongst those that don't have the "Network API Guru" honor badge<g>. Because of the _long_ history of having to work directly at the socket/select/epoll layer, some of us aren't bothered by it (and some of us may even like it<g>). But this stuff is just unapproachable for anyone less than an advanced developer, is error prone at best and just plain tedious. Hard to tell how much I like select, eh? :) The interfaces I proposed provide a very clean and complete interface, IMHO<g>, and the implementation can be quite scalable (for those that need it to be) and I know it is extremely portable, even to non-socket based systems. Best regards, Don __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Don G wrote:
From a quick read, it looks similar to the implementation path I had used at my work, but probably more flexible in how event_dispatcher relates to event_handler. I think there is a good likelihood that the two are complementary.
Ok could you expand a bit? What do you mean by complementary?
The code I wrote for work had an implementation-side abstraction with an abstract base like event_handler, though I called it Selectable. It had a signal mask that mapped to your event_handler::event_type_t enum. The thread pool internal to the network object managed calls to select (or whatever) based on the signal mask.
Ok the reason I expose it is for the possibility to write single threaded reactive servers.
At least for Windows, one cannot mix sockets with other types of objects (files or whatnot). And, while I have not tried it, the docs seem to suggest that sockets for IPv4 and IPv6 may not be able to go to a single select call. All this makes matching event_handler instances to an appropriate event_dispatcher tricky and not portable. In other words, something that should be hidden from the user.
You could implement a select like replacement using WSAEventSelect and WaitForMultipleObjects to mix and match (though it would support only 64 handles, which also select does by default).
The real complexity, though, comes when the number of objects gets large, in particular larger than a single call to select() can handle. This necessitated some load balancing logic to shuffle Selectable objects between threads, create new threads, cleanup extra threads, etc.. All of this is stuff that should only be written once! :) As a user of the library (again, at my work), I never have to worry about that mundane stuff. That is the job of the network library.
Both yes and no. An implementation such as active_event_dispatcher could provide this choice and functionality, at lower layer there should be implementations that let the user of the library make the choices (and even going reactive single threaded). And eg using iocp the threading is basically done by the os (just as long as you have a pool of thread dequeing packages). For the other notification mechanisms i guess a leader follower pattern should be implemented.
And it scales pretty well. Though I haven't tested N*1000 simultaneous connections using drones across the network to simulate heavy CPU load, I have stress tested the library upto the limit of Windows XP Pro (not Server) and on Linux with N*1000 where N < 3, using lots of threads to generate the load.
While all thread pooling and connection aggregation is managed behind the scenes, some of this could be exposed in an abstract manner to allow user code to influence the load balancing algorithm. This, I think, would be for only the very advanced user and not something to tackle initially.
I think this should be added as another layer on top of of the more primitive dispatchers. Look eg at TP_Reactor from ace that uses a single threaded dispatcher and a leader follower pattern to handle events from multiple threads. This is also described in POSA2 (http://www.amazon.com/exec/obidos/ASIN/0471606952/ref%3Dsim%5Fbooks/103-8124...) . Which in general contains a lot of good patterns and advice for networked programming.
1. Should the network library provide different choices of implementation to the user? For the most part, the platform will dictate the available solutions, so having the user decide would make their code non-portable. [my thought so far is no]
2. Would it be better perhaps to only provide behavior choices (sync, async, non-blocking) as part of the interface? [my thought is yes]
I think two but not providing choices just support them all automatically and efficiently. And give the user a possibilty to plugin own implementations of the concepts.
This is where our goals diverge I guess ;). I don't see any point in the user deciding these details, especially implementing their own concrete classes. And I do see it creating confusion amongst those that don't have the "Network API Guru" honor badge<g>.
Well implementing own concrete classes should only be for the ones with badge and advanced users understanding the concepts, there arent that many implementing streambufs today either though the possibility exists, except for advanced users.
The interfaces I proposed provide a very clean and complete interface, IMHO<g>, and the implementation can be quite scalable (for those that need it to be) and I know it is extremely portable, even to non-socket based systems.
I think ut looks fairly good as well and agree whith calebs and have kind of don't like the extensive use of shared_ptr I think ownership and resource managment should be handed over to the user and not mandated by the library. /Michel
Hi Michel,
Ok the reason I expose it is for the possibility to write single threaded reactive servers.
I have written such using the library I proposed. The missing ingredient is a function queue mechanism. I posted one to the list a couple weeks ago, but feel that it is not technically a networking thing. Of course, the network layer could be coupled to it in some key places which would make things simpler to use. The downside I see with a single-threaded server is that it can only scale so far (even coded directly to select/epoll), because it can't take advantage of multiple processors. So, in my mind I didn't consider this style one on which to focus great concern for optimal performance. For high performance, IMHO, you need a multi-threaded solution.
Both yes and no. An implementation such as active_event_dispatcher could provide this choice and functionality, at lower layer there should be implementations that let the user of the library make the choices (and even going reactive single threaded). And eg using iocp the threading is basically done by the os (just as long as you have a pool of thread dequeing packages). For the other notification mechanisms i guess a leader follower pattern should be implemented.
I have a hard time seeing why users should care, unless they have grown up in this area and just want to know/control everything. As a library user, I want to focus on my problems. Networking was something I was forced into and have had to make the best of it<g>. I wanted a library that did the right thing, did it efficiently and offered me choices on how I wanted to know when operations were complete. Some choices trade convenience for efficiency. Maybe I am guilty of making this "simpler than possible" to paraphrase Einstein, but I don't think so. I see this like basic_string<>: it will make most of the people happy most of the time. It will never be able to dislodge direct approaches where portability is of little concern and absolutely optimal performance is required. In such efforts, iostreams, FILE* and other standard facilities start being eliminated as well in favor of native API's (i.e., "down to the metal"). I think the times that this will be necessary given my proposal will be very few. No library can reasonably expect to be spared in the quest for optimal performance. ;)
I think ut looks fairly good as well and agree whith calebs and have kind of don't like the extensive use of shared_ptr I think ownership and resource managment should be handed over to the user and not mandated by the library.
I'm not totally in love with shared_ptr either, but it was a more or less direct port from the original. The shared_ptr does bring a good deal of inefficiency with it (though that is improving steadily). If others feel the same, I will look at how to best do without. Best, Don __________________________________ Yahoo! Mail Mobile Take Yahoo! Mail with you! Check email on your mobile phone. http://mobile.yahoo.com/learn/mail
Hi Don and Michel, ----- Original Message ----- From: "Don G" <dongryphon@yahoo.com> To: <boost@lists.boost.org> Sent: Thursday, April 14, 2005 4:07 PM Subject: [boost] Re: async socket i/o [long]
Hi Michel,
Ok the reason I expose it is for the possibility to write single threaded reactive servers.
I have written such using the library I proposed. The missing ingredient is a function queue mechanism. I posted one to the list a couple weeks ago, but feel that it is not technically a networking thing. Of course, the network layer could be coupled to it in some key places which would make things simpler to use.
[snip long useful text] I agree and nicely written :-) At least that stuff feels better. I have to admit I'm still tangled up with the broader scope. Wondered if this might clarify things; [4] boost --|-------------------serialization-------------------------| [3] boost --|-----------|---------------|- sync file --|-- sync socket -| [2] boost --|-----------|---------------|- async file -|- async socket -|-------------- [1] boost --|- sync file -|- sync socket -|-----------------------------|- async flow - [0] system -|- sync file -|- sync socket -|- async file -|- async socket -|- async flow - Level [0] shows the platform APIs. The repeated names at levels [1] and [2] are the Boost interface to the same names at system level; the boost versions are platform independent. The boost async libraries for file and socket I/O are bumped up to level [2] to highlight the need for a generic "async flow" library. A means of passing control from one dis-joint code fragment to another. Sync file and sync socket appear again on level [3] as that behaviour may be re-implemented over the async level. Serialization is shown at the top for obvious reasons. I suppose for any given platform there may be holes, e.g. an old MacOS may not support async file. According to this diagram at least, sync file and sync socket are optionally implementable as either a layer over the platform API or as something that makes use of the boost async file and async socket. While this might be all a bit pie-in-the-sky, if sync file and sync socket can be implemented over a platform independent interface, the only time you wouldnt do so would be when the platform has no async support? Or for absolute best speed as you described in your text?
participants (3)
-
Don G -
Michel André -
Scott Woods