State of giallo, (sockets and demultiplexing)
Spend a few days off line and you suddenly miss a whole conversation :-( It's going to take a while to catch up... Recent IOstreams discussion has indeed been very interesting, espescially as I am finally getting round to implementing buffers and filters for giallo (boost.sockets). At the moment I am leaning towards ACE style message buffer, using the apache filter idea of flush/eos/eoc control (but trying to avoid any knowledge of the underlying resource). So where is giallo today (apart from being temporarily broken on gcc due to mpl changes)? i) Low level, platform header hiding, object based wrappers of sockets, named pipes, select, and completion ports (and a POSIX aio wrapper on linux that I have tried to get to work in many different ways and failed so far). ii) proactor and reactor interfaces, with policy based implementation, provided by bridge classes to the low level completion port and select wrappers above. iii) 'event completion notification' style connector, acceptor and connection classes for sockets and named pipes. These classes combine a demultiplexer with a resource (socket, pipe, etc), and can use a proactor or reactor demultiplexer interchangeably (via template parameter). The connection selects the synchronisation mode (block, non-blocking, asynchronous) of the socket based the demultiplexer. The library is thread aware (can run multithreaded, will create threads if asked to, uses threads to do asynchronous connect if not natively supported, etc) There is a streambuf implementation, but I have not found a way to implement this without it degenerating to a thread per connection model. I feel quite strongly that the decision to implement connection, connector and acceptor concepts with a single (in this case 'event completion' style) notification interface, regardless of the demultiplexer mechanism used, is very important, and can lead to a stream/filter concept that is independent of demultiplexer, of blocking/non-blocking/asynch mode and thus independent of the resource used.
Hugo Duncan wrote:
i) Low level, platform header hiding, object based wrappers of sockets, named pipes, select, and completion ports (and a POSIX aio wrapper on linux that I have tried to get to work in many different ways and failed so far).
One concern I have is that this will not mesh well with other clients of demultiplexors, such as process management or GUIs (network libraries and gui libraries are both notorious for monopolizing control flow). These things apparently need to be run in a separate thread. My primarily interest in demultiplexors is that they should be universal and unintrusive. Its my feeling that once an elegant, universal, and free C++ demultiplexor exists, people will stop reimplementing it over and over, and spend more time writing their networking library or GUI. Aaron W. LaFramboise
Aaron W. LaFramboise wrote:
Hugo Duncan wrote:
i) Low level, platform header hiding, object based wrappers of sockets, named pipes, select, and completion ports (and a POSIX aio wrapper on linux that I have tried to get to work in many different ways and failed so far).
One concern I have is that this will not mesh well with other clients of demultiplexors, such as process management or GUIs (network libraries and gui libraries are both notorious for monopolizing control flow). These things apparently need to be run in a separate thread.
I am not sure how your comment arises from the part of my post that you quoted. The proactor and reactor classes in giallo know nothing of sockets etc, and they can run in whichever thread you want.
My primarily interest in demultiplexors is that they should be universal and unintrusive. Its my feeling that once an elegant, universal, and free C++ demultiplexor exists, people will stop reimplementing it over and over, and spend more time writing their networking library or GUI.
Agreed! I am sure there are deficiencies in the giallo demultiplexors, but the design aims are congruent with your interests. Hugo
On Wed, Sep 15, 2004 at 04:05:00PM -0400, Hugo Duncan wrote:
The proactor and reactor classes in giallo know nothing of sockets etc, and they can run in whichever thread you want.
Where exactly can I read about the API of this part? I cannot imagine at the moment how this would work :/ (see my previous post) -- Carlo Wood <carlo@alinoe.com>
Carlo Wood wrote:
On Wed, Sep 15, 2004 at 04:05:00PM -0400, Hugo Duncan wrote:
The proactor and reactor classes in giallo know nothing of sockets etc, and they can run in whichever thread you want.
Where exactly can I read about the API of this part? I cannot imagine at the moment how this would work :/ (see my previous post)
documentation is sorely lacking... but giallo adds a level of indirection to what I have understood of your current thinking: reads, writes etc are not directly initiated by the demultiplexer, rather by the connection class, which is responsable for combining the socket (pipe, or whatever else) class with the demultiplexer, hiding the details of which demultiplexer is used and ensuring the socket is in the correct blocaking/non-blocking/ asynch mode.
On Wed, Sep 15, 2004 at 04:55:05PM -0400, Hugo Duncan wrote:
documentation is sorely lacking... but giallo adds a level of indirection to what I have understood of your current thinking:
reads, writes etc are not directly initiated by the demultiplexer, rather by the connection class, which is responsable for combining the socket (pipe, or whatever else) class with the demultiplexer, hiding the details of which demultiplexer is used and ensuring the socket is in the correct blocking/non-blocking/ asynch mode.
Ok, I considered that to be obvious. Consider the abstract Source 'device'. This device is capable to delivering a (byte) stream of data and/or a stream of data in the form of 'packets'. Also the byte-stream will will be delivered in packets (significantly more than 1 byte at the time) but in that case the size of a 'message' or packet is unknown. Then what is needed is the following: 1) An object that encapsulates the 'device' abstration. 2) A way to request more data (this exists of a function call to which the user provides a buffer pointer, and the size of that buffer) 3) An entry point for a thread to call to 'sleep/wait'. 4) A callback (or completion) function that is called when a new packet was received. It should be possible that point 4) is *entirely* handled in the user code, the only thing that the user wants it get control of the thread again once data is available. The user then can examine the received data, manage its own buffering and decide to either decode a message and handle it - or to leave the partial message in the buffer and request for more data. Only once we have this interface we can provide "convenience" classes that do buffering for the user as well. And even think about providing protocol decoders too for common types of protocols (text, envelloped binary, fixed message sizes etc). The most frequently used name for the function mentioned in point 2 above that I have seen for sockets is 'async_recv'. Note that async_recv cause its completion routine to be called as soon as new data is available and additional data would cause more delay (blocking). A function async_recv_n, that requests *precisely* 'n' bytes, and not less, can be easily implemented on top of async_recv - or by means of the OS in some cases. -- Carlo Wood <carlo@alinoe.com>
On Wed, 15 Sep 2004 Carlo Wood wrote:
On Wed, Sep 15, 2004 at 04:05:00PM -0400, Hugo Duncan wrote:
The proactor and reactor classes in giallo know nothing of sockets etc, and they can run in whichever thread you want.
I cannot imagine at the moment how this would work :/ (see my previous post)
On Wed, 15 Sep 2004 Carlo Wood wrote:
On Wed, Sep 15, 2004 at 04:55:05PM -0400, Hugo Duncan wrote:
documentation is sorely lacking... but giallo adds a level of indirection to what I have understood of your current thinking:
reads, writes etc are not directly initiated by the demultiplexer, rather by the connection class, which is responsable for combining the socket (pipe, or whatever else) class with the demultiplexer, hiding the details of which demultiplexer is used and ensuring the socket is in the correct blocking/non-blocking/ asynch mode.
Ok, I considered that to be obvious.
Ok, so I seem to have misunderstood what you were asking.
1) An object that encapsulates the 'device' abstration.
in giallo, this is the socket class.
2) A way to request more data (this exists of a function call to which the user provides a buffer pointer, and the size of that buffer)
in giallo, this is "connection<>::recv".
3) An entry point for a thread to call to 'sleep/wait'.
in giallo, after calling send/recv the calling function just returns. notification is always of completion and is not necessarily in the same thread.
4) A callback (or completion) function that is called when a new packet was received.
in giallo, user callback is attached to connection
It should be possible that point 4) is *entirely* handled in the user code, the only thing that the user wants it get control of the thread again once data is available. The user then can examine the received data, manage its own buffering and decide to either decode a message and handle it - or to leave the partial message in the buffer and request for more data.
that is how the connection class in giallo works. it contains no "buffer management".
Only once we have this interface we can provide "convenience" classes that do buffering for the user as well. And even think about providing protocol decoders too for common types of protocols (text, envelloped binary, fixed message sizes etc).
agreed. Hugo
On Thu, Sep 16, 2004 at 11:15:19AM -0400, Hugo Duncan wrote:
in giallo, this is "connection<>::recv".
Is there online documention about 'connection<>'? I tried to look in the only thing that seems to be available for giallo, the cvs web interface - and I still had to click around for 10 minutes before I even found "connection<>::recv". Then another 2 minutes before I found the base class implementation of that and at that point I gave up finding and understanding how it worked further (it was calling m_demux.add() there...) A library without excellent documentation is normally pretty useless. I'd rather start from scratch (and document the new library while developing it) than to try and understand an undocumented library that might not even be robust and/or capable of what I want to use it for :/ -- Carlo Wood <carlo@alinoe.com>
On Wed, Sep 15, 2004 at 02:37:06PM -0500, Aaron W. LaFramboise wrote:
One concern I have is that this will not mesh well with other clients of demultiplexors, such as process management or GUIs (network libraries and gui libraries are both notorious for monopolizing control flow). These things apparently need to be run in a separate thread.
Very good point.
My primarily interest in demultiplexors is that they should be universal and unintrusive. Its my feeling that once an elegant, universal, and free C++ demultiplexor exists, people will stop reimplementing it over and over, and spend more time writing their networking library or GUI.
I also think that we should concentrate on a *minimal* interface. That is: just wrap whatever is now being handled by select/poll etc. However, recently I realized that this is not possible (after reading http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/win...) The only way to provice a portable interface is by providing a 'socket' class, that is explicitly a socket. After all, in the case of windows you will need to wrap WSARecv which is socket specific. Under UNIX you can ignore if handles are sockets or files etc, and just treat everything as 'int' handles (making it the responsibility of the user to make them non-blocking), but that is not possible with the interface that windows provides. You cannot make an interface for an abstraction 'class Handle'. The reason for that is that too many details are important for the demultiplexor when calling WSARecv, we cannot leave that to the user. I am afraid that we have no other option then to wrap the notions of 'handles' completely, and instead provide an interface that talks about specific 'devices' (sockets, files, pipes, timers etc.) -- Carlo Wood <carlo@alinoe.com>
On Wed, 15 Sep 2004 07:48:15 -0400, Hugo Duncan wrote
Spend a few days off line and you suddenly miss a whole conversation :-( It's going to take a while to catch up...
Well, I've been involved and I still not caught up...
Recent IOstreams discussion has indeed been very interesting, espescially as I am finally getting round to implementing buffers and filters for giallo (boost.sockets). At the moment I am leaning towards ACE style message buffer, using the apache filter idea of flush/eos/eoc control (but trying to avoid any knowledge of the underlying resource).
Are you talking about the ACE Message Block / ACE Message Queue stuff? http://www.dre.vanderbilt.edu/Doxygen/Current/html/ace/classACE__Message__Bl... http://www.dre.vanderbilt.edu/Doxygen/Current/html/ace/classACE__Message__Qu... If so, I'm really hoping we can do better. I understand that the key desire here is to avoid lots of copying and I agree with that. But, honestly I think at the socket boundary layer we really just want an IOStream that I can read/write to and control the send/recv. The reason is that typically I want / need to make one transformation at the process boundary anyway. So, for example, when I recieve an inbound message I'm going to unserialize whatever it is into a native structure. So I have to allocate (or take from a pool) and go field by field thru the data anyway realizing it into the native structure. Then I process the data. I'll never make any copies in the processing core b/c I'm going to pass around shared_ptr's or something. Going outbound I've built up a complex structure that needs to be serialized into a buffer that gets sent after I'm done. I'm passing this all around using smart pointers until I get to the socket stream and it's ready to go outbound. So if I'm writing some network code, I'm thinkin I just want to use boost.serialize to do the serialization and it needs a stream to do that. And actually, I'm curious if the file descriptor wrapper in the IOStream library wouldn't pretty much take care of it? The only exception I can think of is if the data is to be 'multicast' down 2 TCP connections...
So where is giallo today (apart from being temporarily broken on gcc due to mpl changes)?
...snip lots of good status info...
There is a streambuf implementation, but I have not found a way to implement this without it degenerating to a thread per connection model.
I would think it would tend to be a streambuf per connection regardless of the threading? Anyway, glad you're back -- I hope between all the interested parties we can get enough momentum to sustain things to the point of actually producing a boost library for demultipexing and sockets.... Jeff
Jeff Garland <jeff <at> crystalclearsoftware.com> writes:
So, for example, when I recieve an inbound message I'm going to unserialize whatever it is into a native structure. So I have to allocate (or take from a pool) and go field by field thru the data anyway realizing it into the native structure. Then I process the data. I'll never make any copies in the processing core b/c I'm going to pass around shared_ptr's or something.
Going outbound I've built up a complex structure that needs to be serialized into a buffer that gets sent after I'm done. I'm passing this all around using smart pointers until I get to the socket stream and it's ready to go outbound.
I think it would be very nice if we could have a higly efficient solution to the general and recurring problem of the construction of an object from a data stream together with the deconstruction of an object into a data stream. Patterns touching on this topic have been written by, among others, Steve Berzuk at www.berczuk.com and Dirk Riehle (http://www.riehle.org/computer- science/research/1996/plop-1996-serializer.pdf) Tommy
Hi Jeff, On Wed, 15 Sep 2004 Jeff Garland wrote:
On Wed, 15 Sep 2004 07:48:15 -0400, Hugo Duncan wrote
Recent IOstreams discussion has indeed been very interesting, espescially as I am finally getting round to implementing buffers and filters for giallo (boost.sockets). At the moment I am leaning towards ACE style message buffer, using the apache filter idea of flush/eos/eoc control (but trying to avoid any knowledge of the underlying resource).
Are you talking about the ACE Message Block / ACE Message Queue stuff?
The Message Block stuff, yes, but not wholesale. Trying to make it easier to use.
If so, I'm really hoping we can do better. I understand that the key desire here is to avoid lots of copying and I agree with that. But, honestly I think at the socket boundary layer we really just want an IOStream that I can read/write to and control the send/recv. The reason is that typically I want / need to make one transformation at the process boundary anyway.
Isn't this just one use case? Is a streambuf all you need to be able to support this? How would you implement a protocol stack?
So if I'm writing some network code, I'm thinkin I just want to use boost.serialize to do the serialization and it needs a stream to do that. And actually, I'm curious if the file descriptor wrapper in the IOStream library wouldn't pretty much take care of it?
using boost.serialize and spirit are two examples that I would like to build. I have started an http_server that uses spirit to parse the header, but early days yet...
There is a streambuf implementation, but I have not found a way to implement this without it degenerating to a thread per connection model.
I would think it would tend to be a streambuf per connection regardless of the threading?
What I was trying to say was that I have been unable to implement a streambuf that uses asynchronous IO without having the implementation of the streambuf degenerate to thread per connection. The only way I could think of getting asynchronous IO notification to work is to have the streambuf block on a condition variable after read/write and have that condition variable notified in the io completion routine. ie, as far as I can see, the streambuf concept is broken if you do not want a thread for each connection.
Anyway, glad you're back -- I hope between all the interested parties we can get enough momentum to sustain things to the point of actually producing a boost library for demultipexing and sockets....
I second that. Hugo
On Thu, 16 Sep 2004 07:49:20 -0400, Hugo Duncan wrote
Are you talking about the ACE Message Block / ACE Message Queue stuff?
The Message Block stuff, yes, but not wholesale. Trying to make it easier to use.
Ok, at the end of the game it's main use in ACE seems to be for implementing queues for active objects and sending data across threads. Of course it's ugly because it's not typesafe (at least without alot of effort). In a fairly recent project a co-developer wrote his own using a condition variable and std::queue -- he did this instead of using the ACE queues b/c it was cleaner.
Isn't this just one use case?
Well, I suppose there are others but this is the one I run into all the time..
Is a streambuf all you need to be able to support this?
No, I actually need the stream on top of the streambuf so I can do formatted writes/reads to the buffer. Then I need to be able to use the buffer contents to send via the socket -- ideally without copying. BTW, I've done this using std::stringstream before. It was 'horribly inefficient' since I had to bulk copy the streambuf contents to the socket after I was done serializing on write and had to bulk copy to the buffer on read (although I'm not 100% sure I had to copy on read now that I think about it). Anyway, the code was simple and clean and the peformance hit for this app was minimal.
How would you implement a protocol stack?
I'd probably build up a series of serializeable objects that contain other serializeable objects. Or I might have a specialized archiver that knows how to build protocol headers since often times the protocol wrappers need to know nasty little details like the length of the message.
So if I'm writing some network code, I'm thinkin I just want to use boost.serialize to do the serialization and it needs a stream to do that. And actually, I'm curious if the file descriptor wrapper in the IOStream library wouldn't pretty much take care of it?
using boost.serialize and spirit are two examples that I would like to build. I have started an http_server that uses spirit to parse the header, but early days yet...
HTTP server sounds like an ambitious example :-) But sure serialize and spirit a huge pieces of the puzzle here.
What I was trying to say was that I have been unable to implement a streambuf that uses asynchronous IO without having the implementation of the streambuf degenerate to thread per connection. The only way I could think of getting asynchronous IO notification to work is to have the streambuf block on a condition variable after read/write and have that condition variable notified in the io completion routine. ie, as far as I can see, the streambuf concept is broken if you do not want a thread for each connection.
Yeah, I see. No doubt we are going to need to enhance the streambuf concept to deal with async i/o. I'll have to ponder this a bit if I can ever get my head above water on the email chain, etc... Jeff
participants (5)
-
Aaron W. LaFramboise -
Carlo Wood -
Hugo Duncan -
Jeff Garland -
Tommy