Re: [boost] Re: Re: Boost.Thread : IO multiplexing
(Michael Glassford <glassfordm@hotmail.com>) wrote:
Sean Kelly wrote:
I'm not sure that much if anything would need to be done to provide support for IO multiplexing in Boost. ...
That, of course, would be the best solution. Have you looked at the code in the orignal post to see if such a solution would solve his problem?
...
I guess the question becomes, is there a need for such an IO library in Boost? The nature of the project would demand a good bit of custom code for each platform, which isn't typical of Boost components.
I have been off for a few days now, and I like to see that I have arrived at the very same conclusions than you. 1) I believe now that most likely everything can be done with boost.thread as is (question still is performance) 2) I used an approach as 1 thread / WFMO instead beeing able to directly wait on condvars _and_ IO simulateneously. (Altough this approach isn't easily portable, as it contains a lot of platform dependencies. Also this approach does not scale very well for the multiplexing case since WMFO/select scenarios are limited in the number of waitable things. This is where IO completion should enter the scene.) 3) I agree that the real question should be whether there should be a such an IO library in boost. Personally I would vote for such a thing, perhaps based on already existing "C" async_io libs, much like the pthreads lib. If anyone is interseted I could try to post my files, but I am not sure where I should post them since they are more a bit of example appliaction code than library code. Would the boost files area be adequate? Roland
It seems that network sockets are the most common use case for nonblocking I/O multiplexing (i.e. the reactor pattern). If a library restricts itself to those alone, an efficient portable implementation may be possible. Note, for example, that AFAIK, polling multiple file descriptors for reading or writing (this is not the same as waiting for asynchronous completion) is not possible under the Windows API, while under POSIX, Linux, and BSD, AFAIK, it is not possible to wait on a set of mutexes or condition variables while also polling a set of file descriptors. These two constraints effectively rule out creating a boost i/o library that can handle polling anything but sockets. Clearly, using multiple threads to attempt to emulate this type of polling is not a viable solution, because it would likely be highly inefficient. On the other hand, I believe it would be possible to create a portable library which utilized either WSAAsyncSelect or select on Windows platforms, kqueue on BSD platforms, epoll on Linux kernels that support it, and poll on other POSIX platforms. (On Solaris, I believe it would be possible to utilize /dev/poll.) As far as asynchronous read/write (on sockets) handling, I do not believe it is possible to unify waiting for asynchronous completion with the type of polling described above, at least on POSIX platforms. It would be possible to create a portable library for waiting (only) for asynchronous completion of a read or write. One issue with the POSIX asynchronous i/o interface is that it does not, AFAIK, support reading or writing asynchronous out-of-band data on TCP sockets. I suppose though that since out-of-band data is not used all that often, this is less of an issue; still, it is a reason for people to use polling rather than asynchronous read/write, particularly since in a network server, it would be necessary to have at least one thread waiting to accept new clients. -- Jeremy Maitin-Shepard
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org]On Behalf Of Jeremy Maitin-Shepard Sent: Wednesday, January 28, 2004 2:51 PM To: boost@lists.boost.org Subject: Re: [boost] Re: Re: Boost.Thread : IO multiplexing
<snip>
As far as asynchronous read/write (on sockets) handling, I do not believe it is possible to unify waiting for asynchronous completion with the type of polling described above, at least on POSIX platforms. It would be possible to create a portable library for waiting (only) for asynchronous completion of a read or write.
you entire message sounded right to me. would make the following suggestion as a "way out". agree with the "[difficult] to unify waiting for async". maybe its not required? does an application truly want to "unify" at the level of sockets events and filesystem I/O completions? my suggestion is that unification should be occuring at a higher level and therefore it would be valid to cope with sockets in one thread, I/O completion in another thread,... if (quick and ugly example) a (async) socket is open to an FTP server, is the application interested in the next FD_READ or the next complete, SMTP-conformant, message, e.g. "user"? don't know if i've explained myself very well but in my mind at least this absolves you of any guilt as you narrow your focus onto the "select" family. cheers, scott
Jeremy Maitin-Shepard wrote:
It seems that network sockets are the most common use case for nonblocking I/O multiplexing (i.e. the reactor pattern). If a library restricts itself to those alone, an efficient portable implementation may be possible.
I would say restrict it to socket and file i/o and expect that it would be used almost exclusively for socket i/o.
On the other hand, I believe it would be possible to create a portable library which utilized either WSAAsyncSelect or select on Windows platforms, kqueue on BSD platforms, epoll on Linux kernels that support it, and poll on other POSIX platforms. (On Solaris, I believe it would be possible to utilize /dev/poll.)
I'd like to focus on completion routines or methods that could mimic them. IOCP in Windows, /dev/poll in Solaris and either /dev/poll or epoll in Linux, perhaps kqueue in BSD? Would all of those serve? My experience is mostly with IOCP but there's a Dr. Dobbs article this month that uses /dev/poll in the same model.
As far as asynchronous read/write (on sockets) handling, I do not believe it is possible to unify waiting for asynchronous completion with the type of polling described above, at least on POSIX platforms. It would be possible to create a portable library for waiting (only) for asynchronous completion of a read or write. One issue with the POSIX asynchronous i/o interface is that it does not, AFAIK, support reading or writing asynchronous out-of-band data on TCP sockets. I suppose though that since out-of-band data is not used all that often, this is less of an issue; still, it is a reason for people to use polling rather than asynchronous read/write, particularly since in a network server, it would be necessary to have at least one thread waiting to accept new clients.
Windows has asynch listen but it's kind of annoying to use. I've always used one or more threads for the task--overhead is minimal since they're basically always blocking. Beyond that, I think the i/o layer should be a thread pool that does i/o processing exclusively. Since it's a thread pool some synchronization is already required and extending execution into the rest of the application code seems like an invitatation to trouble. One issue with this design is that the i/o layer should likely only pass complete messages back to the appliction code, which implies the need for a protocol layer. And if encryption is a possibility then the model would need to allow the protocol layer to initiate its own responses for the negotiation process. The risk here is that application code could leak into the i/o code. Any suggestions? It would be nice to get a different perspective on all of this. Sean
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org]On Behalf Of Sean Kelly Sent: Thursday, January 29, 2004 6:18 AM To: Boost mailing list Subject: Re: [boost] Re: Re: Boost.Thread : IO multiplexing
<snip>
As far as asynchronous read/write (on sockets) handling, I do not believe it is possible to unify waiting for asynchronous completion with the type of polling described above, at least on POSIX platforms. It would be possible to create a portable library for waiting (only) for asynchronous completion of a read or write. One issue with the POSIX asynchronous i/o interface is that it does not, AFAIK, support reading or writing asynchronous out-of-band data on TCP sockets. I suppose though that since out-of-band data is not used all that often, this is less of an issue; still, it is a reason for people to use polling rather than asynchronous read/write, particularly since in a network server, it would be necessary to have at least one thread waiting to accept new clients.
Windows has asynch listen but it's kind of annoying to use. I've always used one or more threads for the task--overhead is minimal since they're basically always blocking. Beyond that, I think the i/o layer should be a thread pool that does i/o processing exclusively. Since it's a thread pool some synchronization is already required and extending execution into the rest of the application code seems like an invitatation to trouble.
One issue with this design is that the i/o layer should likely only pass complete messages back to the appliction code, which implies the need for a protocol layer. And if encryption is a possibility then the model would need to allow the protocol layer to initiate its own responses for the negotiation process. The risk here is that application code could leak into the i/o code.
Any suggestions? It would be nice to get a different perspective on all of this.
tried to say the same thing in a previous mail. agree totally. if your going to mention encryption then you may as well go all out and also worry about session and presentation (which include encryption). only allude to the network model (ISO) cos its such a convenient and complete reference. doesnt mean that the entire beast has to be tackled; or even should be. by the time you have a generic environment that supports 1 or more "multiplexing threads" that exchange "complete messages" then it starts to take on the appearance of an app framework. cheers.
Sean Kelly <sean@ffwd.cx> writes:
Jeremy Maitin-Shepard wrote:
It seems that network sockets are the most common use case for nonblocking I/O multiplexing (i.e. the reactor pattern). If a library restricts itself to those alone, an efficient portable implementation may be possible.
I would say restrict it to socket and file i/o and expect that it would be used almost exclusively for socket i/o.
As far as I know, it is not possible to poll file descriptors (HANDLEs) using the Windows API, either through something like WSAAsyncSelect (i.e. sending messages to a Windows message queue), WSAEventSelect (waiting on an event, which is not a great solution since it requires use of WFMO, which is limited to 64 events). This can't even be solved by using multiple threads --- I believe that polling is simply not supported on Windows. Asynchronous completion of socket read operations and file read operations can indeed be unified, but I would not suggest that a boost library not implement polling.
On the other hand, I believe it would be possible to create a portable library which utilized either WSAAsyncSelect or select on Windows platforms, kqueue on BSD platforms, epoll on Linux kernels that support it, and poll on other POSIX platforms. (On Solaris, I believe it would be possible to utilize /dev/poll.)
I'd like to focus on completion routines or methods that could mimic them. IOCP in Windows, /dev/poll in Solaris and either /dev/poll or epoll in Linux, perhaps kqueue in BSD? Would all of those serve? My experience is mostly with IOCP but there's a Dr. Dobbs article this month that uses /dev/poll in the same model.
AFAIK, neither /dev/poll nor epoll provide similar functionality to completion ports. /dev/poll, epoll and kqueue poll a file descriptor for _available_ data to read, _available_ buffer space to write, _available_ out of band data to read, in the case of a connecting socket, a socket which has connected, and in the case of a listening socket, an _available_ new connection to accept. Windows completion ports, in contrast, allow you to determine when a particular write operation or read operation has been completed. Thus, this is a fundamentally different model. Windows completion ports are similar to using the aio_* family of functions on POSIX platforms, but on POSIX platforms, it is generally more convenient and efficient to use polling. The nature of the aio_* functions is that they do not efficient scale to waiting on a large number of concurrent operations. Specifically, it often necessary to check the status of each operation after the aio_suspent function returns. Systems like epoll, /dev/poll, and kqueue scale very well to a large number of file descriptors. On Windows platforms, it seems that systems like WSAAsyncSelect, WSAEventSelect, select, and WFMO do not scale very well to a large number of socket descriptors/handles. The fact that Windows only scales efficiently using one model, while UNIX platforms only scale efficiently using the other model, is an added and particularly problematic issue in writing a portable library. Avoiding this would require a very substantial amount of abstraction, and I would argue that mandating that layer of abstraction on all users would make the library less flexible to certain tasks.
[snip]
Windows has asynch listen but it's kind of annoying to use. I've always used one or more threads for the task--overhead is minimal since they're basically always blocking. Beyond that, I think the i/o layer should be a thread pool that does i/o processing exclusively. Since it's a thread pool some synchronization is already required and extending execution into the rest of the application code seems like an invitatation to trouble.
I believe it would be possible to create a portable library which used either asynchronous operations or polling, and automatically accepted all incoming connections, read all available data, and provided a send buffer which would get sent automatically. This could all be done using only a single thread. The user could specify certain limits on the number of connections to accept, and the size of the read and write buffers. I do not think it is possible to create a significantly lower level interface which is also portable and efficient.
[snip]
-- Jeremy Maitin-Shepard
Jeremy Maitin-Shepard wrote:
As far as I know, it is not possible to poll file descriptors (HANDLEs) using the Windows API. . . This can't even be solved by using multiple threads --- I believe that polling is simply not supported on Windows. Asynchronous completion of socket read operations and file read operations can indeed be unified, but I would not suggest that a boost library not implement polling.
IOCP works for both file and socket operations. Perhaps not a poll method but I'm not much interested in the low-end solutions anyway.
AFAIK, neither /dev/poll nor epoll provide similar functionality to completion ports. /dev/poll, epoll and kqueue poll a file descriptor for _available_ data to read, _available_ buffer space to write, _available_ out of band data to read, in the case of a connecting socket, a socket which has connected, and in the case of a listening socket, an _available_ new connection to accept. Windows completion ports, in contrast, allow you to determine when a particular write operation or read operation has been completed. Thus, this is a fundamentally different model.
That doesn't mean they can't use the same basic model though. With IOCP when a read operation unblocks the data is already in the buffer while with the other methods a read operation has to be done. Wrap that bit in the IO subsystem. The DDJ article is actually good in this respect--it serves as a useful example of how to combine the two methods. Just grab it from their website, it's in the Feb. 2004 issue, file is iomulti.zip. I think it needs a bit of work, but the basic approach isn't too bad. What remains an issue in my mind is how far to take this. Socket and file classes for each platform at least... what else? Sean Sean
(Sean Kelly <sean@ffwd.cx>) wrote:
What remains an issue in my mind is how far to take this. Socket and file classes for each platform at least... what else?
TTY, USB, FireWire, Parallel, .....? Since file classes normaly also imply all other types of communication resources this should be enough I think. But then different methods of opening and parametizing need to be abstracted as well. Perhaps by an ioctl method or similar? Or what about a library/framework where the user optionally can plug in the system dependent low level part while providing some implementation for the usual resources. This could be done like the filebuffer concept in streams? Roland
Roland <roland.schwarz@chello.at> writes:
(Sean Kelly <sean@ffwd.cx>) wrote:
What remains an issue in my mind is how far to take this. Socket and file classes for each platform at least... what else?
The obvious thing to support is win32 named pipes (in addition to various types of sockets, including unix domain sockets on unix platforms). One unrelated issue that occurs to me is elegant handling of out-of-band data on various types of sockets (particularly TCP).
TTY, USB, FireWire, Parallel, .....?
Since file classes normaly also imply all other types of communication resources this should be enough I think. But then different methods of opening and parametizing need to be abstracted as well. Perhaps by an ioctl method or similar? Or what about a library/framework where the user optionally can plug in the system dependent low level part while providing some implementation for the usual resources. This could be done like the filebuffer concept in streams?
It might be possible for the library to allow user extension, provided that the user extension uses, on UNIX platforms, a file descriptor that can be polled using the various polling methods employed by the library, or on Windows platforms, a Windows HANDLE that can be used with asynchronous completion ports. Having this level of platform dependence in this extension interface seems somewhat undesirable, however. On the other hand, it is very useful to be able to support things like tty's and pipe(2) pipes that would otherwise be effectively impossible to support. -- Jeremy Maitin-Shepard
Sean Kelly <sean@ffwd.cx> writes:
Jeremy Maitin-Shepard wrote:
[snip]
That doesn't mean they can't use the same basic model though. With IOCP when a read operation unblocks the data is already in the buffer while with the other methods a read operation has to be done. Wrap that bit in the IO subsystem.
To do it most efficiently, both read and write buffers are needed, since there should be at least two concurrent asynchronous reads and writes. I don't think that is a major problem though --- buffers can be used in almost all.
[snip]
What remains an issue in my mind is how far to take this. Socket and file classes for each platform at least... what else?
In addition to the asynchronous/non-blocking socket system, it would be useful to provide a blocking socket system. This could be have a streambuf interface. -- Jeremy Maitin-Shepard
participants (4)
-
Jeremy Maitin-Shepard -
Roland -
scott -
Sean Kelly