Secret handles and descriptors: Enemies of reuse, conspirators with frameworks.
Boost now contains several classes that provide abstraction for operating system resources that are logically _waitable_. A waitable resource is any resource for which an operation on it might not complete immediately. Examples include threads and processes, files and the file system, and sockets and pipes. A notification of some activity on a waitable resource is often called an _event_. In modern software, it is often necessary to wait for events asynchronously. Quite often threading is a good solution for this, but quite often it isn't. In these latter cases, it is necessary to have some sort of _demultiplexor_ entity that handles the system-specific mechanics of waiting for events. In both UNIX and Windows, to wait for events relating to a resource, one must have a handle or a descriptor associated with that resource. In general, this must be provided by the class that maintains the abstraction itself. If a class managing a waitable resource, through encapsulation, allows no way for a demultiplexor to access the handle needed for waiting, waiting is impossible. Despite having all the functionality that a programmer may require, this class is useless to the programmer because it can not be demultiplexed. This is unacceptable. To solve this design deficiency, a particular Boost demultiplexor class might be created. Boost classes that are associated with waitable resources might provide the needed handles to the demultiplexor—perhaps using _friend_—and so make asynchronous waiting possible. Unfortunately, there is a problem with this approach. Resources managed by classes outside of Boost would, in general, be incompatible with the demultiplexor. In addition, demultiplexors other than the one provided by Boost—that might be able to handle these classes incompatible with the Boost demultiplexor—would likely be unusable. The nature of typical demultiplexor implementations is that they monopolize control flow (for reasons that are usually unavoidable), and may only cooperate within the same program by using separate processes or threads, which may be inappropriate if there is no other logical reason for such separation. The affect is that waitable resources provided by libraries other than Boost are useless to the Boost programmer, as they are fundamentally incompatible. This is the mark of a conniving, conspiratorial framework, which lures its victims in with attractive features, and then locks them in its dungeon by preventing them from using other libraries. This is also unacceptable. It is the unfortunate truth that until the C++ world decides on universal resource concepts there will always need to be some piece of code that knows the interfaces of both the waitable resource and the demultiplexor. However, there is no need for this dependent code to be part of either the demultiplexor or the resource. This code might be an adapter for the Boost thread class that allows it to be compatible with a particular third party demultiplexor class. In consideration of the these issues, one may conclude that there must be a mechanism in Boost classes to allow arbitrary code to access the encapsulated handle or descriptor that describes the underlying resource for purposes of implementing demultiplexing. While this may, at first, seem a potential violation of encapsulation, it might also be seen as proper application of the Open-Closed Principle. One simple mechanism for achieving this is presented: class unix_thread_handle { protected: pthread_t handle() { return hand; } private: pthread_t hand; }; class thread : public unix_thread_handle, public boost::noncopyable { // ... }; Demultiplexor adaptors are permitted to access the underlying handles by deriving from a system-specific type and copying from an existing resource. Such access is subject to the lifetime of the origin object and documented constraints about the meaning of the handle. A demultiplexor should also be designed to be compatible. The demultiplexor must provide some mechanism to allow code outside of Boost to obtain control flow when particular system events occur. In addition, interoperability should be maintained not only with foreign waitable resource classes, but with other demultiplexors, to the maximum extent possible. For example, on a system where an event is delivered through a chain of handlers (such as Windows hooks or UNIX sigaction), the demultiplexor must be careful to pass on the event when appropriate. In light of recent interest and development relating waiting and demultiplexing, it is especially important that the above points are considered. Writing a naïve demultiplexor that will forever plague future programmers who use third party libraries is a temptation that should be strenuously ignored. There are already plenty of frameworks that have implemented this unfortunate xenophobia. If Boost is able to offer an interface that makes its waitable resources and future demultiplexor openly compatible with the rest of the world, it will continue to be popular long after other libraries have outlived their usefulness. Any thoughtful consideration and comments would be greatly appreciated. If there is some interest in these things for Boost, I’d like to prepare some proof-of-concept alterations for waitable parts of Boost. I also have designed a cross-platform demultiplexor class, but unfortunately it is not in a state that is immediately ready to be shared with Boost. However, if there was interest, I would consider polishing it up. Aaron W. LaFramboise
On Sun, 12 Sep 2004 18:53:41 -0500, Aaron W. LaFramboise wrote
Any thoughtful consideration and comments would be greatly appreciated. [UTF-8?] If there is some interest in these things for Boost, Iâd like to prepare some proof-of-concept alterations for waitable parts of Boost. I also have designed a cross-platform demultiplexor class, but unfortunately it is not in a state that is immediately ready to be shared with Boost. However, if there was interest, I would consider polishing it up.
Aaron -- Obviously the time is ripe for this discussion. Carlo Wood is indicating an interest in working this very issue. He is trying to start some discussion. We are pondering taking some of the discussion off-list so we can get to some solid footing before getting everyone involved. Anyway, have a look at http://lists.boost.org/MailArchives/boost/msg70913.php. and some of the subsequent email. Anyway I'd say the answer is there is a clear interest. Some discussion and work has come before, but it isn't the last word on this topic by any means... http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostSocket/M... Jeff
"Jeff Garland" <jeff@crystalclearsoftware.com> writes:
On Sun, 12 Sep 2004 18:53:41 -0500, Aaron W. LaFramboise wrote
Any thoughtful consideration and comments would be greatly appreciated. [UTF-8?] If there is some interest in these things for Boost, Iâ..d like to prepare some proof-of-concept alterations for waitable parts of Boost. I also have designed a cross-platform demultiplexor class, but unfortunately it is not in a state that is immediately ready to be shared with Boost. However, if there was interest, I would consider polishing it up.
Aaron --
Obviously the time is ripe for this discussion. Carlo Wood is indicating an interest in working this very issue. He is trying to start some discussion. We are pondering taking some of the discussion off-list so we can get to some solid footing before getting everyone involved. Anyway, have a look at http://lists.boost.org/MailArchives/boost/msg70913.php.
and some of the subsequent email. Anyway I'd say the answer is there is a clear interest. Some discussion and work has come before, but it isn't the last word on this topic by any means...
http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostSocket/M...
Before you take the discussion away, could someone please give a simple description of what's meant by "demultiplexor?" Both Aaron's and Carlos' posts seem to assume the reader knows what it is (they hint, but the hints aren't enough for me). -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
On Sun, 12 Sep 2004 21:52:22 -0400, David Abrahams wrote
http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostSocket/M...
Before you take the discussion away, could someone please give a simple description of what's meant by "demultiplexor?" Both Aaron's and Carlos' posts seem to assume the reader knows what it is (they hint, but the hints aren't enough for me).
If read the wiki page above that will give you a detailed idea of what we are talking about. Other terms, that are probably more precise, are 'reactor' (ala Doug Schmidt's patterns), dispatcher, or event dispatcher/loop. Jeff
David Abrahams wrote:
Before you take the discussion away, could someone please give a simple description of what's meant by "demultiplexor?" Both Aaron's and Carlos' posts seem to assume the reader knows what it is (they hint, but the hints aren't enough for me).
I said:
In modern software, it is often necessary to wait for events asynchronously. Quite often threading is a good solution for this, but quite often it isn't. In these latter cases, it is necessary to have some sort of _demultiplexor_ entity that handles the system-specific mechanics of waiting for events.
To clarify my meaning a little bit: my_waitable_resource r(...); r.on_some_event(my_callback_functor); The trick is getting the control flow into my_waitable_resource when the event happens so it can call that callback. Many different resources, that may not know about each other, all need to cooperate somehow to get control flow when they need it. Ultimately, there will need to be some blocking system call that waits for events. No one resource can block by itself, because it will stop control flow, and prevent other resources from handling their events. I've been calling the peice of code that arbitrates and wait for events the 'demultiplexor.' On UNIX, a demultiplexor probably will be a wrapper for select(). On Windows, it might be a wrapper for WaitForMultipleObjectsEx(). I'd expect it to be able to handle pipes, sockets, files, signals/exceptions, and microsecond timers. Aaron W. LaFramboise
Aaron W. LaFramboise wrote:
On UNIX, a demultiplexor probably will be a wrapper for select(). On Windows, it might be a wrapper for WaitForMultipleObjectsEx(). I'd expect it to be able to handle pipes, sockets, files, signals/exceptions, and microsecond timers.
I forgot to add: it should also be able to handle thread and process events, such as child process termination.
On Sun, 12 Sep 2004 22:11:57 -0500, Aaron W. LaFramboise wrote
Aaron W. LaFramboise wrote:
On UNIX, a demultiplexor probably will be a wrapper for select(). On Windows, it might be a wrapper for WaitForMultipleObjectsEx(). I'd expect it to be able to handle pipes, sockets, files, signals/exceptions, and microsecond timers.
I forgot to add: it should also be able to handle thread and process events, such as child process termination.
I would add -- GUI events, third party tool events (eg: messages from a framework like TIBCO). Bottom line is it needs to be extensible. Jeff
"Aaron W. LaFramboise" <aaronrabiddog51@aaronwl.com> writes:
Aaron W. LaFramboise wrote:
On UNIX, a demultiplexor probably will be a wrapper for select(). On Windows, it might be a wrapper for WaitForMultipleObjectsEx(). I'd expect it to be able to handle pipes, sockets, files, signals/exceptions, and microsecond timers.
I forgot to add: it should also be able to handle thread and process events, such as child process termination.
Thanks. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
"Aaron W. LaFramboise" wrote:
On UNIX, a demultiplexor probably will be a wrapper for select(). On Windows, it might be a wrapper for WaitForMultipleObjectsEx().
Have you considered a completion port to get around 64 handles limitation? Tony
Tony Juricic wrote:
"Aaron W. LaFramboise" wrote:
On UNIX, a demultiplexor probably will be a wrapper for select(). On Windows, it might be a wrapper for WaitForMultipleObjectsEx().
Have you considered a completion port to get around 64 handles limitation?
Yes, I agree that this mechanism is the way to go. Using it, I do not think the handle limitation should be a problem. Aaron W. LaFramboise
Hi Aaron (Rabid Dog?), Read your post with interest. An observation relating to the outlined architecture follows. The demultiplexor that requires all waitable services to expose their handles (sounds positively pornographic); does it have to be that way? In the work by D. Schmidt (already referenced by J. Garland) that describes Active Objects there is an object (i.e. instance of a particular class) that is pretty much equivalent to a thread. There may be a variety of these objects (instances of types derived from a common base "thread" class) in a running process. Each Active Object Scheduler (AKA demultiplexor or reactor) waits on 1 or more waitable services and translates the related events into a generic event that may be routed anywhere in the process. If a Scheduler is created around a troublesome waitable service (i.e. one that refuses to expose its handles), doesnt this achieve the uniformity across asynchronous activity that you are pursuing? Cheers, Scott
On Mon, 13 Sep 2004 16:14:47 +1200, Scott Woods wrote
Hi Aaron (Rabid Dog?),
Read your post with interest. An observation relating to the outlined architecture follows.
The demultiplexor that requires all waitable services to expose their handles (sounds positively pornographic); does it have to be that way?
Actually, I believe it probably does. For the simple reason that there needs to be a list of these handles for the OS to wait on....
In the work by D. Schmidt (already referenced by J. Garland) that describes Active Objects there is an object (i.e. instance of a particular class) that is pretty much equivalent to a thread. There may be a variety of these objects (instances of types derived from a common base "thread" class) in a running process.
Each Active Object Scheduler (AKA demultiplexor or reactor) waits on 1 or more waitable services and translates the related events into a generic event that may be routed anywhere in the process.
Yes, but even to build that concurrency model you need to have a multiplexor with access to the underlying resource. The reason is that your active object might want to do several things with its' single thread of execution. For example, it might manage several sockets, devices, or timers in one thread because that makes up a logical object of some kind. Thus it can't block the its' thread just to wait for input or a timeout. It needs the mutliplexor to call it back asynchronously when input or timeout is available...
If a Scheduler is created around a troublesome waitable service (i.e. one that refuses to expose its handles), doesnt this achieve the uniformity across asynchronous activity that you are pursuing?
You totally lost me here... Jeff
----- Original Message ----- From: "Jeff Garland" <jeff@crystalclearsoftware.com> To: <boost@lists.boost.org> Sent: Monday, September 13, 2004 4:31 PM Subject: Re: [boost] Secret handles and descriptors: Enemies of reuse,conspirators with frameworks. <snip>
If a Scheduler is created around a troublesome waitable service (i.e. one that refuses to expose its handles), doesnt this achieve the uniformity across asynchronous activity that you are pursuing?
You totally lost me here...
If we want to be able to respond asynchronously to all waitable services by executing some chunk of predetermined code, then the Active Objects pattern can be applied to achieve Aarons goal. This application looks similar to Aarons but different in that there would be one _or more_ demultiplexors. The reasons for splitting an app into multiple demultiplexors would include those troublesome waitable services; just dedicate an entire demultiplexor to the varmint and see if that solves the problem. Dont know if that has helped. After reading your response I suspect that even applying this pattern there may be some APIs (to those damn services) that might still prevent the ultimate goal. Sorry but I've run out of time. I will check on this thread tomorrow. Cheers, Scott
Scott Woods wrote:
Hi Aaron (Rabid Dog?),
I didn't pay attention to what my email address generator had done until it was too late. (I use a separate email address for each community or correspondent.)
The demultiplexor that requires all waitable services to expose their handles (sounds positively pornographic); does it have to be that way?
Discounting polling, as its generally unacceptable, you always will need access to that underlying system resource handle, unless . . .
If a Scheduler is created around a troublesome waitable service (i.e. one that refuses to expose its handles), doesnt this achieve the uniformity across asynchronous activity that you are pursuing?
. . . you have a thread dedicated to waiting on that particular resource. A problem with this is that it is hugely inefficient, requiring one thread per resource. A thread is a rather expensive price for maintaining a somewhat artificial encapsulation boundary. In addition, this might not really buy you much in practice. If you're going to go ahead and create a separate thread for each waitable resource, you might as well do so directly and not use any sort of multiplexor at all, and probably save yourself some trouble. I agree that this scheme might be acceptable as a last resort method for forcing grossly incompatible objects into the system, but I do not think it should be the norm where it is easily avoidable. Aaron W. LaFramboise
----- Original Message ----- From: "Aaron W. LaFramboise" <aaronrabiddog51@aaronwl.com> To: <boost@lists.boost.org> Sent: Monday, September 13, 2004 5:31 PM Subject: Re: [boost] Secret handles and descriptors: Enemies of reuse,conspirators with frameworks. <snip>
Discounting polling, as its generally unacceptable, you always will need access to that underlying system resource handle, unless . . .
If a Scheduler is created around a troublesome waitable service (i.e. one that refuses to expose its handles), doesnt this achieve the uniformity across asynchronous activity that you are pursuing?
. . . you have a thread dedicated to waiting on that particular resource. A problem with this is that it is hugely inefficient, requiring one thread per resource. A thread is a rather expensive price for maintaining a somewhat artificial encapsulation boundary.
In addition, this might not really buy you much in practice. If you're going to go ahead and create a separate thread for each waitable resource, you might as well do so directly and not use any sort of multiplexor at all, and probably save yourself some trouble.
I agree that this scheme might be acceptable as a last resort method for forcing grossly incompatible objects into the system, but I do not think it should be the norm where it is easily avoidable.
Ah yes. Beautifully captured. I agree with all the points you make, i.e. the disadvantages of "a dedicated thread". The language used is a little emotive ("I object Your Honor!"). While no "universal demultiplexor" exists as long as vendors provide closed APIs there is _no_ golden solution. So perhaps the different approaches are fine for different targets. At least until the Great Day when we can see all those handles. For some applications (i.e. non real-time) the loss of performance may be perfectly acceptable? The spin-off benefits (e.g. code maintainability) of achieving a uniformity of code WRT asynchronous events are particularly nice, because they are enjoyed by us :-) Cheers, Scott
participants (5)
-
Aaron W. LaFramboise -
David Abrahams -
Jeff Garland -
Scott Woods -
Tony Juricic