On 4/13/05, Don G <dongryphon@yahoo.com> wrote:
On 4/13/05, Caleb wrote:
Overall it looks pretty nice.
Thanks. :) And thanks for taking the time to look at the proposal.
A few observations: * port_t should be unsigned short, not unsigned int
That is, of course, true of TCP/IP :)
Hm, hadn't thought of non-IP. What protocols out there which support a port/channel concept use a larger port range than TCP/IP? Just curious.
* timeout constructors are inconsistent. One takes seconds + microseconds and one takes milliseconds, at least according to the names of the arguments.
Yeah, that was one of the adjustments I tried to make as I boost-ified the code. The Windows way is millisecs, but Unix is microsecs. To do microsecs, I think we would have to use uint64's. That would be fine by me, but I wasn't sure how widely supported it is. I know that for Solaris (SunPro), gcc, MSVC, CodeWarrior, uint64 is good. Of course, a timespec doesn't use uint64. La de da... :)
What about Bob Bell's suggestion of using double? Would there be concerns about the overhead in all of the double -> integer conversions necessary to interface with the various OS-level APIs? Having constructors from a few possible argument types (e.g. double, int seconds + int <subseconds type TBD>) seems to make sense. As far as the appropriate subseconds type goes, we should probably pick the highest-possible resolution that makes sense, which I'd contend is probably microseconds. Some operating systems may be able to slice time (and signal events) at resolutions below milliseconds, but I doubt any can go deeper than microseconds.
Perhaps "address_specifier" would be better? I can imagine some sort of "stream factory" class with pluggable protocols (e.g. http, https, ftp, etc) that would take fully qualified URLs to create new streams, but in this case you're dealing with at most a hostname and port.
Well, the semantics of such things quickly become non-streams. The intention of the scheme ("http") is to establish a port by means of name lookup. In the end, yes the URL is resolved to address/port. If an HTTP library was built on top, it would do no transformation before passing its URL to new_address(). Hence, the type "url".
I think I see your point here, but I'm not 100% convinced that this is the right place to put url. It seems to me that the concept of URL belongs at a higher level, since the only portion you can use at the network level is a hostname and port. It might be expedient for a high-level HTTP library to be able to pass URLs all the way down to the network level, but it would be nearly as easy for that library to make use of its own URL object which had methods for extracting the "address_specifier" information. Does anyone else have an opinion on this?
I'd contend that in general, an operating system supplied multiplexing facility will scale better than one that uses a thread to handle each connection.
Absolutely. The code I have in mind uses a thread pool and connections are managed by as few a number of threads as necessary. On Windows, each I/O thread can handle 64-ish connections. On Unix, it is much higher. By default, I create one I/O thread per processor and divvy up the connections.
Without some sort of multiplexing facility, how do you know when a channel is ready for I/O? It seems that your proposal is to use a pool of threads to handle async/non-blocking operations, but I don't see any interfaces defined to control or manage these operations. Is that just TBD?
I have contemplated just how a user would want to interact with the thread pool behind the scenes, but, it is ideally an issue of minimal concern. Of course, for advanced users it could be something that needs to be tweaked. In that sense, there is a TBD aspect. I just have yet to see a good abstraction/mechanism for doing this.
OK, having read a number of your posts on this subject, I think I am starting to understand your position w/r/t event dispatch. Correct me if I'm wrong, but your contention is that these mechanisms are best left hidden from the user, and the interface should expose only synchronous operations or async/non-blocking operations with some sort of callback mechanism. How those are implemented is not exposed. This is a bit of a paradigm shift for someone (e.g. me) who is comfortable with select and fd_sets, etc, and even some of the higher level abstractions like ACE_Reactor. But I can see the value in this hiding approach and might warm to it if the implementation is easy to use and performs well.
I think the point I am trying to make is that there isn't necessarily one right answer. Pools of threads are good for some things, and I/O multiplexing facilities are good for some things. And in some cases, taken both together they are a good thing as well.
The threads in this pool are only doing I/O multiplexing, so I'm not sure I understand your concern. The kind of thread pool in my implementation is just to have roughly the minimum number of threads compared to connections based on multiplexing limits. Another kind of thread pool would be one/two per connection to handle blocking activity. The blocking methods in the proposal could be used when that was the right answer. Or did I miss your point here?
The point was several paragraphs of navel-gazing about how the underlying multiplexing implementation might work. Clearly you understand the mechanisms involved, so my dialectic was wasted :) Anyway, yours is a new approach to me, but I think I can see its value and would be interested in trying it out. -- Caleb Epstein caleb dot epstein at gmail dot com