Hi Andy, --- Andy Little <andy@servocomm.freeserve.co.uk> wrote:

Attempting to compile the synchronous timer example. I get the message xx\boost\asio\detail\socket_types.hpp "sys/ioctl.h" no such file or directory. compiler VC7.1 in Windows XP. Any thoughts?

It can only get into the #else branch that includes sys/ioctl.h if BOOST_WINDOWS is not defined. This would occur if BOOST_DISABLE_WIN32 is being defined somewhere, such as when compiling with the /Za option. If you are please try removing it. Cheers, Chris

--- Andy Little <andy@servocomm.freeserve.co.uk> wrote:

I am currently looking at the asynchromous timer tutorial example. I'm not clear where the timer thread starts. Does it start with the call to t.async_wait(print);

No, the thread is started in the deadline_timer's constructor. To be more specific, it is started in the first deadline_timer constructed on a given demuxer, by the deadline_timer_service that they all share, and the thread is shared between all timers created on that demuxer (as well as currently being used for any asynchronous socket connect operations too). Note that this info is for Win32 only. Please see this page for details: http://asio.sourceforge.net/boost-asio-proposal-0.3.6/libs/asio/doc/design/d... (You can find it in your local copy of the documentation by clicking "Design" in the top right hand corner and then clicking "Platform-Specific Implementation" from the list.) Cheers, Chris

Andy Little wrote:

Hi Cristopher,

Some impressions of the Asio library, bearing in mind that I am a complete novice in this area, so the impressions are written from an ignorant viewpoint!

For what it's worth, I must concur with Andy on this one ... I too find myself a novice in this domain, and while I do follow the examples, I find that there is a disturbing lack of motivation in the documentation as to why we do this or that. That being said, I realize that we are speaking of a discipline that is learnt with study and a great deal of practise, and so, I also don't expect the documentation to be a substitute for much of that either. Nonetheless, I reach very much the same conclusions as Andy had done, and for what it is worth I also vote Yes, for inclusion of Asio into Boost - if only on the back of others praise for the work. I think Christopher has done an admirable job on this work, and it still amazes me that Asio remains a header only implementation - it certainly is a most comendable effort. ... I certainly like to think that I'll use this library in future, even if only for my own erudition in the subject. Great job Chris!

[snip ... pertinent dialogue of Andy's]

Cheers, -- Manfred Doudar MetOcean Engineers www.metoceanengineers.com

Victor A. Wagner Jr. wrote:

I think we _could_ have picked a worse time to have a review if we'd only tried a little harder. How about delaying it a week and _really_ hitting the holidays... le't's see, we could close the formal review at the end of the Rose Bowl game just for pizazz.

Sorry guys, I think this is one of the worst times to try to get volunteers to do anything, let alone a serious job of reviewing a new library. IMO, it doesn't matter if the library is perfect, there are enough of us< who understand the problems thoroughly, who just won't be able to cobble together the time to do an honest review.

So, given that I can't review it, I'll vote NO now (I can't vote YES on anything I haven't read (in stark contrast to the entire House of Representatives in the U.S. Congress)).

I do appeal to delay the actual review until AFTER New Years Day.

I understand it is a bad time. So is the whole summer when people are off on vacations. In short, it's always a bad time for someone. Personally I think one hour should be enough to do a basic review focused on one aspect -- say documentation -- or examples. I think 3 hours should be enough for an in-depth review. Also, I've done some off-list recruiting for review submissions and I'm hopeful that we will have good coverage. Finally, if the end of the review is reached and it is insufficient then the library will not be accepted and we'll have to work something else out. Jeff

On 12/10/05, Victor A. Wagner Jr. <vawjr@rudbek.com> wrote:

Sorry guys, I think this is one of the worst times to try to get volunteers to do anything, let alone a serious job of reviewing a new library. IMO, it doesn't matter if the library is perfect, there are enough of us< who understand the problems thoroughly, who just won't be able to cobble together the time to do an honest review.

So, given that I can't review it, I'll vote NO now (I can't vote YES on anything I haven't read (in stark contrast to the entire House of Representatives in the U.S. Congress)).

This is a complete cop-out and an insult to the people who *do* plan to spend time reviewing and using this library, not to mention its author. I personally plan to write a review of asio and have been spending time using it in advance of this. A modern C++ networking library is one of the most fundamental pieces of functionality from Standard C++. Reviewing this library is therefore worth *anyones* time, particularly someone who "understand the problems thoroughly". Perhaps you are unable to devote the necessary time to write a formal review, but to then cast a backhanded vote rejecting it without any analysis is offensive and spiteful. As suchm I don't think this vote should be counted towards any formal tallys. I am sure there will be enough thoughtful reviews and criticism from other Boosters who can and will spend the time necessary, that this formal review will be completed before the Grinch hits town. -- Caleb Epstein caleb dot epstein at gmail dot com

Reece Dunn wrote:

Also, what would be interesting is an asynchronous serialisation archive so you can serialise/deserialise classes across a network like you can in Java.

There is an example that uses boost.serialization in conjunction with asio to facilitate client/server data communications: http://asio.sourceforge.net/boost-asio-proposal-0.3.6/libs/asio/doc/examples... The core of the code using serialization is here: http://asio.sourceforge.net/boost-asio-proposal-0.3.6/libs/asio/doc/examples... Jeff

On 12/10/05, Reece Dunn <msclrhd@hotmail.com> wrote:

Jeff Garland wrote:

...

All -

Today (Dec 10th) is the start of the format review of the Asynchronous I/O library (asio) library by Christopher Kohlhoff. The review will run until Friday December 23rd. I will be serving as review manager.

I am not that well versed in the target domain, but I do have a basic understanding of how everything works.

As has been mentioned, having a (standard) socket API will greatly help C++ compete with other languages that have extensive libraries such as Java.

I have only taken a cursory glance at the docs at the moment. From what I have read, there is built in support for TCP and UDP I/O, but you could extend this to support other transports such as asynchronous files or named pipes. Is the interface generic enough so I can do this easily? How would I provide this functionality? (If this is in the docs, please tell me to RTFM ;) hopefully with a link to where in the docs it is).

For what I worked with the library (which is very little), it is very straightforward to add support for other transports. I have implemented part of a asynchronous file IO with the AIO. Unfortunately I havent found time to continue working on this.

Another question: is there a way I can bind a socket to an I/O stream so I can easily send data across a network.

IMO, Asynchronous IO doesnt couple very well with the concept of streams. What you're probably looking for is the network library of Pedro Lamarão, which is basically an network stream library (had even suggestions to change the name of library to reflect this).

Also, what would be interesting is an asynchronous serialisation archive so you can serialise/deserialise classes across a network like you can in Java.

See the library I told above. It's probably straightforward to achieve that. [comments about the documentation]

- Reece

best regards, -- Felipe Magno de Almeida

Hi, --- christopher baus <christopher@baus.net> wrote:

I'm currently trying to port an app from libevent to asio to test out the library. I have a quick question. Why doesn't the basic_stream_socket::async_connect handler pass a pointer to the socket? Is it assumed that the handler is a stateful functor and keeps a pointer to the socket?

The intention is that you use boost::bind to create the appropriate function object: void connect_handler(const boost::asio::error& error, boost::asio::stream_socket* socket) { if (!error) { s->async_read_some(...); } } int main() { boost::asio::stream_socket* socket = ...; ... socket->async_connect(endpoint, boost::bind(connect_handler, _1, socket) ... // Or if you don't want to remember the index of the // arguments: // socket->async_connect(endpoint, // boost::bind(connect_handler, // boost::asio::placeholders::error, socket)); } Cheers, Chris

Simon said

...

Just wanting to reiterate my view that the asio::demuxer could use a modification to support a customisation for Handler execution on the win32 message pump. [SNIP]

Chris said

I'm of the opinion that asio::demuxer isn't the right place for this. Instead it should be an implementation of the Dispatcher concept. [SNIP] I think the appropriate place to choose how the handler should be delivered is the point where an asynchronous operation is started. For example:

async_read(socket, buffers, locking_dispatcher.wrap(handler));

And following that model, async_read(socket, buffers, win32_pump_dispatcher.wrap(handler)); I agree that would work, so maybe thats good enough. (By which i mean, very good, like the rest of the lib :) However, locking_dispatcher has fairly simple innards, and a win32_pump_dispatcher would require an implementation thats a lot like asio::demuxer -i.e. with an internal queue of Handlers. Opportunity for re-use?

But the primary purpose of the demuxer is as an I/O demultiplexer, so the way it uses threads is aimed specifically at that use case, and it's not really suitable for customisation in the way you describe.

I can't see anything in the demuxer interface which limits it purely this role. Its applicability seems entirely general. (abstracted deferral of Handlers, which in reality can be 'handling' anything). In a program that mixes asio with other callback-originating libraries its attractive to extend the asio thread model to those other libraries as well. Thanks again Simon

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote

--- Arkadiy Vertleyb <vertleyb@hotmail.com> wrote:

...

Is it possible to obtain a printable (single document) version of the library documentation?

I'll see if I can get doxygen to generate something reasonable via latex. However it will probably be four separate documents (reference, tutorial, examples, design) rather than one for the time being.

This would be fine, thanks. I just want to use my commute time to look through your docs before attempting to play with the library. Regards, Arkadiy

Hi Christopher, "Christopher Kohlhoff" <chris@kohlhoff.com> wrote

I've just put up a zip file containing 4 pdf files:

http://prdownloads.sourceforge.net/asio/asio_printable_docs.zip?download

Thanks. Now that I spent some time reading through you tutorial I have a question -- what exactly is the demuxer object? At first I thought that it somehow relates to the "demultiplexer" pattern. However, considering a simple TCP client, which just connects to the server, sends a request, and then reads the response, I don't see how demultiplexer is related here. If I write a similar program using winsock, I can't see anything analogous... Can you ellaborate on this? Thanks, Arkadiy

"Eugene Alterman" <eugalt@verizon.net> wrote

"Arkadiy Vertleyb" <vertleyb@hotmail.com> wrote in message news:dnmui9$5jr$1@sea.gmane.org...

...

Hi Christopher,

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote

...

I've just put up a zip file containing 4 pdf files:

http://prdownloads.sourceforge.net/asio/asio_printable_docs.zip?download

Thanks.

Now that I spent some time reading through you tutorial I have a question -- what exactly is the demuxer object? At first I thought that it somehow relates to the "demultiplexer" pattern. However, considering a simple TCP client, which just connects to the server, sends a request, and then reads the response, I don't see how demultiplexer is related here. If I write a similar program using winsock, I can't see anything analogous...

Can you ellaborate on this?

A hint: What do you think 'asio' stands for?

A question: then why does the tutorial have such examples as "A _synchronous_ TCP daytime client", etc., that also use the demuxer? Does it mean the library emulates synchronous operations using asynchronous facilities? Regards, Arkadiy

"Eugene Alterman" <eugalt@verizon.net> wrote

See "Services and the Demuxer" in asio_design doc. It turns out demuxer also serves as a service repository. I personally don't like that very much and wonder why those 2 different functionalities are lumped together.

So the demuxer is a service repository + demultiplexer(?)... If this is the case, I agree that they don't seem to belong together. Since, according to the design doc, a service is a "wrapper around a platform's implementation of the resource", the "service repository" part of the demuxer seems to be an abstraction layer over the platform. In this case, is the runtime object really neaded for this? Can't this functionality be provided at compile time, through a template parameter to the socket and other classes? (This, of course, would not work if this service repository can somehow change at runtime. Is this the case?) Regards, Arkadiy

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote in message news:20051213205829.28944.qmail@web32611.mail.mud.yahoo.com...

In the past I have considered splitting them into two separate objects, however opted not to because I felt it impacted negatively on usability. Your typical user doesn't want to know about the services (see the many requests for me to split the reference documentation into two parts so that the services and basic_* templates are not intrusive).

Have you considered making default service repository a singleton? That would make it transparent to a user.

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote

--- Arkadiy Vertleyb <vertleyb@hotmail.com> wrote:

...

Now that I spent some time reading through you tutorial I have a question -- what exactly is the demuxer object? At first I thought that it somehow relates to the "demultiplexer" pattern. However, considering a simple TCP client, which just connects to the server, sends a request, and then reads the response, I don't see how demultiplexer is related here. If I write a similar program using winsock, I can't see anything analogous...

As Eugene said it does act as a sort of service repository. In the past I have considered splitting them into two separate objects, however opted not to because I felt it impacted negatively on usability. Your typical user doesn't want to know about the services (see the many requests for me to split the reference documentation into two parts so that the services and basic_* templates are not intrusive).

Therefore I prefer to talk about the demuxer as a multi-faceted object, not completely unlike std::locale. Its primary facet (i.e. the one accessible via asio::demuxer) is the one that everybody must use if they are going to do any asynchronous operations at all, regardless of whether they use sockets, timers or something else.

Can your faucets change at runtime?

As for why the sockets must take the demuxer object even if you are just doing synchronous operations... I mean you could have a socket implementation that excluded the asynchronous operations,

Why is the ability to perform asynchronous operations a property of socket?

and a demuxer wouldn't be required in that case. But ultimately this library is about asynchronous I/O.

I can see some contradiction here, since you tutorial provides a number of syncronous examples. Using the demuxer object in such contexts seems to be a conceptual overhead, and somewhat misleading. Also, your library provides some socket API, and therefore, if it is accepted, I don't expect any other socket class in Boost in the nearest future, which means I would expect your classes to nicely wrap all of sockets, not only acynchronos ones. Regards, Arkadiy

...

Also, your library provides some socket API, and therefore, if it is accepted, I don't expect any other socket class in Boost in the nearest future, which means I would expect your classes to nicely wrap all of sockets, not only acynchronos ones.

I've been thinking about these issues since your email. My feeling is that a use case doesn't have to become very complicated for the asynchronous operations to be beneficial. But where a developer's needs are simple enough, I suspect that's also when they are going to want to use an iostreams-based interface. Since iostreams basically enforce synchronous operation, this seems like a sensible place to hide the demuxer and asynchronicity away.

I strongly, strongly agree. You can build a robust synchronous implementation, iostream based or not, on top of an asynchronous foundantion. The reverse is difficult and tends to be non-scalable. Dave Moore

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote

--- Arkadiy Vertleyb <vertleyb@hotmail.com> wrote:

...

Can your faucets change at runtime?

No.

Than have you considered putting them in a type rather than in an object? I mean defining them as static functions inside a class, and then providing the typedef to this class as a part of your configuration, depending on the platform used? This is better than singleton, since your library could remain header-only. I realise that would not work for demuxer (demultiplexer?), but I am still thinking this is not the best idea to mix it with the platform traits.

...

Why is the ability to perform asynchronous operations a property of socket?

I chose to make asynchronous operations part of asio's socket interfaces because, in my opinion, having them there offers the best ease of use and gives maximum flexibility to provide efficient implementations portably.

See, this doesn't look intuitive to me. I am by no means a networking expert, but I have done some work with sockets. And, IMO, a socket doesn't have to do anything with being asynchronous-enabled or not. Asynchronity is about how the socket is used, not about the socket itself. If you split demuxer from platform traits, and free the socket implementation from the asynchronisity, moving it outside, than all the socket needs to know about is platform traits. This could be used as I suggested above (through the typedef), and your socket constructor would not have to accept any parameters at all.

...

I can see some contradiction here, since you tutorial provides a number of syncronous examples. Using the demuxer object in such contexts seems to be a conceptual overhead, and somewhat misleading.

With this tutorial, I was trying to demonstrate the library's concepts in stages, where each subsequent tutorial introduces something new. I think it's important that, as you move through the tutorial, you can see how the synchronous is mapped to the asynchronous. So the early examples simply don't use all of the functionality that's on offer. They still have to initialise the objects correctly according to the library's interface though.

...

Also, your library provides some socket API, and therefore, if it is accepted, I don't expect any other socket class in Boost in the nearest future, which means I would expect your classes to nicely wrap all of sockets, not only acynchronos ones.

I've been thinking about these issues since your email. My feeling is that a use case doesn't have to become very complicated for the asynchronous operations to be beneficial. But where a developer's needs are simple enough, I suspect that's also when they are going to want to use an iostreams-based interface. Since iostreams basically enforce synchronous operation, this seems like a sensible place to hide the demuxer and asynchronicity away.

Well, personally, I would still like to see a nice socket API, as a part of any networking library accepted into Boost. And I think it should be closely mapped (conceptually) to existing socket APIs, such as Berkeley, winsock, etc., maybe Java sockets. The rest of the library, which implements higher-level abstractions (such as demuxer), could depend on this layer, but not visa-versa. Regards, Arkadiy

"Arkadiy Vertleyb" <vertleyb@hotmail.com> wrote in message news:dnpcuo$qjn$1@sea.gmane.org...

...

Than have you considered putting them in a type rather than in an object? I mean defining them as static functions inside a class, and then

"Eugene Alterman" <eugalt@verizon.net> wrote in message providing

...

the typedef to this class as a part of your configuration, depending on the platform used? This is better than singleton, since your library could remain header-only.

Are you implying that a singleton cannot have a header-only implementation?

I was under such impression... But now when I tried this, looks like Meyers singleton works OK, at least in VC71.

...

...

...

Why is the ability to perform asynchronous operations a property of socket?

I chose to make asynchronous operations part of asio's socket interfaces because, in my opinion, having them there offers the best ease of use and gives maximum flexibility to provide efficient implementations portably.

See, this doesn't look intuitive to me. I am by no means a networking expert, but I have done some work with sockets. And, IMO, a socket doesn't have to do anything with being asynchronous-enabled or not.

Asynchronity

...

is about how the socket is used, not about the socket itself.

But asynchonous i/o implementation does depend on the underlying socket API (platform traits, as you call it). Some implementations prvovide asynchronous operations while on others they need to be emulated by non-blocking socket calls and a reactor framework.

Right. Still doesn't explain while socket (a low-level abstraction) should depend on demultiplexer (a high-level abstraction), does it? Regards, Arkadiy

Hi, Arkadiy Vertleyb wrote...

There are cases when having multiple threads, each working in a synchronous way, is a better approach. [...]

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote

...

Since iostreams basically enforce synchronous operation, this seems like a sensible place to hide the demuxer and asynchronicity away.

Just one question: Would it be possible to "hide away" the demuxer and asynchronicity in a thread-safe way? I would like to use synchronous socket streams in a multi-threaded application. Best regards, Tilman

"Peter Dimov" <pdimov@mmltd.net> wrote

Arkadiy Vertleyb wrote:

...

Let me disagree with this. There are cases when having multiple threads, each working in a synchronous way, is a better approach. First, it's much more intuitive, and easier to debug. Second, sometimes it's the only way to achieve you goal, such as whan you can't (or don't want to) rewrite your processing algorithm in asynchronous way.

And when such a case arrives, it would be nice to have a clean socket class, without build-in asynchronisity.

Even if this is the case, is it the responsibility of a library that offers Asynchronous I/O as a primary focus to supply a clean socket class that has nothing to do with async I/O?

In general, no. But when the library also offers syncronous operations, then yes, I would like this to be implemented cleanly, rather than having asyncronous stuff, even if hidden underneath. Also, let's consider a broader picture. Network programming consists of both synchronous and asynchronous approaches, sometimes mixed together. One can start with one approach, and then switch to another. Maybe I got a wrong impression, but to me it looks like asio is offered (or perceived) as something like Boost Networking Library. And as such, I would expect clean sockets from it. Otherwise, who will provide them? Do we expect another Socket class to be reviewed any time soon? Regards, Arkadiy

On Wed, 14 Dec 2005 12:05:10 -0500 Stefan Seefeld <seefeld@sympatico.ca> wrote:

I think ideally a (network) stream API would thus be layered on top of asio. In fact, if the low level 'transport protocol' is being made a policy, the stream API could be parametrized to only depend on asio if async operations are requested.

However, many of us desire synchronous handling, but want nothing to do with sockets encapsulated as IOStreams.

Arkadiy Vertleyb wrote:

I would expect both asio and a stream API to be layered on top of low-level socket API. This socket API should wrap the platform dependency, but otherwise provide pretty much the same capabilities as current C socket APIs provide, with minimum overhead.

Yes. Though I'm not sure a thin wrapper around the platform API adds much value at all. Too big are the differences. And besides, why should people really care whether the thing is a socket, as long as the semantics are correct ? Regards, Stefan

"Arkadiy Vertleyb" <vertleyb@hotmail.com> writes:

[snip]

I would expect both asio and a stream API to be layered on top of low-level socket API. This socket API should wrap the platform dependency, but otherwise provide pretty much the same capabilities as current C socket APIs provide, with minimum overhead.

The key problem with that approach is that a `thin' wrapper over the platform APIs is basically useless for implementing asynchronous operations, because asynchronous operations must be implemented in widely different ways on the different platforms. For synchronous operations, it happens to be the case that every platform provides basically the same interface, so at least a portable (but not necessarily very good) interface can be created through only a thin wrapper. -- Jeremy Maitin-Shepard

Arkadiy Vertleyb wrote:

My problem with the current approach is that synchronous operations seems to be viewed as second-class citizens. I understand that this is caused by the fact that asio is about the asynchronous IO. But asio is presented as the "best C++ networking library around", and in networking both synchronous and asynchronous approaches are used, and neither is superior for all possible cases.

Consider "Patterns for concurrent and networked objecs" by D. Schmidt, at al. Quite a few patterns described there are related to the synchronous processing.

I think a little refactoring might lead to a better solution than trying to add synchronous operations on top of existing asio.

I think that we should be careful not to break something that has been proven to work well because a little refactoring "might" lead to a better solution (unless the author agrees, of course.) The argument that asio is presented as "the best networking C++ library around" is not enough of an excuse.

"Arkadiy Vertleyb" <vertleyb@hotmail.com> wrote in message news:dnq894$11t$1@sea.gmane.org...

All I was trying to say is that, as a potential client of the library (everybody does some networking now and then), I tried to map it to one of the tasks I had in the past. I happen to have solved that task by using synchronous operations + multiple threads (maybe because it is the most intuitive way for a not very experienced network programmer, which I am, but I still doubt very much that asynchronous approach would have been better).

If you are using socket API or a C++ library that just provides simple wrapper classes for socket API a thread per conection approach is of course the most intuitive. The problem is that it does not scale well.

So, when I am evaluating a "Networking Library", and see that it clearly consideres one approach to networking inferior to the other one, and I think they both are equaly important, that means that I am in fundamental disagreement with the library author on the subject, and makes me wonder whether this is the networking library I would like to see in Boost.

One approach is inferior to the other and they are both important (but not equally) :-)

Besides nobody yet convinced me that socket should depend on demultiplexer.

It does not. The real reason it depends on demuxer is that it contains the socket service. Otherwise demultiplexer could have been passed as an argument only when performing asynchronous operations.

The fact that it "has been proven to work well" is "not enough of an excuse", IMO. Otherwise this review would not make any sense.

I totally appreciate the fact that the library has a lot of positive reviews, and, as you say, proven to work well. All I want is that the synchronous approach to networking was given enough attention, and added cleanly rather than as a fast fix -- a wrapper around asynchronous stuff.

It is not.

Arkadiy Vertleyb wrote:

"Eugene Alterman" <eugalt@verizon.net> wrote

...

...

So, when I am evaluating a "Networking Library", and see that it clearly consideres one approach to networking inferior to the other one, and I think they both are equaly important, that means that I am in fundamental disagreement with the library author on the subject, and makes me wonder whether this is the networking library I would like to see in Boost.

One approach is inferior to the other and they are both important (but not equally) :-)

Is it a proven fact, or just your opinion? :-)

Proven fact. Thread per connection doesn't scale.

"Arkadiy Vertleyb" <vertleyb@hotmail.com> wrote in message news:dnq894$11t$1@sea.gmane.org...

...

So, when I am evaluating a "Networking Library", and see that it clearly consideres one approach to networking inferior to the other one, and I think they both are equaly important, that means that I am in fundamental disagreement with the library author on the subject, and makes me wonder whether this is the networking library I would like to see in Boost.

"Inferiority" is not the point here. The reason asynchronous interface is given more attention is that it is

"Eugene Alterman" <eugalt@verizon.net> wrote the

one that is non-trivial. The real value of this library is the asynchronous part.

OK

There is nothing special about synchronous operations, and they have been implemented in similar ways in dozens of networking libraries.

See, I don't care much about other libraries -- I am not going to use them. But if this one becomes part of Boost, I may start using it, and so I would like it to have what I need, with a clean interface, and implemented in a clean way. Regards, Arkadiy

"Arkadiy Vertleyb" <vertleyb@hotmail.com> wrote:

...

"Eugene Alterman" <eugalt@verizon.net> wrote

...

There is nothing special about synchronous operations, and they have been implemented in similar ways in dozens of networking libraries.

See, I don't care much about other libraries -- I am not going to use

"Ben Artin" <macdev@artins.org> wrote them.

...

But if this one becomes part of Boost, I may start using it, and so I would like it to have what I need, with a clean interface, and implemented in a clean way.

IMO this is completely beside the point. This is not the boost networking library, it's the boost async IO library, and as such whether it supports sync IO or not is completely irrelevant.

When I was reading through the library tutorial half of examples there were related to the synchronous IO. And, in all these examples a demuxer object was used, that has nothing to do with synchronous IO. Do you think this is a clean interface? Do you think having to create and pass around an object, that is conceptially not required, is irrelevant? If the library doesn't want to support synchronous IO this is fine, but since it does support it, how it is supported becomes relevant.

There are really two possibilities here:

1) You may some day use an async IO library, in which case this library will be useful to you and you should review it for its merit with regards to its stated design intent or 2) You will never use an async IO library, in which case this library will never be useful to you, and you should still review it for its merit with regards to its stated design intent, while acknowledging that it does not necessarily fit your problem domain.

However, right now you seem to saying that you are not interested in async IO, and that this library, whose stated intent is to provide async IO, is

I was under impression that I am reviewing it right now :-) There is a third possibility, which IMO is more probable, that I will want to use some combination os sync and async features. That's why I would like to see them nicely fit together. therefore

unacceptable to you.

This is a total mis-interpretation of what I have been saying.

You have a nail. This library is not a hammer.

IMO, if you want a clean synchronous IO API, you should not be looking for it in an async IO library.

If the lirary has the support for synchronous IO API, it should be clean. Either have a clean support or no support at all. Regards, Arkadiy

Arkadiy Vertleyb wrote:

All I was trying to say is that, as a potential client of the library (everybody does some networking now and then), I tried to map it to one of the tasks I had in the past. I happen to have solved that task by using synchronous operations + multiple threads (maybe because it is the most intuitive way for a not very experienced network programmer, which I am, but I still doubt very much that asynchronous approach would have been better). So, when I am evaluating a "Networking Library", and see that it clearly consideres one approach to networking inferior to the other one, and I think they both are equaly important, that means that I am in fundamental disagreement with the library author on the subject, and makes me wonder whether this is the networking library I would like to see in Boost.

The "thread per connection" model _is_ inferior on all existing platforms. If asio doesn't encourage its use, I consider this a feature.

Arkadiy Vertleyb wrote:

"Peter Dimov" <pdimov@mmltd.net> wrote

...

The "thread per connection" model _is_ inferior on all existing platforms. If asio doesn't encourage its use, I consider this a feature.

I am not sure what exactly the definition of "thread per connection" model is, but I assume this means a new thread is _created_ for every connection (?)

Right.

What I did was to have the main thread accept incomming requests, and put them in the queue, whereas a number of worker threads was taking these requests from the queue, and execute them. I don't see how this model can be absolutely inferior in all possible contexts.

That is usually referred to as 'Thread Pool', and it is usually considered the 'best' strategy among the multi-threaded ones, in particular due to its scalability. Regards, Stefan

"Stefan Seefeld" <seefeld@sympatico.ca> wrote in message news:43A1AFF4.1070507@sympatico.ca...

Arkadiy Vertleyb wrote: [snip]

...

What I did was to have the main thread accept incomming requests, and put them in the queue, whereas a number of worker threads was taking these requests from the queue, and execute them. I don't see how this model can be absolutely inferior in all possible contexts.

That is usually referred to as 'Thread Pool', and it is usually considered the 'best' strategy among the multi-threaded ones, in particular due to its scalability.

Well, this particular concurrency model utilizing thread pool is the simplest but not the best. Ever heard about Leader/Followers pattern?

"Thore Karlsen" <sid@6581.com> wrote

...

What I did was to have the main thread accept incomming requests, and put them in the queue, whereas a number of worker threads was taking these requests from the queue, and execute them. I don't see how this model can be absolutely inferior in all possible contexts.

How are you reading and writing from multiple sockets simultaneously in your main thread?

I was not. I was reading/writig from the worker threads. The main thread was used only to accept connections. Regards, Arkadiy

Thore Karlsen wrote:

...

...

How are you reading and writing from multiple sockets simultaneously in your main thread?

...

I was not. I was reading/writig from the worker threads. The main thread was used only to accept connections.

So in that case you are handling one connection per thread, which does not scale well.

I'm sorry, but your conclusion seems wrong. If a dispatcher thread puts requests into a queue for worker threads to consume, it's most definitely *not* a thread-per-connection design, but a thread pool. It would be a thread-per-connection if threads were created as a result of new connection requests being put into the queue. But that he (apparently) isn't doing. Regards, Stefan

On Thu, 15 Dec 2005 13:41:32 -0600 Thore Karlsen <sid@6581.com> wrote:

You seem to be reading this differently than I am. To me it looks like he's saying that he has one thread that accepts connections, and then he passes the socket to a worker thread where he does all the synchronous reading and writing to service the client.

He may have the threads already standing by when he accepts new connections, but since he's doing synchronous I/O he can't service more than one connection at a time from a thread. Thus the number of connections he can handle is limited by the number of threads he has, which is why there's a scalability problem.

You do not understand. This is a typical technique for thread pools. Each thread only performs the task requested, and the next task may be on a totally different connection. The number of connections is only limited by the operating system. The number of connections that can be serviced "simultaneously" is limited by the number of threads in the thread pool (please note the quotes around simultaneously -- I am fully aware of the actualities).

On Thu, 15 Dec 2005 14:23:34 -0600 Thore Karlsen <sid@6581.com> wrote:

And how do you presume that Arkadiy is doing that when he's doing all synchronous communication in the worker threads? How can one of his threads service another connection when it's blocked waiting for data from the client, or blocked sending data to a client?

If he is truly servicing multiple connections from a single thread using blocking I/O I'd love to know how he's doing it.

This thread of discussion has gone way past its intent, and I'm not sure this is the place for discussions on thread pool implementations. However, just to give you something to think about... Why does the main thread have to do any I/O at all? One could easily conceive a thread that looks for work to be done, then sends that work to an available thread for processing, without ever performing any I/O. Or, a thread pool that "passes a token" representing the responsibility of finding work. It finds some work to do, then passes the token to another thread before actually doing the work. There are many (some better than others) ways of accomplishing similar things without having one thread do all the input...

On Thu, 15 Dec 2005 16:01:20 -0600 Thore Karlsen <sid@6581.com> wrote:

My point is that once a thread is given a connection to work with, it can't do anything else while that connection is alive. Since all communication is synchronous, the thread will either be processing a request, or it will be blocked trying to read or write. It can only handle one connection at a time, and it has to handle that connection for the lifetime of the connection.

This obviously doesn't scale. If you want to handle 1000 connections simultaneously, you need 1000 threads.

Note that I'm not disagreeing with you in how thread pools should be used in general, I'm only addressing this specific scenario.

But... the thread only owns the connection for the life of one work unit (well, technically, if it were reading, it would read as much as possible, package the information, then process as much as possible). It would then release the "object" and become available for another piece of work. That piece of work may come from the same connection or another, it does not really matter. So, in this instance, I fail to see how it does not scale. It scales remarkably well, actually. I can handle 1000 connections with a thread pool of any size, even 1 (though that would defeat the purpose since all the work would pile up waiting for that one thread).

On Thu, 15 Dec 2005 22:26:26 -0600 Thore Karlsen <sid@6581.com> wrote:

Of course it matters. The client could sit idle for an hour before it sent the next request, and the server thread would be stuck in a blocking read waiting for data from the client, unable to service other clients.

This isn't the place to continue this discussion. I'll give a brief reply, but after that, it should probably go to private email. The server thread is not stuck. I think you are completely missing the point (more likely, I am not clearly explaining the point). The thread is never invoked until some amount of data was available.

So if you can do this with blocking, synchronous I/O, why do you think asio exists? Why do you think people are arguing for asynchronous I/O in favor of synchronous I/O?

Hmmm. I think you are mixing too much here. First, you are assuming stream-based IO, and synchronous IO implies blocking. With "record" based IO (datagrams for IP based stacks), all the data is available, or none is available. Again, if you wish to continue, use private email.

Arkadiy Vertleyb wrote:

"Thore Karlsen" <sid@6581.com> wrote

...

...

What I did was to have the main thread accept incomming requests, and put them in the queue, whereas a number of worker threads was taking these requests from the queue, and execute them. I don't see how this model

can

...

...

be absolutely inferior in all possible contexts.

How are you reading and writing from multiple sockets simultaneously in your main thread?

I was not. I was reading/writig from the worker threads. The main thread was used only to accept connections.

Thus limiting your #simultaneous connections to the number of threads in your thread pool. You can do better by doing all the parsing, request splitting, etc. in a async-using main acceptor thread and handing off smaller bits of work to the threads in the pool. That way you can exploit the fact that some threads may finish some parts of a given request early, and may be available to serve other connections. -- Bardur Arantsson <bardurREMOVE@THISimada.sdu.dk> <bardurREMOVE@THISscientician.net> - Peg, we've been married for 17 years now, can't we just be friends? Al Bundy / Married With Children

On Thu, 15 Dec 2005 14:57:18 -0500, "Arkadiy Vertleyb" <vertleyb@hotmail.com> wrote:

...

Thus limiting your #simultaneous connections to the number of threads in your thread pool. You can do better by doing all the parsing, request splitting, etc. in a async-using main acceptor thread and handing off smaller bits of work to the threads in the pool. That way you can exploit the fact that some threads may finish some parts of a given request early, and may be available to serve other connections.

Thus serving the same connection from different threads? I just don't happen to like how this would impact the processing algorithm...

You can do what I do, which is to always service the same connection from the same thread. Since my clients can pipeline requests, I don't want requests to be handled out of order. Assigning a connection to a thread in the thread pool is an easy way to solve that problem. Packets are read and parsed in the communications thread, and I send out smaller units of work to the thread pool. From the handlers that are executed in the thread pool I can queue up reply packets to be sent by the communications thread. This way I can connect thousands of clients and keep them connected all day, and service them all simultaneously from a very small number of threads. I could even do everything in one thread if I wanted to. I've done thread-per-connection servers as well, and to be honest I think an async design is easier to work with. If you have a good framework like asio, it can actually end up being cleaner. (At least to my eyes.) -- Be seeing you.

Hi Christopher, --- christopher baus <christopher@baus.net> wrote: <snip>

I do think that Arkadily has a point. For better or worse, much of the programming world has been trained to think about sockets synchronously, and eventually boost/C++ should address this.

But the question is how this should be addressed :) I happen to think that the fact many people have been trained to think about sockets synchronously is most definitely for the worse. Of course there are situations where synchronous is more appropriate than asynchronous, but it's the lack of awareness of asynchronous as an alternative that can lead to inferior design choices.

When that happens it would be nice to use the same fundamental types presented in asio, and these users shouldn't have to know about demuxers.

With asio, I was hoping to make asynchronous operations as natural and easy to use as possible, compared to their synchronous counterparts. I think that providing a synchronous-only socket interface is counterproductive. It reinforces the notion that synchronous operations are somehow more useful for the ordinary programmer, and that asynchronous is hard and best left in the realm of the networking guru.

I just want to point out that async reading and writing could be a function of an async I/O demuxer and not the socket itself. The socket could be passed to the demuxer and not vice versa.

The above design is something I can fundamentally disagree with ;) That sort of design, in my opinion, relegates asynchronous operations to "second class citizens". Asynchronous operations are no less part of a socket interface than their synchronous counterparts. A write operation sends the same data on a socket whether it is performed synchronously or asynchronously. It's simply that a synchronous call blocks the calling thread until the operation completes, whereas an asynchronous call executes on a logical background thread and tells you when it is complete. Another goal of asio is to provide a basis for further abstraction. For this I believe it is important to provide a clear correlation between the synchronous and asynchronous operations. Consider a protocol where messages have a fixed length header followed by a body, where the body length is contained in the header. The synchronous read operation might look something like: void read_message(Stream& s, message& m) { ... create buffer for header ... read(s, header_buffer); ... create buffer for body of correct size ... read(s, body_buffer); ... populate message structure ... } The equivalent asynchronous code would be: void async_read_message(Stream& s, message& m, Handler h) { ... create buffer for header ... async_read(s, header_buffer, bind(header_handler ...)); } void header_handler(error e, Stream& s, message& m, Handler h) { ... check for failure ... ... create buffer for body of correct size ... async_read(s, body_buffer, bind(body_handler ...)); } void body_handler(error e, Stream& s, message& m, Handler h) { ... check for failure ... ... populate message structure ... h(e); // indicate that operation is complete } There is a relatively straightforward mapping between one and the other. In the future I want to exploit this mapping further by investigating the use of expression templates (perhaps similar to Boost.Lambda) to permit the encoding of a sequence of operations in a synchronous programming style. The operations could then be executed either synchronously or asynchronously as needed. Cheers, Chris

Hi Arkadiy, --- Arkadiy Vertleyb <vertleyb@hotmail.com> wrote:

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote

...

With asio, I was hoping to make asynchronous operations as natural and easy to use as possible, compared to their synchronous counterparts.

I am afraid this might be impossible :-( People usually make/think about things sequentially.

I am more optimistic about people's abilities.

"1 2 3 4" is much more intuitive than "1 2(call 3 when finished(and 3 will schedule 4).

What makes you think that the above approach is the only way to use asynchronicity? :) It has been stated elsewhere in this thread that applications will generally either be synchronous or asynchronous. However, I mentioned in an earlier email that asio's design lets you develop mixed-mode programs. Your entire program does not have to centre around a call to demuxer::run() to benefit from the asynchronous operations. A predominantly synchronous program can still make use of asynchronous calls when concurrency is required. I have written an example (attached) for a protocol I will call AAoIP (ASCII Art over IP). This protocol uses a TCP control connection for managing subscriptions, and UDP for delivering the "frame" data. The client program is written synchronously and implements port hopping (like Skype). To ensure uninterrupted rendering of the ASCII art, the client uses asynchronous operations to continue to receive the frames while the (potentially long running) port renegotiation is in progress. The asynchronous operations are only used for the port renegotiation. Overall, the program flow is still synchronous. Concurrency without threads is what it's about. Cheers, Chris #include <boost/asio.hpp> #include <boost/lambda/lambda.hpp> #include <boost/lambda/bind.hpp> #include <boost/lambda/if.hpp> #include <boost/shared_ptr.hpp> #include <algorithm> #include <cstdlib> #include <exception> #include <iostream> #include <string> #include "protocol.hpp" using namespace boost; int main(int argc, char* argv[]) { try { if (argc != 3) { std::cerr << "Usage: client <host> <port>\n"; return 1; } using namespace std; // For atoi. std::string host_name = argv[1]; unsigned short port = atoi(argv[2]); asio::demuxer demuxer; // Determine the location of the server. asio::ipv4::host_resolver host_resolver(demuxer); asio::ipv4::host host; host_resolver.get_host_by_name(host, host_name); asio::ipv4::tcp::endpoint remote_endpoint(port, host.address(0)); // Establish the control connection to the server. asio::stream_socket control_socket(demuxer); control_socket.connect(remote_endpoint); // Create a datagram socket to receive data from the server. shared_ptr<asio::datagram_socket> data_socket( new asio::datagram_socket(demuxer, asio::ipv4::udp::endpoint(0))); // Determine what port we will receive data on. asio::ipv4::udp::endpoint data_endpoint; data_socket->get_local_endpoint(data_endpoint); // Ask the server to start sending us data. control_request start = control_request::start(data_endpoint.port()); asio::write(control_socket, start.to_buffers()); unsigned long last_frame_number = 0; for (;;) { // Receive 50 messages on the current data socket. for (int i = 0; i < 50; ++i) { // Receive a frame from the server. frame f; data_socket->receive(f.to_buffers(), 0); if (f.number() > last_frame_number) { last_frame_number = f.number(); std::cout << "\n" << f.payload(); } } // Time to switch to a new socket. To ensure seamless handover we will // continue to receive packets using the old socket until data arrives on // the new one. std::cout << " Starting renegotiation"; // Create the new data socket. shared_ptr<asio::datagram_socket> new_data_socket( new asio::datagram_socket(demuxer, asio::ipv4::udp::endpoint(0))); // Determine the new port we will use to receive data. asio::ipv4::udp::endpoint new_data_endpoint; new_data_socket->get_local_endpoint(new_data_endpoint); // Ask the server to switch over to the new port. control_request change = control_request::change( data_endpoint.port(), new_data_endpoint.port()); asio::error control_result; asio::async_write(control_socket, change.to_buffers(), lambda::var(control_result) = lambda::_1); // Try to receive a frame from the server on the new data socket. If we // successfully receive a frame on this new data socket we can consider // the renegotation complete. In that case we will close the old data // socket, which will cause any outstanding receive operation on it to be // cancelled. frame f1; asio::error new_data_socket_result; new_data_socket->async_receive(f1.to_buffers(), 0, ( // Note: lambda::_1 is the first argument to the callback handler, // which in this case is the error code for the operation. lambda::var(new_data_socket_result) = lambda::_1, lambda::if_(!lambda::_1) [ // We have successfully received a frame on the new data socket, // so we can close the old data socket. This will cancel any // outstanding receive operation on the old data socket. lambda::var(data_socket) = shared_ptr<asio::datagram_socket>() ] )); // This loop will continue until we have successfully completed the // renegotiation (i.e. received a frame on the new data socket), or some // unrecoverable error occurs. bool done = false; while (!done) { // Even though we're performing a renegotation, we want to continue // receiving data as smoothly as possible. Therefore we will continue to // try to receive a frame from the server on the old data socket. If we // receive a frame on this socket we will interrupt the demuxer, // print the frame, and resume waiting for the other operations to // complete. frame f2; done = true; // Let's be optimistic. if (data_socket) // Might have been closed by new_data_socket's handler. { data_socket->async_receive(f2.to_buffers(), 0, ( lambda::if_(!lambda::_1) [ // We have successfully received a frame on the old data // socket. Interrupt the demuxer so that we can print it. lambda::bind(&asio::demuxer::interrupt, &demuxer), lambda::var(done) = false ] )); } // Run the operations in parallel. This will block until all operations // have finished, or until the demuxer is interrupted. (No threads!) demuxer.reset(); demuxer.run(); // If the demuxer.run() was interrupted then we have received a frame on // the old data socket. We need to keep waiting for the renegotation // operations to complete. if (!done) { if (f2.number() > last_frame_number) { last_frame_number = f2.number(); std::cout << "\n" << f2.payload(); } } } // Since the loop has finished, we have either successfully completed // the renegotation, or an error has occurred. First we'll check for // errors. if (control_result != asio::error::success) throw control_result; if (new_data_socket_result != asio::error::success) throw new_data_socket_result; // If we get here it means we have successfully started receiving data on // the new data socket. This new data socket will be used from now on // (until the next time we renegotiate). std::cout << " Renegotiation complete"; data_socket = new_data_socket; if (f1.number() > last_frame_number) { last_frame_number = f1.number(); std::cout << "\n" << f1.payload(); } } } catch (std::exception& e) { std::cerr << "Exception: " << e.what() << std::endl; } return 0; } #include <boost/asio.hpp> #include <boost/bind.hpp> #include <boost/shared_ptr.hpp> #include <cmath> #include <cstdlib> #include <exception> #include <iostream> #include <set> #include "protocol.hpp" typedef boost::shared_ptr<boost::asio::stream_socket> stream_socket_ptr; typedef boost::shared_ptr<boost::asio::deadline_timer> deadline_timer_ptr; typedef boost::shared_ptr<control_request> control_request_ptr; class server { public: // Construct the server to wait for incoming control connections. server(boost::asio::demuxer& demuxer, unsigned short port) : acceptor_(demuxer, boost::asio::ipv4::tcp::endpoint(port)), timer_(demuxer), datagram_socket_(demuxer, boost::asio::ipv4::udp::endpoint(0)), next_frame_number_(1) { // Start waiting for a new control connection. stream_socket_ptr new_socket( new boost::asio::stream_socket(acceptor_.demuxer())); acceptor_.async_accept(*new_socket, boost::bind(&server::handle_accept, this, boost::asio::placeholders::error, new_socket)); // Start the timer used to generate outgoing frames. timer_.expires_from_now(boost::posix_time::milliseconds(100)); timer_.async_wait(boost::bind(&server::handle_timer, this)); } // Handle a new control connection. void handle_accept(const boost::asio::error& e, stream_socket_ptr socket) { if (!e) { // Start receiving control requests on the connection. control_request_ptr request(new control_request); boost::asio::async_read(*socket, request->to_buffers(), boost::bind(&server::handle_control_request, this, boost::asio::placeholders::error, socket, request)); // Start waiting for a new control connection. stream_socket_ptr new_socket( new boost::asio::stream_socket(acceptor_.demuxer())); acceptor_.async_accept(*new_socket, boost::bind(&server::handle_accept, this, boost::asio::placeholders::error, new_socket)); } else if (e == boost::asio::error::connection_aborted) { // Try again. acceptor_.async_accept(*socket, boost::bind(&server::handle_accept, this, boost::asio::placeholders::error, socket)); } } // Handle a new control request. void handle_control_request(const boost::asio::error& e, stream_socket_ptr socket, control_request_ptr request) { if (!e) { // Delay handling of the control request to simulate latency on the // network. deadline_timer_ptr delay_timer( new boost::asio::deadline_timer(acceptor_.demuxer())); delay_timer->expires_from_now(boost::posix_time::seconds(2)); delay_timer->async_wait( boost::bind(&server::handle_control_request_timer, this, socket, request, delay_timer)); } } void handle_control_request_timer(stream_socket_ptr socket, control_request_ptr request, deadline_timer_ptr delay_timer) { // Determine what address this client is connected from, since // subscriptions must be stored on the server as a complete endpoint, not // just a port. boost::asio::ipv4::tcp::endpoint remote_endpoint; socket->get_remote_endpoint(remote_endpoint); // Remove old port subscription, if any. if (unsigned short old_port = request->old_port()) { boost::asio::ipv4::udp::endpoint old_endpoint( old_port, remote_endpoint.address()); subscribers_.erase(old_endpoint); std::cout << "Removing subscription " << old_endpoint << std::endl; } // Add new port subscription, if any. if (unsigned short new_port = request->new_port()) { boost::asio::ipv4::udp::endpoint new_endpoint( new_port, remote_endpoint.address()); subscribers_.insert(new_endpoint); std::cout << "Adding subscription " << new_endpoint << std::endl; } // Wait for next control request on this connection. boost::asio::async_read(*socket, request->to_buffers(), boost::bind(&server::handle_control_request, this, boost::asio::placeholders::error, socket, request)); } // Every time the timer fires we will generate a new frame and send it to all // subscribers. void handle_timer() { // Generate payload. double x = next_frame_number_ * 0.2; double y = std::sin(x); int char_index = static_cast<int>((y + 1.0) * (frame::payload_size / 2)); std::string payload; for (int i = 0; i < frame::payload_size; ++i) payload += (i == char_index ? '*' : '.'); // Create the frame to be sent to all subscribers. frame f(next_frame_number_++, payload); // Send frame to all subscribers. We can use synchronous calls here since // UDP send operations typically do not block. std::set<boost::asio::ipv4::udp::endpoint>::iterator j; for (j = subscribers_.begin(); j != subscribers_.end(); ++j) { datagram_socket_.send_to(f.to_buffers(), 0, *j, boost::asio::ignore_error()); } // Wait for next timeout. timer_.expires_from_now(boost::posix_time::milliseconds(100)); timer_.async_wait(boost::bind(&server::handle_timer, this)); } private: // The acceptor used to accept incoming control connections. boost::asio::socket_acceptor acceptor_; // The timer used for generating data. boost::asio::deadline_timer timer_; // The socket used to send data to subscribers. boost::asio::datagram_socket datagram_socket_; // The next frame number. unsigned long next_frame_number_; // The set of endpoints that are subscribed. std::set<boost::asio::ipv4::udp::endpoint> subscribers_; }; int main(int argc, char* argv[]) { try { if (argc != 2) { std::cerr << "Usage: server <port>\n"; return 1; } boost::asio::demuxer d; using namespace std; // For atoi. server s(d, atoi(argv[1])); d.run(); } catch (std::exception& e) { std::cerr << "Exception: " << e.what() << std::endl; } return 0; } #ifndef AAOIP_PROTOCOL_HPP #define AAOIP_PROTOCOL_HPP #include <boost/array.hpp> #include <boost/asio.hpp> #include <cstring> #include <iomanip> #include <string> #include <strstream> // This request is sent by the client to the server over a TCP connection. // The client uses it to perform three functions: // - To request that data start being sent to a given port. // - To request that data is no longer sent to a given port. // - To change the target port to another. class control_request { public: // Construct an empty request. Used when receiving. control_request() { } // Create a request to start sending data to a given port. static const control_request start(unsigned short port) { return control_request(0, port); } // Create a request to stop sending data to a given port. static const control_request stop(unsigned short port) { return control_request(port, 0); } // Create a request to change the port that data is sent to. static const control_request change( unsigned short old_port, unsigned short new_port) { return control_request(old_port, new_port); } // Get the old port. Returns 0 for start requests. unsigned short old_port() const { std::istrstream is(data_, encoded_port_size); unsigned short port = 0; is >> std::setw(encoded_port_size) >> std::hex >> port; return port; } // Get the new port. Returns 0 for stop requests. unsigned short new_port() const { std::istrstream is(data_ + encoded_port_size, encoded_port_size); unsigned short port = 0; is >> std::setw(encoded_port_size) >> std::hex >> port; return port; } // Obtain buffers for reading from or writing to a socket. boost::array<boost::asio::mutable_buffer, 1> to_buffers() { boost::array<boost::asio::mutable_buffer, 1> buffers = { boost::asio::buffer(data_) }; return buffers; } private: // Construct with specified old and new ports. control_request(unsigned short old_port, unsigned short new_port) { std::ostrstream os(data_, control_request_size); os << std::setw(encoded_port_size) << std::hex << old_port; os << std::setw(encoded_port_size) << std::hex << new_port; } // The length in bytes of a control_request and its components. enum { encoded_port_size = 4, // 16-bit port in hex. control_request_size = encoded_port_size * 2 }; // The encoded request data. char data_[control_request_size]; }; // This frame is sent from the server to subscribed clients over UDP. class frame { public: // The maximum allowable length of the payload. enum { payload_size = 32 }; // Construct an empty frame. Used when receiving. frame() { } // Construct a frame with specified frame number and payload. frame(unsigned long number, const std::string& payload) { std::ostrstream os(data_, frame_size); os << std::setw(encoded_number_size) << std::hex << number; os << std::setw(payload_size) << std::setfill(' ') << payload.substr(0, payload_size); } // Get the frame number. unsigned long number() const { std::istrstream is(data_, encoded_number_size); unsigned long number = 0; is >> std::setw(encoded_number_size) >> std::hex >> number; return number; } // Get the payload data. const std::string payload() const { return std::string(data_ + encoded_number_size, payload_size); } // Obtain buffers for reading from or writing to a socket. boost::array<boost::asio::mutable_buffer, 1> to_buffers() { boost::array<boost::asio::mutable_buffer, 1> buffers = { boost::asio::buffer(data_) }; return buffers; } private: // The length in bytes of a frame and its components. enum { encoded_number_size = 8, // Frame number in hex. frame_size = encoded_number_size + payload_size }; // The encoded frame data. char data_[frame_size]; }; #endif // AAOIP_PROTOCOL_HPP

On Wed, 14 Dec 2005 17:31:22 -0500 Stefan Seefeld <seefeld@sympatico.ca> wrote:

For me a 'socket class' is two things (let's ignore datagram sockets in this context): a way to create socket endpoints, as well as manipulate associated options, and on the other hand some means to read and write.

First, you can't ignore datagrams, as that is a large part of modern networking, especially in large data distribution systems. But, for the sake of conversation...

For the synchronous case I think this is nicely described by a streambuf / stream pair of interfaces. A 'socket_stream' class would allow to create socket endpoints, and set socket options. A socketbuf class would implement read and write operations.

I've looked at a ton of IOStream network implementations over the years. Two of my major problems with IOstreams for networking: error handling and performance (followed closely by several other issues). I've yet to see anything close to reasonable. For that matter, I've yet to see anything implemented with the IOStreams that isn't a problem for anything but the most simple applications. As soon as I see IOStreams, my antennae go crazy. Sure, it's a nice concept... I think it is appropriate to offer an IOStreams interface for those who desire such an interface. However, if that is the best way to use synch sockets in boost, I won't go near it. For many years, I have been in the business of very high performance, highly distributed network applications, and my paranoia is well placed ;-)

On the other hand, asynchronous I/O over sockets would be handled very differently. In fact, given that platforms provide quite different means to wait for readable data (say), I'm not even sure that the 'socket' concept should be discussed independently from the sync / async policy at all.

I can see your point.

Wow, what a thread. Rather than replying to everything individually, I'm going to try to address the issues in this one email. --- Arkadiy Vertleyb <vertleyb@hotmail.com> wrote:

My problem with the current approach is that synchronous operations seems to be viewed as second-class citizens.

On the contrary, asio attempts to elevate asynchronous operations to the same level as their synchronous counterparts. For virtually every blocking synchronous operation in asio, there is an asynchronous equivalent. The intent is to improve ease of use for asynchronous operations to be as close as possible to synchronous. Far from synchronous operations being given second class status, considerable effort has gone into ensuring that the composed operations (such as asio::[async_]read and asio::[async_]write) have the same semantics for both synchronous and asynchronous calls. Further levels of abstraction should also be achievable in a similar vein. Let me state as clearly as possible: the synchronous operations are not implemented in terms of the asynchronous operations. They call the underlying synchronous system call with little overhead. The question was raised over whether a synchronous-only sockets API should be provided, and asio implemented in terms of that. I rejected this approach early on in the development of asio. As others have already noted, such an API is not sufficient for the development of asynchronous operations, and you are forced to step outside of it to use operating system-specific functionality. Instead, asio is itself intended as a thin, portable API that provides both synchronous and asynchronous support. So, the nub of your issue appears to be the conceptual difficulties you perceive in having the socket constructor take a demuxer, and for the interface to include both synchronous and asynchronous operations. Let's consider the alternative approach proposed: to split the socket class into separate sync_socket and async_socket classes. The drawbacks of this approach are increased redundancy and complexity in the programming interface. If this approach were already in place, I can imagine that I would be getting questions about why there are so many socket classes, and which one is the most appropriate to use. There are trade-offs in every design decision. I elected to give synchronous and asynchronous operations equal weight, and combine them in a single interface. This has the added advantage of allowing mixed-mode programming, where synchronous operations can be used alongside asynchronous in a single application. As you point out, there are situations where synchronous is more appropriate, and these differences can occur within an application. I don't think that the need to pass a demuxer to the socket constructor is conceptually difficult to explain. It's there for the asynchronous operations, if and when you need them. I also do not agree that it is onerous to have to pass the demuxer to the constructor, or to pass it wherever it's needed in the program. You can make it a global, declare a new one for each use, or even use one associated with an existing object, as in: stream_socket s(my_acceptor.demuxer()); Therefore I stand by my design choices as the best balance between usability, functionality and flexibility. Now, to turn to more philosophical matters, namely synchronous versus asynchronous approaches. As you pointed out, synchronous socket programming is easier to debug. However, there are also disadvantages, notably increased resource usage (with a thread required per connection) reducing scalability, and the need for explicit synchronisation between threads. Correct programming in the presence of threads is a difficult art, and I see this as a major disadvantage of the synchronous approach. Asynchronous socket programming, on the other hand, has the advantage of allowing concurrent operations without synchronisation. Programs can be written as though they exist in a single-threaded environment, and still scale to handle numerous connections. The main disadvantage of the asynchronous approach is program complexity, since operation initiation is separate from completion, and the flow of control is inverted. However I have designed asio to mitigate these as much as possible, particularly in providing the ability to combine asynchronous operations to allow the implementation of higher-level abstractions. When people first start network programming, it is natural for them to gravitate towards synchronous operations. They may then progress to writing more complex applications, such as servers, that must handle multiple connections. If all they have been exposed to is synchronous network programming, then it is likely they will end up with a design that requires many threads. By making both synchronous and asynchronous operations readily available, asio may inform programmers that there are alternatives to multithreading. If this helps them to make a more appropriate choice for their application, then that has got to be a good thing. Cheers, Chris

Christopher Kohlhoff <chris@kohlhoff.com> writes:

[snip]

I don't think that the need to pass a demuxer to the socket constructor is conceptually difficult to explain. It's there for the asynchronous operations, if and when you need them.

I also do not agree that it is onerous to have to pass the demuxer to the constructor, or to pass it wherever it's needed in the program. You can make it a global, declare a new one for each use, or even use one associated with an existing object, as in:

stream_socket s(my_acceptor.demuxer());

Therefore I stand by my design choices as the best balance between usability, functionality and flexibility.

It seems that a possible solution is to provide a `dummy' demuxer that simply makes asynchronous operations fail. This `dummy' demuxer could then be the default parameter to the socket constructors, thus eliminating the need to specify a demuxer for synchronous-only use. It seems that this would indeed clean up the interface for synchronous-only use.

[snip]

-- Jeremy Maitin-Shepard

Hi Christopher, "Christopher Kohlhoff" <chris@kohlhoff.com> wrote

--- Jeremy Maitin-Shepard <jbms@cmu.edu> wrote:

...

It seems that a possible solution is to provide a `dummy' demuxer that simply makes asynchronous operations fail. This `dummy' demuxer could then be the default parameter to the socket constructors, thus eliminating the need to specify a demuxer for synchronous-only use.

It seems that this would indeed clean up the interface for synchronous-only use.

After pondering this for a few days, I've come to the conclusion that doing the above isn't that dissimilar to having a singleton demuxer. Such a demuxer would be used as the default argument to constructors, as Jeremy said. It certainly wouldn't be any different in terms of runtime performance. What do people think of this approach?

Could you provide some (high-level) justification of why it is beneficial to have a socket dependency upon a demuxer? Maybe I am missing something, but IMO such a dependency is unnatural even for asynchronous IO. Hiding it behind default parameter, singleton, etc., just masks what I believe is a design problem. If you absolutely believe this dependency is necessary for async IO, having two separate classes, such as sync_socket and async_socket would be much cleaner. But again, I can't see what justifies such a dependency even for async IO. Regards, Arkadiy Regards, Arkadiy

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote

--- Arkadiy Vertleyb <vertleyb@hotmail.com> wrote:

...

Could you provide some (high-level) justification of why it is beneficial to have a socket dependency upon a demuxer?

Maybe I am missing something, but IMO such a dependency is unnatural even for asynchronous IO.

This has not come up in the discussion thus far, but the dependency on the demuxer is required to enable an efficient and portable implementation of asynchronous I/O. This is one of the primary goals of asio. The asio interface has been designed to reflect efficient asynchronous I/O mechanisms across major operating systems.

Let's take the specific case of Windows, where asynchronous I/O is implemented using I/O completion ports. Once a socket is associated with an I/O completion port it cannot be disassociated.

In asio, the demuxer represents the I/O completion port. The asio interface enforces this association by constructing the socket with its associated demuxer/IO-completion-port.

...

From the point of view of the library user, any asynchronous operations started on the socket will have the completion handler delivered through the associated demuxer (as it is with I/O completion ports).

Separating the asynchronous I/O operations from the socket class would no longer enforce this requirement at compile time. (That's even leaving aside questions about where the operation logically belongs.)

Understand and agreed. For async IO, socket doesn't seem to make a sence without a demuxer. Passing the demuxer to the socket constructor in this case makes perfect sense.

...

Hiding it behind default parameter, singleton, etc., just masks what I believe is a design problem.

If you absolutely believe this dependency is necessary for async IO, having two separate classes, such as sync_socket and async_socket would be much cleaner. But again, I can't see what justifies such a dependency even for async IO.

So the question is really whether there should be separate sync and async socket classes, versus a combined class as there is now. I believe that, on balance, separate classes would be harmful. Let's reiterate some of the points:

- Synchronous and asynchronous operations are essentially the same, in that they perform the same operation and aim to fulfil the same contract. How they differ is simply that one blocks the current thread, whereas the other executes in a background logical thread and tells you when it is complete. I want there to be a clear mapping from one to the other.

That should not be difficult to achieve with derivation or some sort of policy. Looks like async_socket might be derived from sync_socket (or contain it).

- Separate classes reinforces the notion that synchronous operations are somehow more useful for the ordinary programmer, and that asynchronous is hard and best left in the realm of the networking guru. I do not believe that asynchronicity is difficult to use or understand, it is simply that programmers often approach sockets with preconceived ideas of how an API should behave. But I do believe that separating the classes will continue the lack of awareness of asynchronicity as an option, leading to the needless use of inferior designs.

I don't agree that separate classes reinforce any notion. They just clearly state that there are two different ways of doing things. And I don't believe forcing people to use a demuxer without the need is a right way to raise awareness...

- The greater number of classes and options will increase the size of the library's interface.

One of the fundamental OO design principles states that clients should not depend on what they don't use (Interface segregation principle).

- I have demonstrated that there is a use case for mixed-mode applications. Predominantly synchronous designs can benefit from localised asynchronicity. Asynchronous designs can be simplified by selective use of synchronous calls.

So one would use async sockets in this case.

- A line of reasoning has been presented which says that, because files already have a synchronous-only interface and could be given an asynchronous interface, sockets should also have a synchronous-only interface. However, sockets are inherently different to files due to the long timescales involved in many operations. Developers will look for some form of concurrency to address this, and I believe asynchronicity *should* be promoted over threads as a solution.

It is worth noting that even .NET combines the sync and async socket operations on a single interface.

I do find that there is a compelling case for a synchronous-only interface to sockets at the iostreams level. But that is because of the established use and wide applicability of iostreams, not because it is synchronous.

So let me turn the question around. Why must there be a low-level synchronous-only interface at all? What benefits will it bring? What use cases do you have that cannot be met by a combined interface or an iostream interface?

I don't like the the combined interface because it doesn't satisfy the interface segregation principle. Also I don't like when things have to be specified that are not used. iostream might be alright, but it makes things kind of asynchronous (in the other sence of this word). Also you don't win anything -- you will have two interfaces, the same amount you would have with sync_ and async_socket. Regards, Arkadiy

"Peter Petrov" <ppetrov@ppetrov.com> wrote in message news:do81b5$4ir$1@sea.gmane.org...

Arkadiy Vertleyb wrote:

...

That should not be difficult to achieve with derivation or some sort of policy. Looks like async_socket might be derived from sync_socket (or contain it).

This was also my thought - public derivation. I.e., the current socket class is refactored into two classes - base (sync_socket) and and derived (async_socket). The sync_socket class won't know anyhing about demuxers (it doesn't need to).

Except that demuxer is currently also a service repository. And wouldn't the service repository if decoupled from a demuxer be a legitimate candidate for a singleton?

Eugene Alterman wrote:

"Peter Petrov" <ppetrov@ppetrov.com> wrote in message

...

This was also my thought - public derivation. I.e., the current socket class is refactored into two classes - base (sync_socket) and and derived (async_socket). The sync_socket class won't know anyhing about demuxers (it doesn't need to).

Except that demuxer is currently also a service repository. And wouldn't the service repository if decoupled from a demuxer be a legitimate candidate for a singleton?

My understanding is that the "service repository" is actually only needed for the asynchronous operations.

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote in message news:20051221090730.33683.qmail@web32603.mail.mud.yahoo.com...

For asynchronous operations, the fact that a demultiplexer is used is an implementation detail. The public interface of asio does not prohibit an implementation that uses, say, a thread per operation.

Does it make any difference? Thread completion still needs to be demultiplexed.

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote in message news:20051221090730.33683.qmail@web32603.mail.mud.yahoo.com...

- You must run() the io_system object for it to perform asynchronous operations on your behalf.

Just to be precise, if asynchronous i/o is real (not emulated) you don't have to do anything for the operation to be performed, and run() will just demultiplex completion events. This BTW brings up a question of possible portability issues resulting from such a difference in behavior. I also think that 'run' is not descriptive enough, especially if the class name becomes 'io_system'. Something like dispatch_events() may be?

Christopher Kohlhoff <chris <at> kohlhoff.com> writes:

I propose that it could be called something like "io_system". (Other naming suggestions will be appreciated.)

Based on the statements below, I think io_engine might be appropriate.

Consider the following statements:

- An io_system object must be initialised before sockets (or other I/O objects) can be used. See the portability requirement above.

- An io_system object is an extensible collection of I/O services (drivers?).

- Synchronous operations implicitly run the io_system object for an individual operation.

- You must run() the io_system object for it to perform asynchronous operations on your behalf.

- You can partition your program by using multiple io_system objects.

I wonder if 'service' is the best name for the set of I/O objects encapsulated by the library, but I can't offer any alternatives... Matt

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote in message news:20051221090730.33683.qmail@web32603.mail.mud.yahoo.com...

...

So I wonder if the problem is really just one of naming. Specifically, the name of the demuxer class.

Asio started life as a library intended for asynchronous use-cases, and for developers with that sort of background I believe the name demuxer (i.e. demultiplexer) is perfectly natural.

Now that it has grown into general purpose use, perhaps the name should reflect the expectations of a wider audience.

I think that would be very helpful.

I propose that it could be called something like "io_system". (Other naming suggestions will be appreciated.)

"io_system" seems a little broad. I think of an "io_system" as encompassing all I/O mechanisms an operating system supports. A web search turns up broad phrases like "The basic model of the UNIX I/O system is a sequence of bytes that can be accessed either randomly or sequentially." "io_engine" implies to me something at a very low level, like the set of device drivers. How about "io_service"? That seems both narrowly focused and about the right level to me. I've changed "io_system" to "io_service" in your text below. Reads quite well IMO.

Consider the following statements:

- An io_service object must be initialised before sockets (or other I/O objects) can be used. See the portability requirement above.

- An io_service object is an extensible collection of I/O services (drivers?).

- Synchronous operations implicitly run the io_service object for an individual operation.

- You must run() the io_service object for it to perform asynchronous operations on your behalf.

- You can partition your program by using multiple io_service objects.

--Beman

"Peter Dimov" <pdimov@mmltd.net> wrote

Beman Dawes wrote:

...

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote in message news:20051221090730.33683.qmail@web32603.mail.mud.yahoo.com...

...

...

I propose that it could be called something like "io_system". (Other naming suggestions will be appreciated.)

"io_system" seems a little broad. I think of an "io_system" as encompassing all I/O mechanisms an operating system supports. A web search turns up broad phrases like "The basic model of the UNIX I/O system is a sequence of bytes that can be accessed either randomly or sequentially."

"io_engine" implies to me something at a very low level, like the set of device drivers.

How about "io_service"? That seems both narrowly focused and about the right level to me.

I think that a "driver" matches the description quite well:

"dispatcher"? Regards, Arkadiy

"Christopher Kohlhoff" <chris@kohlhoff.com> wrote in message news:20051228012139.77878.qmail@web32615.mail.mud.yahoo.com...

So what about io_broker? For people who have used CORBA, similar rules apply in respect of calling run():

- No need to call run() in a client making synchronous calls.

- Need to call run() if using asynchronous method invocation so that results can be delivered.

A possible disadvantage of io_broker is that the name conveys too much meaning, in that users may feel it is a concept they need to understand before using the facilities it provides. I don't think a name like io_services suffers from this.

I have a mild preference for io_services. But io_drivers or some other variation on Peter's "driver" suggestion would also work for me. For the "too much meaning" rationale you give, I don't particular like io_broker, but even that seems better than demuxer, which I always found confusing. Thanks for worrying about such quibbles - good names do help a library. --Beman

Hi Paul, --- Paul A Bristow <pbristow@hetp.u-net.com> wrote:

I have strong preference for Beman's original suggestion:

io_service

I think it just hits the spot.

That's good enough for me, I'll go with io_service.

asio is short and snappy - does it stand for Async AND Sync Input Output?

Nah, it stands for Australian Security Intelligence Organisation ;) Seriously though, it got the name because my wife thought it was funny. Cheers, Chris

Peter Dimov <pdimov@mmltd.net> writes:

The problem with the above apporach is that most code will end up using the global demuxer because this is the path of least resistance. When it turns out that the code needs to be refactored to use a specific demuxer, the programmers would need to go over it with a fine-toothed comb

We had this exact problem with a large application built atop ACE. Within ACE, there are a few areas where one can refer to "/the/ reactor" rather than "/a/ reactor" or "/this/ reactor", mostly for convenience. We had our reasons for creating and passing around our own reactor, but it was difficult to ensure that all the reactor-dependent code explicitly used the proper reactor instance. -- Steven E. Harris

On Mon, 19 Dec 2005 19:43:00 -0500, Cliff Green wrote

...

...

The problem with the above apporach is that most code will end up using the global demuxer ...

...

We had this exact problem with a large application built atop ACE. Within ACE, there are a few areas where one can refer to "/the/ reactor" rather than "/a/ reactor" or "/this/ reactor ...

I really dislike the ACE singleton interface in the Reactor (and in some other classes), for the same reasons already described. In particular, an application might use a library that internally uses the Asio singleton demuxer - now the application's use of the singleton demuxer might conflict. This dislike is based on real-life use cases (with ACE), btw. Please don't add a singleton demuxer interface.

One more voice against this...I've had similar issues with singletons in ACE and other places. There would have to be extreme justification to introduce a singleton into the asio interface. Small simplification for end users isn't enough for me. Peter's points are right on the mark... Jeff

Hi Peter, --- Peter Dimov <pdimov@mmltd.net> wrote:

You should dislike singletons because they are global variables.

The problem with the above apporach is that most code will end up using the global demuxer because this is the path of least resistance. When it turns out that the code needs to be refactored to use a specific demuxer, the programmers would need to go over it with a fine-toothed comb and replace every implicit reference to the global demuxer with an explicit demuxer reference, passing that reference wherever necessary. It'd be a pain, and it'd be easy to miss some. Not to mention that the code that implicitly uses the global demuxer may be in a library for which the source is not available.

So yes, the easy case will be easier, but the hard case will be harder.

Good points. Scratch that idea. Cheers, Chris

"Eugene Alterman" <eugalt@verizon.net> wrote in message news:dnv3c8$eo2$1@sea.gmane.org...

I just want to point out (hopefully without getting off topic too much) that there is an important difference between what you mean by 'synchronous' and the way this term is usually used in this context (and in particular in the publication you are mentioning).

By 'synchronous' you seem to mean *blocking* synchronous i/o while in those patterns it is understood as synchronous i/o event demultiplexing and handling, meaning that a synchronous i/o operation is issued only if there is a ready event and the operation would not block.

This is not accurate of course :( Please disregard the whole thing - I just got carried away. :-)

"Arkadiy Vertleyb" <vertleyb@hotmail.com> wrote in message news:dnpe9q$vk6$1@sea.gmane.org...

...

Let me disagree with this. There are cases when having multiple

"Eugene Alterman" <eugalt@verizon.net> wrote threads,

...

each working in a synchronous way, is a better approach. First, it's much more intuitive, and easier to debug. Second, sometimes it's the only way to achieve you goal, such as whan you can't (or don't want to) rewrite your processing algorithm in asynchronous way.

And when such a case arrives, it would be nice to have a clean socket class, without build-in asynchronisity.

Hold on a second...

Where has this perception that asio implements synchronous operations on top of asynchronous come from?

...

From what I can see in docs and sources this does not seem to be the case. Notice that demuxer::run() is not called in the tutorial synchronous samples.

Synchronous operations are simply implemented as regular blocking socket calls.

This is good. Now what's left is to remove the dependency of the socket on the demuxer. Otherwise it's confusing: if demuxer is used, it has to be used for some purpose, right? Chris admitted that the socket class has built in features for asynchronous processing. This IMO is a conceptual overhead, and reminds of MFC's CWindow class, which is always an activeX container. Just in case. Regards, Arkadiy

Hi Chris

...

I know that's not correct asio - but what's a recommended way to achieve this without wrapping each method of the interface.

I haven't really thought about it that deeply, but since you talk about a fan-out, cannot the dispatcher wrap be done on a function just before the fan-out occurs? I.e. a function that takes a complete message before a decision about what type it is has been made -- before the one becomes many?

Yes, i believe thats necessary. Problem is - whats a clean way for the recipient of the fanned out callbacks to control the dispatcher policy. (see post about polymorphic_dispatcher for one approach). Cheers Simon

Hi Simon, --- simon meiklejohn <simon@simonmeiklejohn.com> wrote:

Is it feasible to accomplish something like this with a class something like the following (but more correct hopefully) <snip code>

Yes, I think it's feasible. It would probably be something more like: class abstract_dispatcher { public: virtual ~abstract_dispatcher(); virtual void post(function<void(void)> f) = 0; virtual void dispatch(function<void(void)> f) = 0; // Note: non-virtual template member function. Can reuse // asio::detail::wrapper_handler to implement it. template <typename Handler> unspecified wrap(Handler h); }; template <typename Dispatcher_Impl> class dispatcher : public abstract_dispatcher { public: // Forwarding constructors here ... void post(function<void(void)> f) { impl_.post(f); } void dispatch(function<void(void)> f) { impl_.dispatch(f); } private: Dispatcher_Impl impl_; }; And to use: // An abstract dispatcher that owns its implementation dispatcher<my_dispatcher> d(... args here ...); // An abstract dispatcher that doesn't own the implementation. demuxer d1; ... dispatcher<demuxer&> d2(d1); Cheers, Chris

Hi Chris,

I made a few quick changes and re-ran the daytime3 server to observe the memory allocations. What I changed was:

<some code>

Looks good. Eliminating "null()" makes things much better. However the underlying shared_ptr<socket> is quite important. Until you find a better solution, you could try to use a simple segregated storage (boost::pool or similar) for socket allocation to minimize memory use (in 32 systems unsigned int has the same size as a pointer) and the user could define BOOST_SP_USE_QUICK_ALLOCATOR so that shared_ptr's shared_count objects are also pooled. This way, memory waste will be minimum and the allocation very fast. Now we just need a way to provide a custom allocation for async_xxx operations. I will follow Christopher Baus' proposal. Regards, Ion

If ::operator new isn't terribly broken (if it is, use dlmalloc), memory waste will aready be minimal and allocation would already be fast for small objects. Custom per-component allocators are rarely worth the trouble, and should never be the default.

In some experiments, I see that for example, in windows, the default allocator uses pools for small objects, but this is not true in all systems. For example, in many embedded operating systems (for example, QNX, eCos, Nucleus... and correct me if someone knows these OS better than me), there is no optimized new. In these systems, more than speed problems, you can have fragmentation problems. The fastest, is of course, avoiding dynamic allocation for small objects. If we need reference counting, the second one would be to use a intrusive pointer. Even in windows, you can test your own pool against "new" and you will see a (very) small speed gain. But if you don't need multi threaded allocation (some very possible with asio, since from just one thread you can control all async operations) you can avoid mutexes in the allocator, the speed gain is noticeable, because the allocation algorithm in a simple segregated storage is much faster than a mutex lock. If I were developing a router/rock stable embedded system using asio, I would prefer controlling the allocations myself, when I know that each repetitive operations leads to several calls to "new". Maybe a bit of paranoia, but not all "new" implementations are optimized and pooled for small allocations. I agree with you that this shouldn't be default, though. "new" should be default, but I suggest a mechanism to override this mechanism with user defined allocations. I would love to see that mechanism in shared_ptr for the counter, because for the object itself you can use a custom deleter. But this is another history. Regards, Ion

Hi Ion, --- Ion Gaztañaga <igaztanaga@gmail.com> wrote:

Looks good. Eliminating "null()" makes things much better. However the underlying shared_ptr<socket> is quite important.

I have now also eliminated the allocation of the socket by adding a separate member to impl_type for storing the socket, and changing to use shared_ptr<void> with a no-op deleter object. The shared_ptr member of impl_type is now just used as a token to indicate cancellation. Thanks for pointing out these inefficiencies! :) Cheers, Chris

Ion Gaztañaga wrote:

...

I have now also eliminated the allocation of the socket by adding a separate member to impl_type for storing the socket, and changing to use shared_ptr<void> with a no-op deleter object. The shared_ptr member of impl_type is now just used as a token to indicate cancellation.

Ummm, very clever move! There is no faster allocation than doing nothing. If you want to avoid the counter, you could use intrusive_ptr and the party is over.

In case you decide to go with the intrusive_ptr, I've just put up a wrapper template, shared_proxy<T>, that attaches a reference count to a class. I wrote shared_proxy specifically to cut the overhead of shared_ptr in those cases where one controls the allocation of shared objects - you might as well go ahead and allocate the reference count with it. You can find the shared_proxy implementation here, http://tinyurl.com/8axpz I believe the implementation to be simple and readable, alas there are no docs for now :-( Use case, #include <breeze/shared_proxy.hpp> #include <boost/intrusive_ptr.hpp> ... typedef breeze::shared_proxy<socket> socket_proxy; typedef boost::intrusive_ptr<socket_proxy> socket_shared_ptr; socket_shared_ptr ptr = new socket_proxy(); // double indirection is required (**ptr).socket_member_function(); While I don't think this would look nice on the user visible interface, shared_proxy could be used in some places as an implementation detail. [ If shared_proxy (or something like it) is considered useful, maybe I could get away with hijacking partial specializations of intrusive_ptr, shared_ptr and weak_ptr to make its use transparent :-] HTH. Best regards, João Abecasis

From: "Christopher Kohlhoff" <chris@kohlhoff.com>

I do want to keep the callback handlers last for consistency, but I'll add an overload of these functions that leaves out the flags parameter.

Sounds good.

Are the QoS settings configured using ioctl? My brief reading of MSDN would indicate so on Windows. If this is the case, then a new implementation of the IO_Control_Command concept could be added for this, which can then be passed to basic*socket::io_control. If you'd like to suggest what interface it needs (or, even better, create an implementation!) I'll look at adding it. BTW, is this an OS-specific thing?

Not up on this myself, though i might be able to look through our code for some answers. Our implementation is purely win32 and is based around a separate socket class. Apparently uses some microsoft specific API, so perhaps not just ioctl. There seems to be a concept of negotiating/guaranteeing some data flow rate through the socket. I'm away from work for a couple of weeks from now, so i wont be able to look further into it till i get back. Any experts out there please feel free to jump in. Cheers Simon

Hi Darryl, --- Darryl Green <darryl.green@unitab.com.au> wrote: <snip>

I think the design as described by the concepts section is very good, but it would be better if the implementation offered finer-grained selection of the desired concepts. Perhaps this is just an aesthetic concern, and it would be easy to modify the existing services, but if these really are separate concepts, why are they bundled together? I'm concerned that there may be some lack of flexibility in the implementation/interface actually offered that isn't apparent from the design. The issue of separating out the sync vs async interface has already been raised;

This has been extensively discussed elsewhere, but in summary I can see no compelling reason why there should be a separate low-level sync-only interface, and many advantages to having them combined. However I still intend to rename the demuxer to make its purpose clearer for all sorts of applications.

I would also like to see the ability to expose a reactive rather than proactive interface by using a reactive_stream_socket_service in place of an async_stream_socket_service. The concepts would seem to allow for this.

This is not consistent with the goal of portability. As I mentioned in the "reactor versus proactor" thread, the proactor is a more portable pattern than the reactor. In the longer term I'm not averse to allowing a reactor abstraction to become part of some secondary public interface, where the restricted portability is explicitly noted.

One item that I have difficulty in discerning a clear design/rationale for is the demuxer. The demuxer as implemented is a repository of services (another wooly concept afaiks) and something that provides the run, interrupt and related functions to control the scheduling/dispatching of handlers in reaction to changes in Async_Object state. Is there anything preventing a more policy-based approach to assembling services?

The demuxer is composed of services in a similar way to how std::locale is composed of facets. The demuxer is not necessarily aware of, or coupled to, the service types that it contains. For example, the SSL implementation uses a service that is only added in to the demuxer if SSL is used. The service used by a particular public interface (e.g. basic_stream_socket) is already a policy template parameter. One example program shows how basic_stream_socket can be instantiated with a custom service to perform logging, where this service is in turn implemented using the standard stream_socket_service. There's no reason why, in principle, policy-driven services could not be added to allow further customisation. But these would be inherently non-portable, and so would only be part of some secondary public interface.

I would like to see the aspects of a socket that have nothing to do with reading/writing better separated, as this would make for a more consistent interface for other forms of I/O such as pipes, message queues etc as well as files.

For stream-oriented I/O, this separation is presented in the form of the Stream concept. This concept is implemented by stream_socket, ssl::stream, buffered_read_stream and so on. It would be implemented by a hypothetical pipe class.

A very simple active object template that supports read and write, but that relies on free functions or helper classes for less universal operations such as bind, setting options, etc, used as a base class, allows appropriate (and necessary in my opinion) use of runtime polymorphism.

I have to disagree on runtime polymorphism: compile-time polymorphism should always be the default. Runtime polymorphism can impose additional costs (such as virtual function calls on a fine-grained interface), and eliminates potential optimisations. This adds up considerably if there are many layers of runtime polymorphism involved. Runtime polymorphism can be layered on top of compile-time polymorphism using some sort of wrapper. E.g. for the stream concept: class abstract_stream { ... }; template <typename Stream> class stream : public abstract_stream { ... }; stream<ssl::stream<stream_socket> > s(...); runtime_polymorphic_function(s); I am not certain if this is something that should be included in asio or not, since the appropriate place to introduce runtime polymorphism can vary according to the design of an application. What do you think? I'd be happy to add it if you think it generally useful. <snip>

The intention was for this general idea to support other types of "file descriptor" that don't offer conventional read and write interfaces, and to allow other interfaces (such as proactive) to use the same demuxing etc infrastructure.

I had hoped to have time to investigate introducing asio as a replacement/alternative to this library, but the timing hasn't been gode for that. In any case, I don't think asio would be a good fit for a system like the following (where the other lib is being used):

Components support tcp and ssl for WAN connections and unix domain sockets locally. Throw in some weird stuff like communicating over special purpose point-to-point links implemented as character devices. There are large chunks of code that have no need to care what exact type of "socket" they are using. Some of this code is actually in dynamically linked libraries itself. There are multiple processes using these libraries.

Short of placing a wrapper around asio to make it more amenable to separate compilation and to introduce runtime polymorphism I can't see how I could use asio effectively this system, or anything similar.

Would the polymorphic wrapper I proposed above address these issues?

As far as I can see asio doesn't even allow interchangeable use of an ssl and a tcp stream.

As I mentioned above, asio addresses this using compile time polymorphism, with the Stream concept being implemented by stream_socket and ssl::stream. For example, both work equally well with free functions like asio::async_read.

I think this is unacceptable in a general purpose library. I'm also concerned about the potential code bloat from unnecessary duplication of code (not from differnt template parameters, but from outright duplication in differnt templates).

Can you please point these out? I'm always in favour of reducing code bloat.

Implementation

The code is clean, however I'm very concerned about the performance (or lack of) and weight that a combination of the implementation and some design decisions have introduced. The documentation mentions the memory usage overhead of the proactor emulation on systems using select/epoll. However it is clear that there is considerable allocation overhead and construction overhead involved (and not only on those systems). While much discussion has centered on allocator performance, I'm concerned that the user/library constructor code invoked for each i/o operation is likely to be non-trivial as well. For bulk transfers, it is likely that these issues won't be too severe, however for systems where low-latency handling of small messages is critical, it is quite possible that these overheads will be comparable to or larger than any other part of the processing for a message/transaction.

I don't deny that there may be room for performance improvement in the implementation. But as I recently tested the custom allocation with Win32 I/O completion ports, and found that it performs almost identically to synchronous I/O, I don't think the public interface imposes any unnecessary inefficiencies.

I find the way the design supports, and the documentation encourages, breaking up handling of structured messages into a small header read followed by a larger read (with appropriate handler for expected body/chunk) while the implementation appears to do some quite heavyweight operations compared to just reading a small header to be almost an encouragement of an antipattern for efficient I/O.

I don't agree that asio favours this approach over others. But I do believe that this approach is perfectly valid and extremely useful, since it permits asynchronous code to be written in terms of "contracts". I.e. perform this operation (or operations) and don't come back to me until it's all done or an error has occurred. For code where absolute efficiency is not required, this technique allows a clearer and more elegant expression of intent. Furthermore, the cost of multiple small read system calls can be mitigated by leveraging the compile-time polymorphism of the Stream concept and replacing: stream_socket my_socket; with: buffered_read_stream<stream_socket> my_socket;

I would expect an optimistic reactive approach (read the header when notified, determine size, handler etc and opertunistically attempt a read of the body) to be significantly faster.

Let's take your example of a fixed sized header followed by a body, and compare and contrast a reactive approach with the alternatives offered by asio. --------------------- 1. Reactive. Here you might have a single handler that is informed when a socket is ready for reading. In this handler you perform non-blocking reads. void handle_read(...) { switch (state_) { case reading_header: // Attempt non-blocking read of header. size_t bytes_read = read(sock_, header_buf_ + header_bytes_so_far_, header_length - header_bytes_so_far_); header_bytes_so_far_ += bytes_read; // Check if we have complete header. if (header_bytes_so_far_ < header_length) return; // Keep waiting for more. // Got the whole header now. body_length_ = decode_body_length(header_buf_); body_buf_ = ...; // size body buffer appropriately body_bytes_so_far_ = 0; state_ = reading_body; // Fall through to make opportunistic read of body. case reading_body: // Attempt non-blocking read of body. bytes_read = read(sock_, body_buf_ + body_bytes_so_far_, body_length_ - body_bytes_so_far_); body_bytes_so_far_ += bytes_read; // Check if we have complete body. if (body_bytes_so_far_ < body_length_) return; // Keep waiting for more. // Got whole body now. notify_app(...); // Start over. state_ = reading_header; } } 2. Asio with multiple reads. This is the approach where we issue asynchronous reads for exactly the length of the header, and then the body. It may be less efficient, but only because of the additional pass through the demultiplexer, not because it makes more read() system calls than the above. However feel free to compare in terms of readability :) The main reason it's clearer is that you no longer have to deal with short reads -- async_read's contract takes care of that for you. void start_read() { // Issue read for entire header. async_read(sock_, buffer(header_buf_, header_length), handle_read_header); } void handle_read_header(const error& e) { if (!e) { // Got header, determine body length. body_length_ = decode_body_length(header_buf_); body_buf_ = ...; // size body buffer appropriately // Issue read for entire body. async_read(sock_, buffer(body_buf_, body_length_), handle_read_body); } } void handle_read_body(const error& e) { if (!e) { // Got whole body now. notify_app(...); // Start over. start_read(); } } 3. Asio with multiple reads + opportunistic body read. This extends number 2 by adding in the opportunistic read of the body. In effect this makes it equivalent to your reactive read approach, but I think you'll still find it's easier to follow -- again because most of the code to handle short reads is eliminated. void start_read() { // Issue read for entire header. async_read(sock_, buffer(header_buf_, header_length), handle_read_header); } void handle_read_header(const error& e) { if (!e) { // Got header, determine body length. body_length_ = decode_body_length(header_buf_); body_buf_ = ...; // size body buffer appropriately // Make opportunistic non-blocking read of body. error read_error; size_t bytes_read = sock_.read_some( buffer(body_buf_, body_length_), assign_error(read_error)); // Did we get whole body? if (bytes_read == body_length_ || read_error) { handle_read_body(read_error); } else { // Issue read for rest of body. async_read(sock_, buffer(body_buf_ + bytes_read, body_length_ - bytes_read), handle_read_body); } } } void handle_read_body(const error& e) { if (!e) { // Got whole body now. notify_app(...); // Start over. start_read(); } } 4. Asio with bulk reads. This approach is exemplified by the HTTP server example. Essentially it reads as much as is available (up to some limit of course) in one go, and then processes it using some state machine similar to that in number 1. It is generally the most efficient since it requires the fewest number of read system calls or passes through the demuxer. void handle_read(const error& e, size_t bytes_read) { if (!e) { process_data(buffer_, bytes_read); async_read(sock_, buffer(buffer_, buffer_length), handle_read); } } 5. Asio with transfer_at_least(...). In certain applications you may be able to combine the opportunistic read into the initial asynchronous read operation, by using the transfer_at_least() completion condition. This executes the operation with a contract that it transfer a minimum number of bytes. In this way you can avoid short read handling for the header, while still maximising the number of bytes transferred in a single operation. void start_read() { // Issue read for entire header, and maybe we'll get some // body as a bonus. boost::array<mutable_buffer, 2> bufs = { buffer(header_buf_, header_length), buffer(body_buf, max_body_length) }; async_read(sock_, bufs, transfer_at_least(header_length), handle_read_header); } ... ---------------------

It is implied that an appropriate demuxer/demuxer service can be used to implement various multi-threaded processing schemes. This may well be true. However, an alternative approach is to simply consider the affinity of an Async_Object and its demuxer(s) (one for read, one for write) to be dynamic and run a number of separate (per thread) demuxers, moving Async_Objects between them as required. In this way the handlers for a given Async_Object will only run in the expected context.

This is not portable. With Win32 I/O completion ports for example, once a socket is associated with an I/O completion port it cannot be disassociated. The demuxer design reflects this.

I imagine other approaches are possible to achieve similar results, but it is not clear (to me) from the design documentation or a quick look at the code that this is the case. I haven't reviewed the code to a level of detail where I can be sure I am understanding the dispatching done by the epoll and select reactors, but it looks to me as though multiple threads may all wait on the same fdset (or equivalent)? Is this right? Intended? I had expected some form of modified leader/followers pattern here?

No, only one thread in the pool waits on select/epoll/kqueue. Any other threads in the pool wait on a condition.

Conclusion

As I mentioned above, had the review been at a different time I would have at least attempted to port some code (even if only test/benchmark code) from an in-house reactor library to asio. As it is my comments are based on a fair bit of experience with network and async (in its most general sense) I/O from the bit-level (literally) up to the application level on a number of OSs/kernels/platforms and libraries (including ACE) and I've spent a few hours reading the code and docs. I have a healthy distrust of my (and everyone elses) guesses regarding performance, and the author and review manager should consider my points above regarding performance as areas for investigation only.

A fair amount of the above appears to be a request for reactor support (which it is, partly) but I would consider mechanisms that support very light-weight proactor use highly desirable irrespective of reactor support. In particular, it would seem that some form of re-use of a single handler and buffer for an Async_Object instance without any allocation/(copy)construction of anything heavier than a pointer should be quite practical.

Asio's interface already supports "lightweight", reusable handlers, since ultimately a handler is just a function object. If you wish, you can implement your handlers so that they are no more expensive than a pointer: class my_handler { public: explicit my_handler(my_class* p) : p_(p) {} void operator()(const asio::error& e) { p_->handle_event(e); } private: my_class* p_; }; <snip> Thanks for taking the time to write such an extensive review. Cheers, Chris

Hi Darryl, --- Darryl Green <darryl.green@myrealbox.com> wrote: <snip>

I would like to see fine-grained composition from policies similar to the concepts just to make it easier to maintain and extend the lib (or maybe more the other way around ;-).

None of the above implies any major change of design, rather it takes advantage of the design to offer adaptability and portability without limiting either. I think it would be unfortunate if forms of IO that are not 100% portable were excluded from "first class" status in the library. Encouraging the utilization of the design's strengths by making it easy to re-use and extend the library can only encourage its acceptance and wider use and development. This would perhaps require some refactoring of the services and documenting/exposing some of the detail, but not much else.

What do you think?

I do appreciate where you're coming from, but I think we must have fundamentally different design philosophies in mind. To summarise my understanding of it, you want to be able to leverage commonality of implementation on the platforms that you target, and compose policies to influence how the abstractions, such as stream_socket, behave. With asio, the public interface instead says nothing about commonality of implementation, since such commonality cannot be portably assumed. The implementation detail can make use of such commonality, certainly. For the user of asio, commonality is provided through implementation of concepts, such as the Stream concept implemented by stream_socket and ssl::stream. I am not averse to compile time or runtime tweaks to alter implementation behaviour. Or, dare I say, completely different implementations of the public interface that provide alternative QoS characteristics. But ultimately I am not in favour of the sort of framework that allows endlessly customisable behaviour via policy composition for a library such as this. Of the programmers developing networking code, I imagine that very few have both the skills and inclination to make use of this level of customisation in source code. Besides, the more combinations, the greater the chance of something going wrong, and the less time I can spend testing each one! The number of permutations allowed by such a policy-driven interface would be a barrier to what I believe is a more important goal: the development of portable, reusable, higher-level abstractions. Cheers, Chris

Hi Darryl,

If the library just magically supports every form of IO on a platform out of the box, with the choice of I/O mechanism being largely transparent to higher-level code, then there will be no need for customization. I don't think that is a realistic requirement.

Here I think you have identified precisely what I intend asio to be in the longer term. It is what I mean when I say that a goal of asio is to take a toolkit, rather than framework, approach. Asio is intended for the development of applications that need to use the I/O services of their target platforms in an efficient and scalable way, without: - having to be aware of the details (or even the existence) of demultiplexing - needing to worry about platform differences. Asio is a codification of experience with demultiplexing and low-level networking APIs. It should provide an interface that requires none of this knowledge. I think it is realistic and, more importantly, widely beneficial for this experience to be captured and presented as a library. It is unrealistic to expect most projects to require their developers to learn all of these I/O mechanisms to the level of detail required to use them effectively.

I'm suggesting that with care the shared code can align with concepts/policies and, once it is (meaning it has a defined interface and semantics) it can be made public. Thats all.

So what it seems you want (and I hope I am now understanding you correctly) is access to asio's implementation details through a public interface, so that you may use them to develop your own I/O mechanisms. Let's review what I see as asio's level of abstraction: high level networking application frameworks and protocol abstractions \ | / \ | / \|/ asio toolkit /|\ / | \ / | \ platform-specific and restricted portability APIs and demultiplexing methods I have said on previous occasions that I would be happy to see asio's implementation details evolve into a secondary public interface. I recently realised that it would be more appropriate as a separate library. This library would package I/O mechanisms at the lower level of abstraction. It would be targeted at developers who require the lower level of abstraction. Asio could potentially be implemented in terms of this library, but this library is not asio. I believe it is important for a library to have all its public interfaces at the same level of abstraction. This better defines the intended scope and uses of the library. To have a library encompass multiple levels of abstraction would lead to tension in trying to support different audiences with differing needs. By making current implementation details part of the library's interface contract, you either restrict the ability to refactor the implementation, or risk creating orphaned interface elements. During the review, some people indicated a desire for lower level APIs, just as others requested higher level networking abstractions. There is nothing wrong with asking for a level of abstraction that asio does not provide. Naturally, it is not sensible to say that a library provides the wrong level of abstraction. The abstraction level is only wrong in the context of a use case. If an application requires a different level of abstraction, then asio may not be the appropriate tool.

I'm not asking for a lot of weird mix-and-match policies - just a few to compose the sorts of services that the lib should (imho) support anyway, but in a user configurable way.

But if your ultimate need is an implementation of specific I/O services (for example, pipes or UNIX domain sockets), why must they be implemented outside of asio? I agree they should be supported, so please, contribute them to the toolkit or assist in their development. Others will benefit from your expertise, just as reciprocally you may benefit from the expertise of others as captured in asio. Cheers, Chris

Christopher Kohlhoff wrote:

...

One item that I have difficulty in discerning a clear design/rationale for is the demuxer. The demuxer as implemented is a repository of services (another wooly concept afaiks) and something that provides the run, interrupt and related functions to control the scheduling/dispatching of handlers in reaction to changes in Async_Object state. Is there anything preventing a more policy-based approach to assembling services?

The demuxer is composed of services in a similar way to how std::locale is composed of facets.

That tells me that there services are objects identified by type that can be obtained from the thing that holds them (locale in iostreams, demuxer in asio) by type. Which is - um - about what I knew already....

The demuxer is not necessarily aware of, or coupled to, the service types that it contains. For example, the SSL implementation uses a service that is only added in to the demuxer if SSL is used.

I gathered that much. But tell me why/what for! Alternatively I'll try to figure it out and write the docs - you can tell me if I get them right... [pause to read code in more detail] Ok - there are a number of service categories and some services depend on other services. The stream_socket_service impl may, for example, be a reactive_socket_service. The reactive_socket_service ultimately needs a demuxer and a reactor, so it gets the reactor (itself a service) from the demuxer via get_service. It is documented that (unlike facets of a locale) the services are constructed on use. This means that there is no way (?) that a custom replacement service can be provided via the demuxer, as the actual construction of the service is done by the service_factory based on the type of service requested from the demuxer. Hence, to customize the service, it has to have a different type from any existing service, and the user of the service has to be modified to use the new type. There is a possibility that some services are used by multiple components (eg the demuxer and the socket) so care is needed in doing this customization. Basically about as unlike a local and facets as you can get? I think this is what threw me to start with. Did I get that right? Having been completely mislead by the docs, I think the real model is something like this: Having obtained the "locale" in the form of the demultiplexer the "socket" can look up services that it needs by type. Thats fine, I supposed initially that this meant a socket would look up something that actually was a concrete implementation of a documented concept, but I found that it looked up a blob-o-stuff documented as a service. As it happens that blob-o-stuff may depend on detail impl that uses detail services, some of which look suspiciously like they do implement the concepts. I'm left feeling that aside from impl detail this is all of no significance except that service instances are 1:1 with demuxer instances so it is a demuxer extension mechanism that is handy if a socket policy needs to associate state with a demuxer not a socket instance. As a special case the demuxing itself is accessed as a service. In the case of policies that are associated with (or include a component associated/implementing) the socket, variations on an idiom of using a service::null function to get a null impl (ptr) and some form of create/open etc to c'truct the actual impl is used. What I was asking for (vaguely) was that the services be more facet-like so that when customizing a socket-like class it can be assembled from appropriate facets/concepts. This might mean pulling some of the detail out of detail (leaving the platform dependent parts there). Alternatively composite services could be assembled (as appears to be the case in detail already) from their basic concepts. In any case, I think a better description of what the service concept is all about (possibly more than 1 concept here - the core service access, services that use the demuxer and demuxer services) are needed if this is to be usable as an accessible customization point of the library.

The service used by a particular public interface (e.g. basic_stream_socket) is already a policy template parameter. One example program shows how basic_stream_socket can be instantiated with a custom service to perform logging, where this service is in turn implemented using the standard stream_socket_service.

I've had a look at that now. Is there some particularly profound reason why logging implemented to fit in with the service idiom is a good thing/example? It seems to achieve nothing special, though it goes through some hoops to create a demuxer (private to the logging service) to perform work in a background thread? Is that supposed to illustrate something? I think this relates back to the need for better docs/concepts in this area. The replacement stream_socket_service is a copy of the original's interface with logging calls added. It wraps (has a reference to) an implementation that is the original asio::stream_socket_service. I agree this illustrates some of the techniques I was asking about should be practical within the design, which addresses some of my concerns. I'm happy (unless you can tell me where I've got it completely wrong!) I understand this area well enough for now. I would want see a much better level of docs and examples of this area in a post-review version of the lib. Regards Darryl Green. -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.371 / Virus Database: 267.14.7/214 - Release Date: 23/12/2005

Andrew Schweitzer wrote: [...]

demuxer's demuxer_service [...] 5) demuxer_service creates a platform-implementation demuxer_service (right?)

Not as far as I know. This is one of the points that some people have contention with. The real relation between demuxer_service and the implementation is closer to a PIMPL idiom, but without the pointer. In other words it's equivalent to: demuxer_service *is-a* detail::win_iocp_demuxer_service And... demuxer_service *is-a* detail::task_demuxer_service<detail::epoll_reactor<false> > And... demuxer_service *is-a* detail::task_demuxer_service<detail::kqueue_reactor<false> > And... demuxer_service *is-a* detail::task_demuxer_service<detail::select_reactor<false> > -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org

Andrew Schweitzer wrote:

...

Andrew Schweitzer wrote:

...

5) demuxer_service creates a platform-implementation demuxer_service (right?)

Rene Rivera wrote:

...

Not as far as I know. This is one of the points that some people have contention with. The real relation between demuxer_service and the implementation is closer to a PIMPL idiom, but without the pointer. In other words it's equivalent to:

demuxer_service *is-a* detail::win_iocp_demuxer_service

[snip]

I think I'm confused. As far as I can tell, there's no inheritance relationship between demuxer_service and win_iocp_demuxer_service, but as you said, it's a PIMPL relation, or perhaps an RIMPL relation since it's a reference not a pointer.

We can consider the reference as just a different kind of pointer in this case. After all it could just as easily be a pointer and the forwarding functions would deref the pointer, instead of the compiler doing the deref for you.

Is PIMPL considered is-a or has-a?

I've always considered it an is-a. A very special case, but equivalent.

I think the library design or naming might be a bit confused on this point (or quite possibly it's me).

The fact that it's confusing, even when looking at the code, is a point against the design ;-) OK, I'm rereading the code, instead of relying on memory...

As far as I can tell two things are true: 1) the demuxer ends up with two actual services in its list after it is constructed: a) demuxer_service b) win_iocp_demuxer_service (or the selected platform-implemenation of demuxer_service)

OK, AFAICT it only ends up with (b) in the service_registry.

2) the demuxer also has a PIMPL relation with demuxer_service, and demuxer_service has a PIMPL relation with win_iocp_demuxer_service

Yes. I guess the key point here is that there's only 1 (singleton) of win_iocp_demuxer_service for all demuxer_service instances in the same basic_demuxer instance? (again AFAICT)

Are you reading the code differently? I might have gotten lost.

I guess, I'm reading it differently :-) -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org

Hi Andrew, --- Andrew Schweitzer <a.schweitzer.grps@gmail.com> wrote: <snip>

Chris, I can spend some hours on doc in the next couple weeks if that's helpful. I'm not sure I know enough about asio or networking to actually write the doc but I'm happy to try or to review or edit. Just let me know.

Some of this stuff is likely to change post review so I'd be afraid that any docs would become out of date before they'd been written! More examples are always welcome, if you have time to spend on them :)

Here's an attempt to describe the relationship between the demuxer and the services. <snip>

Thanks for this stuff. It will make a good start for some design notes on these core classes. I will also turn some of what you wrote, particularly the example, into diagrams to show how the implementation hangs together. Thanks! Cheers, Chris

--- Darryl Green <darryl.green@myrealbox.com> wrote: <snip>

It is documented that (unlike facets of a locale) the services are constructed on use. This means that there is no way (?) that a custom replacement service can be provided via the demuxer, as the actual construction of the service is done by the service_factory based on the type of service requested from the demuxer.

Service construction can be customised by specialising the service_factory template, and then calling get_service e.g.: demuxer.get_service(service_factory<my_service>(...)); The "standard" services in the public interface are not intended for customisation in this way. However this facility can be used for user-defined services. Note: this service_factory template was introduced originally to allow the get_service member template to work with MSVC6.

Hence, to customize the service, it has to have a different type from any existing service, and the user of the service has to be modified to use the new type. There is a possibility that some services are used by multiple components (eg the demuxer and the socket) so care is needed in doing this customization. Basically about as unlike a local and facets as you can get? I think this is what threw me to start with.

Did I get that right?

The thing that makes a facet in a locale replaceable by a derived facet implementation is that the facet itself has overrideable virtual functions. It is not an attribute of the locale object, as far as I know, that determines whether a facet is replaceable. Yes it's true that the services in asio's public interface are not replaceable, as these services do not have virtual functions for reasons discussed extensively elsewhere. :) However other services could be added by the user which do. If you don't feel the comparison to locale is useful, that's fine, as it was just intended as an aid to understanding the design. I guess when I drew the comparison (which I still think is apt) I was thinking of different aspects of the way locale works. <snip>

I've had a look at that now. Is there some particularly profound reason why logging implemented to fit in with the service idiom is a good thing/example?

It aims to demonstrate how socket operations can be customised, in this case to add debugging information. Obviously the log itself is shared between all socket instances.

It seems to achieve nothing special, though it goes through some hoops to create a demuxer (private to the logging service) to perform work in a background thread? Is that supposed to illustrate something? I think this relates back to the need for better docs/concepts in this area.

The demuxer is being used as a thread-safe work queue. I agree this could be explained better in the example. Cheers, Chris

Hi Darryl, --- Darryl Green <darryl.green@myrealbox.com> wrote:

Christopher Kohlhoff wrote:

...

For stream-oriented I/O, this separation is presented in the form of the Stream concept. This concept is implemented by stream_socket, ssl::stream, buffered_read_stream and so on. It would be implemented by a hypothetical pipe class.

Precisely. Why write it/maintain it again?

Well... because although the interface might be the same the implementation is different.

...

...

A very simple active object template that supports read and write, but that relies on free functions or helper classes for less universal operations such as bind, setting options, etc, used as a base class, allows appropriate (and necessary in my opinion) use of runtime polymorphism.

I have to disagree on runtime polymorphism: compile-time polymorphism should always be the default.

A tad dogmatic. Would you like to qualify "always" at all?

I was referring to operations such as read, write, etc. I thought that was clear from the context. If not, my apologies.

What do you mean by "default" - is the default selectable at design/implementation or compilation time?

By "default" I mean that the interface should be designed to use compile time polymorphism with concepts. Runtime polymorphism can be introduced, should the developer choose, through a wrapper.

...

Runtime polymorphism can impose additional costs (such as virtual function calls on a fine-grained interface), and eliminates potential optimisations. This adds up considerably if there are many layers of runtime polymorphism involved.

Can we cut the re-education program for java programmers :-) out of this and deal with the use case constructively, or would you prefer to ignore it?

My design choice is based on experience in developing real-world high-performance networking applications that are often CPU bound. A previous system I worked on made extensive use of runtime polymorphism via the ACE_Streams abstraction, where passing data through each layer involves a virtual function call. Since configuration of the individual layers did not need to occur at runtime, this runtime polymorphism was unnecessary. Removing it markedly improved performance.

I qualified my concerns about performance by saying they lacked actual benchmark results, however I have now seen posts from others backing my concerns (between the time I spent responding and gmane being my only interface at the time I hadn't seen Caleb's results when I posted).

I do not believe that particular test is representative of typical use, as it only involves a single socket on a demuxer receiving data from another socket in the same process. Also as I have indicated elsewhere I have since made some optimisations that have significantly reduced costs.

You are now expressing concern about virtual function overhead etc with no measurements to back it up and no qualification of when they are "bad". In the development of the library I referred to in my post I have measured negligible performance impact from runtime polymorphism. A double indirection is unlikely to be significant compared to the other operations being performed.

Double indirection is not the only cost of runtime polymorphism, as you are no doubt aware. Virtual functions can also result in a loss of locality-of-reference, causing increased cache misses and possibly page faults in memory-constrained systems, and they eliminate the possibility of inlining. Furthermore, the custom allocation optimisation is possible because type information has not been erased, which would have occurred if the interface used virtual functions. Do I have the benchmarks on hand to support this? No, but I have done these tests in the past. Working on high performance systems has taught me that runtime polymorphism has very real costs, and so in general should not be introduced unless runtime variation of behaviour is required. The design of asio reflects this experience. <snip>

...

What do you think? I'd be happy to add it if you think it generally useful.

I do think it is generally useful.

...

...

Short of placing a wrapper around asio to make it more ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Yes to the following - but there is an aspect of code re-use that it doesn't address.

Which aspect would this be?

...

...

As far as I can see asio doesn't even allow interchangeable use of an ssl and a tcp stream. As I mentioned above, asio addresses this using compile time polymorphism, with the Stream concept being implemented by stream_socket and ssl::stream. For example, both work equally well with free

template !

I guess this means that you like templates about as much as I like virtual functions. :)

I'm referring to the potential for bloat if eg a pipe service duplicates much of the socket interface. I know openssl is quite weird enough to be a special case all of its own, but I would expect many systems would be able to share a lot of implementation across different stream-like file/socket/device interfaces at least.

I don't think there would be as much sharing as you might think. And even in cases where there is some sharing, the common functionality can be extracted into a function or class to be called from each place, so I don't see how this design necessarily leads to bloat. Fundamentally, asio's public interface cannot assume sharing, since that assumption is already known to be incorrect. Cheers, Chris P.S. Still reading your other replies. I hope to respond to them soon.

Christopher Kohlhoff wrote:

...

documentation mentions the memory usage overhead of the proactor emulation on systems using select/epoll. However it is clear that there is considerable allocation overhead and construction overhead involved (and not only on those systems). While much discussion has centered on allocator performance, I'm concerned that the user/library constructor code invoked for each i/o operation is likely to be non-trivial as well. For bulk transfers, it is likely that these issues won't be too severe, however for systems where low-latency handling of small messages is critical, it is quite possible that these overheads will be comparable to or larger than any other part of the processing for a message/transaction.

I don't deny that there may be room for performance improvement in the implementation. But as I recently tested the custom allocation with Win32 I/O completion ports, and found that it performs almost identically to synchronous I/O, I don't think the public interface imposes any unnecessary inefficiencies.

Caleb's results on Linux aren't too encouraging though.

...

I find the way the design supports, and the documentation encourages, breaking up handling of structured messages into a small header read followed by a larger read (with appropriate handler for expected body/chunk) while the implementation appears to do some quite heavyweight operations compared to just reading a small header to be almost an encouragement of an antipattern for efficient I/O.

I don't agree that asio favours this approach over others. But I do believe that this approach is perfectly valid and extremely

Ok - my concern is that the examples should show best practice and the lib should make it easy to do things the right way. Using the lib as currently implemented, in the way shown somewhere (I think maybe this was just in a post - I can't find it in the examples which was where I imagined I saw it - sorry for the noise) would give poor performance. This should be able to be addressed by: i) Improving lib efficiency ii) Providing examples of the right way to do it iv) Providing an interface that is more efficient I believe you intend to do (i) and below you offer some alternatives for (ii) so this would seem to be well enough covered.

useful, since it permits asynchronous code to be written in terms of "contracts". I.e. perform this operation (or operations) and don't come back to me until it's all done or an error has occurred.

I tend to implement that at a layer above the raw I/O, but I take your point.

stream_socket my_socket;

with:

buffered_read_stream<stream_socket> my_socket;

Ah - that would be a closer approx to how I would expect this to be done - docs!

...

I would expect an optimistic reactive approach (read the header when notified, determine size, handler etc and opertunistically attempt a read of the body) to be significantly faster.

Let's take your example of a fixed sized header followed by a body, and compare and contrast a reactive approach with the alternatives offered by asio.

---------------------

1. Reactive. 2. Asio with multiple reads.

This is the approach where we issue asynchronous reads for exactly the length of the header, and then the body. It may be less efficient, but only because of the additional pass through the demultiplexer, not because it makes more read() system calls

Yes - precisely the problem. The select-based demuxer needs to be made more efficient by optimization or a modified interface.

than the above. However feel free to compare in terms of readability :) The main reason it's clearer is that you no longer have to deal with short reads -- async_read's contract takes care of that for you.

I think you will find reasonable reactive libs/async buffering providers etc do much the same thing for readability.

3. Asio with multiple reads + opportunistic body read.

This extends number 2 by adding in the opportunistic read of the body. In effect this makes it equivalent to your reactive read

Yes. It seemed like an odd mix of sync and async IO so I didn't consider it as an idiom to recommend in general.

4. Asio with bulk reads.

This approach is exemplified by the HTTP server example. Essentially it reads as much as is available (up to some limit of course) in one go, and then processes it using some state machine similar to that in number 1. It is generally the most efficient since it requires the fewest number of read system calls or passes through the demuxer.

Yes. This is fine for streaming/bulk transfers. My concern was more about an idiom for lots of short request/response type (low latency needed) messages.

5. Asio with transfer_at_least(...).

In certain applications you may be able to combine the opportunistic read into the initial asynchronous read operation,

The circumstances where this applies for TCP are hard to imagine, but if you had char device IO I would have a use for this (mind you, the device in question always returns the full message or nothing, so it is not quite a match either). Thanks for pointing out the alternatives, and maybe at least some of these should get a mention in docs in the context of some more substantial tutorial, but better would be to just ensure that the simplest approach/interface is also efficient and illustrate that.

...

It is implied that an appropriate demuxer/demuxer service can be used to implement various multi-threaded processing schemes. This may well be true. However, an alternative approach is to simply consider the affinity of an Async_Object and its demuxer(s) (one for read, one for write) to be dynamic and run a number of separate (per thread) demuxers, moving Async_Objects between them as required. In this way the handlers for a given Async_Object will only run in the expected context.

This is not portable. With Win32 I/O completion ports for example, once a socket is associated with an I/O completion port it cannot be disassociated. The demuxer design reflects this.

Ah - that makes it a bit messy. Do you consider that the locking dispatcher effectively provides the same result? I suspect it does so long as all access to what amounts to an active object guarded by the dispatcher does in fact go through the dispatcher. I assume this is the intent and other interfaces (other than I/O based ones I mean) should be wrapped and dispatch or post used to run them? An example would help a lot here.

...

case. I haven't reviewed the code to a level of detail where I can be sure I am understanding the dispatching done by the epoll and select reactors, but it looks to me as though multiple threads may all wait on the same fdset (or equivalent)? Is this right? Intended? I had expected some form of modified leader/followers pattern here?

No, only one thread in the pool waits on select/epoll/kqueue. Any other threads in the pool wait on a condition.

I believe you - just can't see how this works when multiple threads call run() on the same demux. I must be missing something in how the service actually gets invoked.

...

In particular, it would seem that some form of re-use of a single handler and buffer for an Async_Object instance without any allocation/(copy)construction of anything heavier than a pointer should be quite practical.

Asio's interface already supports "lightweight", reusable handlers, since ultimately a handler is just a function object. If you wish, you can implement your handlers so that they are no more expensive than a pointer:

class my_handler { public: explicit my_handler(my_class* p) : p_(p) {} void operator()(const asio::error& e) { p_->handle_event(e); } private: my_class* p_; };

Yes I can - but the lib should make simple things simple. I don't anticipate that I would make much use of the dynamic (per request) handler facility for architectural reasons and I image a lot of use would be similar. Providing an easy way to associate a persistent handler with a stream/async object and making this efficient would be relatively easy wouldn't it? Simple use would then use bind and allocate a boost::function or similar once, not per request, with correspondingly better performance. Regards Darryl Green. -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.371 / Virus Database: 267.14.7/214 - Release Date: 23/12/2005