On Mon, Jul 3, 2017 at 9:54 AM, Emil Dotchevski via Boost <boost@lists.boost.org> wrote:

... The question is must Beast be coupled to Asio, not can it be used without something like Asio. And I don't mean just in the case of someone using serializer/basic_parser and nothing else.

In my opinion, arguing that coupling is necessary should begin with defining an interface which Beast can use, which can be trivially implemented in terms of Asio. In designing this interface the only concern should be avoiding the coupling; specifically, performance considerations should be completely ignored. Only then we can have a reasonable discussion is it worth it. Regardless, in my experience this kind of exercise always improves the design of a library.

The question is, can Beast be written against generic concepts which define synchronous and asynchronous operations over buffer oriented streams, instead of being tied to Boost.Asio? First lets imagine what such generic concepts might look like, independent of Beast. We will start with the synchronous case which is the easiest. In the code which follows: * `Stream` is a type representing a buffer oriented I/O stream * `Buffers` is type representing an ordered sequence of zero or more non-owning references to contiguous octet buffers * `Error` is type representing an error code The following synchronous stream operations are defined: /** Read data from a stream synchronously `stream` The stream to read from `buffers` The buffers to read into `error` Set to the error if any occurred. returns: The number of bytes transferred, or 0 on error. */ std::size read( Stream& stream, Buffers const& buffers, Error& error); /** Write data to a stream synchronously `stream The stream to write to `buffers The buffers to write from `error` Set to the error if any occurred. returns: The number of bytes transferred, or 0 on error. */ std::size_t write( Stream& stream, Buffers const& buffers, Error& error); This looks pretty generic and at this point users familiar with Boost.Asio might raise their hands to tell me that the interfaces look identical. This is just a coincidence because, lets face it - synchronous interfaces don't come in that many varieties. I don't think synchronous interfaces are particularly interesting and Beast could certainly be made to use something like this but even with simple interfaces some issues come up: 1. What is the concrete type of Error? Or is Error a template parameter that meets certain requirements? 2. HTTP stream read algorithms need to know when the stream, is closed, what error code represents End-of-File? 3. What are the requirements of Buffers? Beast could be written against generic concepts, but in order to do so the questions above need to be answered (points 1 through 3). Otherwise there is no specification and thus, no way to write an algorithm. We will come back to these questions, but now on to the next topic. We want Beast to also provide HTTP operations on asynchronous streams, so we need to have a generic concept against which to work. There are many models of asynchronous computation. For example there are futures, coroutines, callbacks, and queues (I can't really think of any others at the moment). Lets imagine what callback-based generic asynchronous stream read and write operations might look like: In the code which follows: * `Stream` is a type representing a buffer oriented I/O stream * `Buffers` is type representing an ordered sequence of zero or more non-owning references to contiguous octet buffers * `Error` is type representing an error code * `Callback` is a type which is Callable with the signature `void(Error, std::size_t)` The following asynchronous stream operations are defined: /** Begin an operation to read data from a stream asynchronously `stream` The stream to read from `buffers` The buffers to read into `callback` A callback to invoke when the operation completes. The callback will receive the error if any, and the number of bytes transferred. returns: The number of bytes transferred, or 0 on error. */ void async_read( Stream& stream, Buffers const& buffers, Callback const& callback); /** Begin an operation to write data to a stream synchronously `stream` The stream to write to `buffers` The buffers to write from `callback` A callback to invoke when the operation completes. The callback will receive the error if any, and the number of bytes transferred. returns: The number of bytes transferred, or 0 on error. */ void async_write( Stream& stream, Buffers const& buffers, Callback const& callback); Now anyone who knows Asio is going to stop me and say, hey! This looks a lot like Asio! But that's a side issue because this generic interface I have provided has some defects. Or rather, you can say that it is under-specified. We have the same questions about the requirements for the types Error and Buffers, and the value representing End-of-File. But now there are even bigger questions with significant ramifications: 1. Upon which thread is the callback invoked? 2. Who invokes the callback and how is it invoked? 3. How is access to the `stream` synchronized? 4. What about futures and coroutines? The simple generic free function signatures alone which I provided above are not sufficient to describe a complete model for asynchronous operations on streams. We also need provisions to answer the questions above. Does the implementation provide the threads, or are they provided by the user? Does synchronization use mutexes? Lock-free programming? Queues? Back to this in a moment... Some folks who either dislike callbacks or just happen to love futures may raise an objection that the callback-based generic model I have described does not capture their use case. Actually, Christopher Kohlhoff (author of Boost.Asio) demonstrates how the callback-based model is a proper superset of all other asynchronous computation models. And he shows that callback-based models provide the greatest opportunity for performance and efficiency by doing away with unnecessary synchronization points. This is explained in his paper N3896 "Library Foundations for Asynchronous Operations" and shows how initiating functions like `async_read` and `async_write` above may be trivially modified to support not just callbacks but also futures, coroutines, and user-defined types. Beast fully supports this system and even comes with tutorials and helper classes which let you implement the model yourself in your own initiating functions. "Library Foundations for Asynchronous Operations" (N3896) <http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf> "Writing Composed Operations" (Beast) <http://vinniefalco.github.io/stage/beast/review/beast/using_networking/writing_composed_operations.html> Thankfully this solves one piece of our puzzle. N3896 allows us to say with certainty that our hypothetical generic network model should use callbacks as the foundational notification mechanism for the completion of asynchronous operations. Perhaps we can look towards the author of N3896 for more solutions? Drumroll... Working Draft, C++ extensions for Networking (N4588) a.k.a. "Networking-TS" <http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/n4588.pdf> What is this you ask? Well... this is a working draft which describes a proposed set of C++ extensions for Networking. It has TCP/IP sockets and UDP streams, serial ports I think (?) but lets not worry about that. Most importantly, the Networking-TS defines a generic model for synchronous and asynchronous stream-oriented I/O operations! This means that: Synchronous and asynchronous algorithms written against Networking-TS concepts will work with any stream types that meet the requirements ** Yes, Networking-TS comes with generic stream concepts. Lets restate the original question: "Can Beast be written against generic concepts which define synchronous and asynchronous operations over buffer oriented streams?" The answer is: Yes. Beast is already written against generic concepts which define synchronous and asynchronous operations over buffer oriented streams. These are provided by Boost.Asio. Specifically, these concepts: SyncReadStream <https://timsong-cpp.github.io/cppwp/networking-ts/buffer.stream.reqmts.syncreadstream> SyncWriteStream <https://timsong-cpp.github.io/cppwp/networking-ts/buffer.stream.reqmts.syncwritestream> AsyncReadStream <https://timsong-cpp.github.io/cppwp/networking-ts/buffer.async.read> AsyncWriteStream <https://timsong-cpp.github.io/cppwp/networking-ts/buffer.async.write> Its not the responsibility or goal of Beast to invent a new generic model of synchronous or asynchronous networking. That's already been taken care of. Beast follows it. It is the job of authors to make sure their network implementations meet the requirements of the various Networking-TS stream concepts. Once they do that, then any algorithms written against the concepts can now work with these implementations. Again folks might raise their hand and say "but Beast depends on Boost.Asio!" and they would be right. The reality is that Networking-TS has not entered the standard and is not uniformly available across compiler vendors. Boost.Asio is the closest thing to the standardized version of Networking-TS, so Beast is written against that. Its not perfect, but it is the best we can do and the runner-up is not even in the race (are there any other generic stream designs?). I believe that the burden of proof rests with those who claim that Beast should be written against a different generic stream design. Please show me one such design that is mature, well-specified, and in-use, other than Boost.Asio. tl;dr; Boost.Asio's models of synchronous and asynchronous buffer-oriented streams are already generic, and Beast is written against Boost.Asio. Thanks