Hello all, I have just uploaded a draft version of Boost.Asio to the vault (in the "Input - Output" folder). I'm particularly interested in feedback on the handler-based custom memory allocation and IPv4/v6 protocol independence. Please note: this is not a properly tested release. Caveat emptor. It has the following known issues: - It uses getaddrinfo/getnameinfo, but these functions are not thread-safe on Mac OS X. They will be replaced by thread-safe emulation using getipnodebyname and getipnodebyaddr. - Although builds using MSVC or Borland C++ automatically emulate getaddrinfo/getnameinfo on Windows 2000 and earlier, MinGW does not. Therefore MinGW currently only supports Windows XP and Windows Server 2003 targets. - It has not been compiled or tested at all on Solaris. It has been tested on Mac OS X 10.4, Debian Sarge with 2.6 linux kernel and Windows XP. Cheers, Chris Changes since 0.3.6 =================== (Excluding minor bug fixes and in no particular order.) * Renamed demuxer to io_service. * All services inherit from a common base io_service::service. As part of this change, the demuxer::get_service() member template function has been replaced with three free template functions: use_service(), add_service() and has_service(). * Resource classes inherit from basic_io_object. Sockets, timers, etc inherit from basic_io_object<Service>. This class automatically obtains the correct service object from the io_service, and provides the typedefs and functions that all objects must provide (e.g. an io_service() member function for obtaining the associated io_service object). * All handler objects destroyed during io_service destruction. Previously, if you did not allow demuxer::run() to continue until it had no more work to do, and then destroyed the demuxer, uninvoked handlers would never be destroyed. All stored handler objects that are "owned" by the io_service or any associated services are now destroyed automatically from inside the io_service. This allows quick, clean shutdown of programs using io_service::interrupt(). This handler cleanup has been implemented using "two-phase" service shutdown. Every service that inherits from io_service::service must implement a virtual function called shutdown_service(). The service's implementation of this function is required to destroy all copies of handlers that it owns (directly or indirectly). The cleanup must be done in a separate phase prior to the destruction of the service objects, since the handler destructors may try to access other services. * Renamed locking_dispatcher to strand. A strand is defined as a strictly sequential invocation of event handlers (i.e. no concurrent invocation). Use of strands allows execution of code in a multithreaded program without the need for explicit locking (e.g. using mutexes). Strands may be either implicit or explicit, as illustrated by the following alternative approaches: - Calling io_service::run() from only one thread means all event handlers execute in an implicit strand, due to the io_service's guarantee that handlers are only invoked from inside run(). - Where there is a single chain of asynchronous operations associated with a connection (e.g. in a half duplex protocol implementation like HTTP) there is no possibility of concurrent execution of the handlers. This is an implicit strand. - An explicit strand is an instance of asio::strand. All event handler function objects need to be wrapped using asio::strand::wrap() or otherwise posted/dispatched through the asio::strand object. * Add macros to disable platform-specific implementations. The following macros can be used to disable platform-specific implementation code and revert to portable select-based asynchronous I/O emulation: - BOOST_ASIO_DISABLE_EPOLL - BOOST_ASIO_DISABLE_IOCP - BOOST_ASIO_DISABLE_KQUEUE * Handler-based custom memory allocation. Many asynchronous operations need to allocate an object to store state associated with the operation. For example, the Win32 implementation needs OVERLAPPED-derived objects to pass to Win32 API functions. Furthermore, programs typically contain easily identifiable chains of asynchronous operations. A half duplex protocol implementation (e.g. an HTTP server) would have a single chain of operations per client (receives followed by sends). A full duplex protocol implementation would have two chains executing in parallel. Programs should be able to leverage this knowledge to reuse memory for all asynchronous operations in a chain. Given a copy of a user-defined Handler object h, if asio needs to allocate memory associated with that handler it will execute the code: void* pointer = asio_handler_allocate(size, &h); Similarly, to deallocate the memory it will execute: asio_handler_deallocate(pointer, &h); These functions are located using ADL. The asio library provides default implementations of the above functions in the global namespace: void* asio_handler_allocate(size_t, ...); void asio_handler_deallocate(void*, ...); which are implemented in terms of ::operator new() and ::operator delete() respectively. The implementation guarantees that the deallocation will occur before the associated handler is invoked, which means the memory is ready to be reused for any new asynchronous operations started by the handler. Custom memory allocation support is not yet used throughout the asio implementation. In particular, the reactor-based emulation still uses dynamically allocated objects. It is fully implemented for the Win32-specific I/O completion port code. The custom memory allocation support has also been disabled for Borland C++, since it generated obscure compiler errors. See libs/asio/example/allocation/server.cpp for an example of how to use the handler-based custom memory allocation. * Allocator template parameter removed. Removing the Allocator template parameter reduces the number of core types in asio and makes it easier to develop custom services (i.e. you don't have to worry what Allocator object the io_service has been templated on). It should also make it simpler to create a shared or static library-based implementation. A combination of handler-based custom memory allocation, and the ability for the asio implementation to embed objects directly into object internals (e.g. socket implementations), should minimise dynamic memory allocations. If there is a demonstrable need, I may consider adding an io_service::service-based allocator to allow memory allocation to be further customised on a per-io_service-object basis. * Add support for building without thread support. POSIX platforms only. The Windows implementation still requires threads. * Async waits cancelled by deadline_timer expires_*() setters. Outstanding asynchronous wait operations are now automatically cancelled if you call expires_at() or expires_from_now() to modify the expiry time of a deadline_timer. The previous documented behaviour was that it was "undefined" to modify the expiry time while there were outstanding waits. * Support for reference-counted buffers. The Const_Buffers and Mutable_Buffers concepts have been modified to allow reference counted buffers to be used. In particular: - The {Const|Mutable}_Buffers::iterator typedef and non-const begin()/end() functions have been removed from the concepts' requirements. Implementations of the concepts now need only provide const_iterator support, and do not need to allow in-place modification of the elements. - The asio implementation guarantees that at least one copy of the buffers object will be maintained for as long as the system may need access to the data. - The asio implementation explicitly converts the dereferenced const_iterator into mutable_buffer or const_buffer as needed. See the libs/asio/example/buffers/reference_counted.cpp example program. * Strongly-typed socket classes. Rather than asio::stream_socket, asio::datagram_socket and asio::socket_acceptor classes that were used for all protocols, these socket classes are now specific to each protocol. For example, we now have: ip::tcp::socket ip::tcp::acceptor ip::udp::socket and in the future we might have: unix::stream_protocol::socket bluetooth::rfcomm::socket and so on. These protocol-specific classes are simply typedefs for templates instantiated on the corresponding Protocol class. E.g.: template <typename Protocol> class stream_socket_service; template <typename Service> class basic_stream_socket; class tcp { ... typedef stream_socket_service<tcp> socket_service; typedef basic_stream_socket<socket_service> socket; ... }; New protocols can be added by creating classes like tcp above. * Use inheritance in socket types to prevent "layer violations". The basic_stream_socket and basic_datagram_socket templates now inherit from basic_socket<Service>. This class includes all functions that are common to stream and datagram sockets (everything except send/receive and friends). Stream layers such as ssl::stream return a reference to basic_socket<> from the lowest_layer() function. * IPv4 and IPv6 protocol independence. Socket, endpoint and address classes for IP (i.e. TCP and UDP) are independent of the particular IP version (4 or 6) used. The ipv4 namespace has been removed. For example: - The ipv4::address class has become IP version-independent ip::address. If you need to manipulate addresses for a specific IP version use ip::address_v4 or ip::address_v6. - The protocol classes are now ip::tcp and ip::udp, and the corresponding endpoint types are ip::tcp::endpoint and ip::udp::endpoint. - To open a socket you must specify the protocol version, e.g.: ip::tcp::socket socket(io_service); ... socket.open(ip::tcp::v4()); // IPv4 ... socket.open(ip::tcp::v6()); // IPv6 Note: if you already have an endpoint you can avoid this by using the protocol associated with the endpoint object: socket.open(endpoint.protocol()); - When constructing an endpoint with just a port number, you must specify the IP version: ip::tcp::endpoint(ip::tcp::v4(), 12345); The development of programs that are independent of the IP version is further aided by the new resolver interface outlined below. * Resolver replaces ipv4::host_resolver. The ipv4::host_resolver and ipv4::host classes have been removed in favour of an endpoint-oriented interface that can support different protocols. The functionality of this new interface is based on getaddrinfo and getnameinfo. Use the resolver by constructing a query, calling resolve() (or async_resolve()), and using the returned forward-only iterator to enumerate the endpoints that match the query. For example: ip::tcp::resolver resolver(io_service); ip::tcp::resolver::query query("foobar.com", "http"); ip::tcp::resolver::iterator iter = resolver.resolve(query); ip::tcp::resolver::iterator end; while (iter != end) { ip::tcp::endpoint ep = *iter; std::cout << iter->endpoint() << std::endl; std::cout << iter->host_name() << std::endl; std::cout << iter->service_name() << std::endl; ++iter; } The asynchronous equivalent is: void my_class::start_resolve() { ip::tcp::resolver resolver(io_service); ip::tcp::resolver::query query("foobar.com", "http"); resolver.async_resolve(query, boost::bind(&my_class::resolve_handler, this, asio::placeholders::error, asio::placeholders::iterator)); } void my_class::resolve_handler(const asio::error& e, ip::tcp::resolver::iterator iter) { if (!e) { ip::tcp::resolver::iterator end; while (iter != end) { ... ++iter; } } } There are overloads of resolve()/async_resolve() that take an endpoint and return an iterator. These are intended for reverse lookups. Since the ipv4::host_resolver had a local_host() function that returned information about the local system, a new function ip::host_name() has been added which returns a string containing the name of the system. * Multicast socket options renamed. The IP-specific socket options for multicast have been renamed. They are now also independent of the IP version. The name changes are: - add_membership is now join_group - drop_membership is now leave_group - time_to_live is now hops These multicast socket options are now found in the ip::multicast namespace. * String-to-address conversion now uses from_string function. This change was driven by three things: - The need to indicate an error without throwing if a conversion from string to address fails. - String literal addresses would not be common and so support for them does not need to be convenient. - It is more common to use a resolver to convert strings into complete endpoints. To convert a string into an address you would now write: ip::address address = ip::address::from_string("1.2.3.4"); which throws an asio::error on failure, or: asio::error error; ip::address address = ip::address::from_string( "1.2.3.4", asio::assign_error(error)); to convert without throwing. Similar functions exist on the ip::address_v4 and ip::address_v6 classes. * IPv4 and IPv6 addresses can be converted to and from bytes. The ip::address_v4 and ip::address_v6 classes contain: - A typedef called bytes_type, which is a boost::array<unsigned char, N>. For IPv4, N is 4. For IPv6, N is 16. - A constructor that takes a bytes_type value. - A function to_bytes() that returns a bytes_type value. * Order of endpoint constructor params reversed. The old ipv4::tcp::endpoint class (and udp equivalent) had a constructor that took a port and address in that order. The new ip::tcp::endpoint and ip::udp::endpoint classes have the order reversed, since that is more idiomatic. * Change position of flags param on send_to, receive_from. The order of parameters on the basic_datagram_socket's send_to, async_send_to, receive_from and async_receive_from functions have been changed to put the endpoint before the message flags. Overloads of these functions have been added that do not include the flags parameter, which gives the flags a default value of 0. A similar change has been made for the send, async_send, receive and async_receive functions on basic_datagram_socket and basic_stream_socket. * Don't globally block SIGPIPE if an alternative is available. On Linux, the MSG_NOSIGNAL flag is now passed to the sendmsg function. On Mac OS X, the SOL_SOCKET/SO_NOSIGPIPE socket option is set on all sockets when they are created. The SIGPIPE signal is globally ignored on Solaris. * Reactor implementations run operations until complete. The reactor socket implementation sets non-blocking I/O for asynchronous emulation (non-blocking mode is enabled for a socket the first time an async call is made on that socket). Operations are now restarted automatically if the asynchronous operation returns would_block or try_again, meaning that asynchronous operations are guaranteed never to return these errors. Changes planned for 0.3.7 ========================= * Fix tutorials/examples to not use "C style" code. * Emulation of getaddrinfo/getnameinfo on platforms where it is unavailable or broken. Changes planned for post-0.3.7 ============================== * Iostreams support. * Split_error handler function object. * Overloads of io_service::run() that take a timeout parameter. This is to allow calls to run() to be interleaved with other application code. The run() function will return an enum value indicating why it finished (e.g. timed_out, out_of_work, interrupted, etc). * Polymorphic wrapper for the Dispatcher concept. * Polymorphic wrapper for the Stream concept. * Investigate handler-invocation hooking. * Develop a performance test framework. * Rework reactor implementations to support handler-based custom memory allocation. * New reactor implementations, e.g. /dev/poll and Solaris 10 ports. * Use the Windows API call ConnectEx where available for true asynchronous socket connect operations. * Add better support for line-based protocols. This might take the form of an {async_}read_line function or equivalent. * Add UNIX domain socket support. * Documentation, documentation, documentation. I hope that's everything!