[afio][v1.30 "C++ Now edition" released]
Dear Boost, I am very pleased to announce the v1.30 release of proposed Boost AFIO, a Boost library implementing asynchronous file input/output which has been in the review queue since 2013. This is the version of AFIO which will be overviewed in my talk at C++ Now 2015. It also demonstrates the validity of the clang AST driven BindLib standalone Boost library conversion technique which will also be overviewed at C++ Now. Source distro: https://github.com/BoostGSoC13/boost.afio/archive/v1.30_Boost_v1.57.zi p Docs and CI dashboard: https://boostgsoc13.github.io/boost.afio/ Major new features in the v1.30 release: * Performance was improved by removing the transactional memory support in the v1.2 engine which is now compile time enabled only. Throughput now reaches up to 85% of raw ASIO and average latencies were reduced by 48%, yielding a 15 microsecond average latency with +/- 0.15 microsecond at a 95% confidence interval. This should enable AFIO to handle the latest generation PCIe SSDs with their 80 microsecond latencies reasonably well. * AFIO is now capable of complete standalone usage no Boost dependencies whatsoever, including for its unit testing via a Boost.Test emulation. * Due to its use of libclang AST generated template aliasing to allow user swapping of STL 11 implementation, minimum compilers required are now VS2013, GCC 4.7 and clang 3.2. All the VS2010 workarounds baggage has been eradicated. * All things related to async_data_op_req are now fully metaprogrammed with automatic generation by the compiler of suitable ASIO scatter-gather buffers from input types and STL containers with correct scatter-gather buffer coalescing where safe to do so. This automatic generation is user customisable via free function specialisation. * Support for relative paths from an open file descriptor was added. This lets you specify a file descriptor as a base from which a path fragment operates with the path resolution atomically performed in the kernel, and therefore the path of the base file descriptor may rapidly change safely. * AFIO is now optimally filing system race condition safe on all supported platforms, with per-API documentation of how much race safety is guaranteed per platform. A soak unit test verifies per commit that perfect race safety under severe and rapid filing system change is maintained. * A new afio::path type which very thinly wraps filesystem::path was added. The new afio specific path type makes the management of NT kernel path to Win32 path transition and conversion explicit, and therefore much harder to accidentally get wrong. A new normalise_path() function can convert a NT kernel path to a traditional Win32 path, complete with drive letter. * New platforms supported: Apple OS X and Android 5.0. * New filing system features support: All platforms: sparse file support, storage extent enumeration, extent deallocation ("hole punching"), volume metadata fetching (statfs), open file handle path discovery, hard link support, atomic relink (rename) support. Windows only: file symlinking, NT lock file performance flags (lock file performance is now not far from POSIX), workaround semantics for all the ways Windows will surprise you on the filing system such as files and directories which won't delete. AFIO now does a fair job of replicating POSIX filing system semantics on Windows (and better than Mingw/Cygwin presently). Linux: symlinks are now opened as file descriptors on kernels supporting that feature. * Doubled the size of the tutorial, with all the new material being non-toy examples. There is a particularly nifty "File locking via only atomic appending and hole punching" algorithm implementation in there, and it holds up surprisingly well compared to traditional file locks except on ZFS. Plus, it's completely portable. * Some utility routines in preparation for AFIO v1.4: a cryptographically strong random filename generator, a hugepage TLB STL allocator useful for efficient file copy buffers, a very highly optimised C++ 14 constexpr constructing arbitrary Hamming code SECDED template class (an amazing 22 cycles per byte for the 32784,32768 code), and secure to-hex and from-hex validating converters. * No shortage of bug fixes, helped hugely by additional unit testing for correct error handling. Detailed changelog can be found at https://boostgsoc13.github.io/boost.afio/doc/html/afio/release_notes.h tml. I'll tag a Boost 1.58 release on github once Boost 1.58 is out. This release took far longer than I originally intended mainly due to the extensive debugging of weird filing system races, and for six weeks now I've been working till 5am every night after my day job. I cannot tell you how glad I am to see this release out! My enormous thanks to my family who have put up with a lot for this release, and to my employer for a fairly dopey employee recently. Bed, here I come! Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On 18 Mar 2015, at 04:47, Niall Douglas
wrote: Dear Boost,
I am very pleased to announce the v1.30 release of proposed Boost AFIO, a Boost library implementing asynchronous file input/output which has been in the review queue since 2013.
[...]
This release took far longer than I originally intended mainly due to the extensive debugging of weird filing system races, and for six weeks now I've been working till 5am every night after my day job. I cannot tell you how glad I am to see this release out!
Hi Niall, You are obviously very proud of this library and I believe rightly so. You've worked hard on it and in terms of things like soak testing it probably sets a new high watermark for Boost libraries. You're also a highly active member of this community. So why is the library always "proposed".... When is there going to be a review? :) Regards, Pete
On 18/03/2015 05:47, Niall Douglas wrote:
Dear Boost,
I am very pleased to announce the v1.30 release of proposed Boost AFIO, a Boost library implementing asynchronous file input/output which has been in the review queue since 2013. This is the version of AFIO which will be overviewed in my talk at C++ Now 2015. It also demonstrates the validity of the clang AST driven BindLib standalone Boost library conversion technique which will also be overviewed at C++ Now.
Source distro: https://github.com/BoostGSoC13/boost.afio/archive/v1.30_Boost_v1.57.zi p
Docs and CI dashboard: https://boostgsoc13.github.io/boost.afio/
Hi, this is my first time actually looking at the documentation of the library. I think it would be interesting to develop a couple more things in the design or FAQ sections. Why use a specific asynchronous model with its own when_all etc.? There is a standard way of defining asynchronous tasks, which is getting extended by compiler support using await/yield_value, so why not do something using that? What about the relationship with ASIO? ASIO has its own event-driven main-loop system for asynchronous tasks, can AFIO use that and can ASIO use AFIO's future method for other types of I/O too?
participants (3)
-
Mathias Gaunard
-
Niall Douglas
-
Pete Bartlett