On 29 Nov 2014 at 14:59, Bjorn Reese wrote:
On 11/26/2014 08:10 PM, Olaf van der Spek wrote:
Not many responses... Does noone use native IO?
The other problem with anything Boost.Iostreams based is that Iostreams has a severely flawed design in not supporting async i/o. This makes Iostreams useless for almost any task I've needed it for in recent years. It also doesn't help that it lacks a maintainer, and has many annoying bugs (I recently had to use Iostreams due to Boost.Process 0.5 forcing me, and I lost over a week of my time fighting Iostreams to behave when combined with ASIO. It was quite frankly a disaster).
The reason I have not responded is because I would much rather have an asynchronous file I/O API (along the lines of Asio) with synchronous convenience functions. But this is a major task.
As Bjorn only knows too well but Olaf may not, proposed Boost.AFIO (https://boostgsoc13.github.io/boost.afio/doc/html/afio.html) provides portable asynchronous file i/o implementation library extending ASIO. It however does not use the ASIO handler callback idiom as that is (in my opinion, though TBH I am right on this) unsuited for file i/o where _ordering_ of operations is paramount. The ability to easily program ordering makes an enormous difference to ease of use and tweaking algorithms for performance on cold and warm cache file systems. Instead AFIO uses a functional result passing design currently based on plain old futures. Benchmarks for the v1.2 engine are more than plenty for even a top end SSD (https://boostgsoc13.github.io/boost.afio/doc/html/afio/FAQ/closure_pe rformance.html) but latencies are in my opinion not good enough (https://boostgsoc13.github.io/boost.afio/doc/html/afio/FAQ/closure_la tency.html), so the work plan is as follows: v1.3 engine: Port Boost.AFIO to proposed Boost.BindLib, thus making AFIO standalone capable. This part is nearly done, it's mainly all dotting i's and crossing t's to go, mainly on older compilers like GCC 4.6. The red items on the CI dashboard (https://boostgsoc13.github.io/boost.afio/) are those todo fixme support items apart from anything before VS2013 support for which I am dropping. v1.4 engine: Replace spinlocked unordered_map with proposed Boost concurrent_unordered_map. This should yield a substantial performance increase due to hugely improved concurrency and a large decrease in latencies due to eliminating much use of malloc. v1.5 engine: Replace async_io_op with a custom future type based on the non-allocating constexpr continuations based basic_future I proposed on this list some time ago. This should completely eliminate malloc from the AFIO core engine, and I would assume we shall see perfectly optimal performance and latencies in AFIO. A "free" bonus is that AFIO becomes monadically programmable using either proposed Boost.Hana or proposed Boost.Expected, or indeed any monadic C++ programming library of your choice (if it provides basic_future support). This ought to make "simple" use of AFIO much more amenable and its current batch based API is a big gnarly for occasional users. If approved to present at C++ Now, I shall present on all this there for those interested, and I currently intend to submit a talk or two to CppCon. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/