On 21 Jul 2015 at 9:48, Robert Ramey wrote:

...

...

Did the review start yesterday?

Short answer: Sadly I believe not. I was unhappy with the tutorial in the documentation.

Longer answer: I've been investing 40 hours per week after work into

I wouldn't stress about this. A Boost project different than a work project. At work we have to get it done more or less on time even if we have to compromise features, quality, documentation, or whatever.

Here we have to get it perfect and complete - whatever it takes. It's an obsession. Schedule means nothing here. (Note this is development - maintainence is a totally separate issue.)

This is one of the key things that makes boost different. Schedule, effort spent, time, personal relationships, .... everything takes the back seat to quality and completeness. We aspire to provide the definitive implementation of any given idea. This is why only we can do this work. This is what makes us special and irreplaceable.

Good luck with this.

Thanks for the support Robert. I wouldn't say we aim here to get a Boost library perfect and complete though, rather I'd say we prioritise what ought to be prioritised if one is aiming for the best possible engineering (according to each of our personal opinions on that) tempered by the scarcity of personal resources and time and family demands. For example, this v1.4 API refactor I should have done last year, but refactoring APIs is tedious and boring, and the old one works just fine, it's just quite far from the Concurrency TS now so I put it off. So getting ready for this review was really about motivating myself to invest 350 hours of outside of work time in doing all the stuff I kept meaning to do but couldn't motivate myself to normally do because it's so tedious. Indeed, the lightweight monadic future promises I was banging on about eighteen months ago, the big hill for me to climb was actually writing some code to implement them as it was always going to be a 250 hour minimum, which is tough to find after your day job. Still, it's done now, and like APIBind I think lightweight monads are a real game changer for how one writes C++, not least the heterogeneous future wait composure. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 21 Jul 2015 at 20:07, Glen Fernandes wrote:

I hope you're not making this harder on yourself than it needs to be. Perhaps I need to understand better: You must have felt the library was functionally stable and maybe even fit for production code at some point (because it was in the review queue for long time), right?.

Firstly thank you for your comments. End of summer 2013 after GSoC until start of 2015 the API barely changed. I did rewrite the internal engine twice and kept adding more testing, but externally all you would have seen was a lot more performance and much fewer faults (zero eventually) in the thread sanitiser.

At some point in the last year you decided on a major update (i.e. involving the "lightweight future promises" you mentioned) that alter the interface dramatically, that would require updating the tutorial and documentation.

Strictly speaking, if you did a find and replace in all your files replacing "async_io_op" with "future<>", and replacing all use of async_io_op::get() with get_handle(), your previous source code would now work with the "majorly updated" API. This is the present tutorial code examples.

Without going into too much detail about that change: Does it significantly make using AFIO easier?

Quite a few people have disliked (a) the choice of future continuations over ASIO's async_result pattern and (b) the batch API. Those two observations have come up repeatedly - Bjorn and Robert on this list have both publicly found issue there, and neither was alone in their opinion. I'm not dropping futures in favour of async_result - I don't think that helps ease of use because in file i/o you really do want strong i/o ordering, and you usually don't care about ordering much for network i/o. Forthcoming C++ 1z coroutines are also futures based, and that decision is not going to be reversed now. Futures are our future as it were. However I could do something about the performance penalty that futures have over async_result. I believe I have eliminated it (untested claim) and you can emulate async_result easily in lightweight futures by a const lvalue ref consuming continuation (which you can add infinite numbers of). That should make those preferring async_result happy. That leaves the batch API. I personally quite like it, but I can see it confuses people. So the new API presents a more traditional unsurprising API which I think Robert amongst others should prefer.

Does it significantly improve AFIO performance?

In no meaningful way, no. The cost of the i/o is many orders of magnitude higher than anything I could do in AFIO. I could stick a for loop counting to a million in there and I doubt anyone would notice. Where there was a performance problem was in the continuations. If you were scheduling non-i/o continuations, the overhead of the continuations machinery was onerous because we were vectoring through the stable ABI layer, so everything had to be type erased and reconstituted at least three times. Lightweight future-promise is I believe very close to optimally lightweight now as it all happens in the TU at the point of compilation, no ABI traversals.

...

All that said, apart from the tutorial any early observations about anything within https://boostgsoc13.github.io/boost.afio/ are welcome.

Some notes from a brief first glance at the code yesterday:

1. I might be mistaken, but are you using undocumented NT APIs for the Windows specific implementation?

Exclusively yes. As of v1.3 I avoid Win32 entirely. There were bugs when I tried to use both together, and since I dropped Win32 completely things are much better.

I was under the impression that you wanted AFIO to be used in production code; i.e. this is intended to be a more practical library than an experimental one. I'm surprised the use of undocumented APIs has not backfired yet in your testing.

The NT kernel API is exceptionally stable as any changes to it cost Microsoft and anyone who writes device drivers dearly. Before I use any NT kernel API I examine when it entered NT and if it has ever changed in any release since, with the Windows XP kernel being my minimum supported kernel. To my knowledge, apart from a few small easily removed places AFIO should even work perfectly on NT 4.0 - I have been very conservative in my choices of what kernel APIs to use. The only backfire found to date is the asynchronous directory enumeration via the WOW64 layer where Microsoft has a bug in their WOW64 syscall parameter repacking code, so this only affects x86 binaries on a x64 kernel. I had a buddy in Microsoft ask around about it, and it turns out that precisely nowhere in any Microsoft code is anyone ever doing an asynchronous directory enumeration, and hence that bug (which is confirmed) was never noticed till now. They were actually quite surprised that asynchronous directory enumeration works. It's a wontfix unfortunately, but it's very easy for me to workaround because the segfault is caused by ASIO expecting a pointer to come back from IOCP and it's randomly getting a truncated invalid pointer it tries to dereference. If I don't hand off the async to ASIO I work around the problem.

2. Examples are a little alarming: If an example in the documentation contain #ifdef WIN32 or #ifdef __FreeBSD__, it makes someone wonder how portable AFIO really is. Your examples should not make using the library look complicated by being longer than they need to be: If they contain #if 0 blocks, they are just that much harder to read.

This is a very thought provoking observation which has changed what I'm going to do about the tutorial. Thank you. Unlike some other C++ libraries, this is a platform abstraction library rather than a C++ abstraction library the same way ASIO is. Where it is possible without too much performance loss, I hide platform specific quirks as ASIO does. However, where those quirks are unavoidable, it's really best to document and push the problem onto the end user (also same as ASIO). For example, NTFS has lazy metadata flushing across handles, so some of the code examples you'll see in the docs quite literally sleep for five seconds to let the NTFS lazy flusher synchronise metadata across multiple file handles. Another example is that FreeBSD cannot track file renames, only directory renames, so you may see me ifdef in an otherwise unnecessary shim directory to make the code example work properly. Why not refactor the code example to remove file tracking? Because the FreeBSD kernel folk are still actively considering fixing BSD, so for all I know it could get fixed next BSD release. I personally think that if you are a person who needs something as niche as asynchronous file i/o, then you are *very* interested in what platform specific quirks you'll need to know about if you're writing high performance filing system code. Hence I chose to leave in ifdef quirks workarounds in the tutorial and code examples, and for those type of people with that use case those are very valuable to know about. The same rationale applies to choice of filesystem traversal algorithm, I think that's why I left in #if 0 alternatives for people to experiment with themselves (I know they do, I've had email from people who switch on the other ifdef branches and then email me to ask why I think they are so much slower, I tell them my best speculations, but in the end who really knows?). All that said, it makes the tutorial much less of a tutorial and much more of a cheatsheet on filing system quirks. I am kinda assuming the reader is only here because they absolutely need async file i/o, and are therefore highly skilled in that topic much as someone approaching uBLAS probably has a maths degree. What your comment has made me realise is that it would make much more sense if there were a "normal persons tutorial" and an "advanced users tutorial" where the former is a nice hand holding all-portable baby steps thing, and the latter is stuff like writing distributed mutual exclusion algorithms solely via atomic append onto the filesystem like is in the current tutorial. How does that plan sound? Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 21 Jul 2015 at 21:51, Glen Fernandes wrote:

...

The NT kernel API is exceptionally stable as any changes to it cost Microsoft and anyone who writes device drivers dearly.

Even when I worked at Microsoft (not that long ago; I left in 2014), in the teams/orgs I was in, we wouldn't even consider using any undocumented NT APIs in our code: What you're doing is brave. I'm suprised no AFIO users (if people are already using AFIO) have quoted an old post like http://blogs.msdn.com/b/oldnewthing/archive/2005/07/01/434648.aspx after seeing you use those APIs.

...

What your comment has made me realise is that it would make much more sense if there were a "normal persons tutorial" and an "advanced users tutorial" where the former is a nice hand holding all-portable baby steps thing, and the latter is stuff like writing distributed mutual exclusion algorithms solely via atomic append onto the filesystem

...

is in the current tutorial. How does that plan sound?

It's your library, and you know your target audience. I would only suggest you consider that for any given example that you have decorated with platform-specific code / conditional code to illustrate some point or the other, just make sure it passes the test that any real potential customer of AFIO is going to issue: "Could I implement what this example is trying to achieve using much less [perhaps platform-specific] code, and achieve the same level of

Sorry, I didn't see you originally said "undocumented APIs". It's like 3am here :) AFIO doesn't use any undocumented APIs nor behaviours. All the kernel APIs it uses are from the official Microsoft kernel documentation e.g. here is the docs for NtCreateFile https://msdn.microsoft.com/en-us/library/windows/hardware/ff566424(v=v s.85 ). aspx. The link you quoted is where someone was using truly undocumented structure layouts which suddenly changed. That isn't a problem with the official and fully documented NT kernel API. If I said undocumented APIs anywhere in the AFIO docs, please do tell me where. like performance?". If

the answer is "no", then you have nothing to worry about.

Really with async file i/o it's always more a question of control rather than performance. In the naïve case performance will usually be higher by going synchronous. The last section in the current tutorial demonstrates this: your naïve algorithm which is great with a warm cache can perform extremely poorly on a cold cache hard drive. AFIO lets you more easily tune your algorithm through trial and error to work well in both cold and warm cache scenarios, and moreover to do so portably. That was its original use case. Now with the race free filesystem extensions, and given there is absolutely no portable reusable library I am aware of which abstract those across platforms anywhere at all, things have changed. If you were wanting to write portable code capable of working under a changing filesystem you get no choice but AFIO right now in any language (that I am aware of). I think I should be presenting at CppCon about the race freedom stuff. Right now I'm between jobs, so I may have to cancel Seattle if my new employer requires it :( Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 21 Jul 2015 at 23:31, Glen Fernandes wrote:

...

I might be mistaken, but are you using undocumented NT APIs for the Windows specific implementation?

The reason I asked the question was because, when glancing your repository, I saw comments like: "From http://undocumented.ntinternals.net".

Ah, sorry. That site is basically a reference site for WINE and ReactOS programmers. It's essentially Microsoft's kernel docs, but with useful notes from the coal face as it were.

You wrote:

...

All the kernel APIs it uses are from the official Microsoft kernel documentation

Ah; this changes things. If you're only using documented APIs, that's not alarming.

After having slept on it, I lied to you earlier. There is exactly one place I use an undocumented behaviour and it is here at https://github.com/BoostGSoC13/boost.afio/blob/master/include/boost/af io/v2/detail/impl/afio_iocp.ipp#L370 where I use NtQueryObject with the deprecated query ObjectNameInformation. ObjectNameInformation was removed from the official docs if I remember around NT 3.5 because it hangs when fed certain types of handle (none of which AFIO would ever use), but it still works right up to Windows 10. The reason I use it is because the alternatives are all vastly slower and path() is on a performance critical path, and if it ever were to change in the future the unit tests and all APIs would all spectacularly fail so this is not something which would be missed. According to the ReactOS source code, ObjectNameInformation is used very extensively throughout Microsoft code, including Explorer and a raft of Office code. I judged it worth the risk for the gains.

The bug you've hit and the response you received, though, might be an indication that the internal APIs aren't terrific to use in your portable library either. They're certainly acceptable to use in any system software or driver that is tested thoroughly for, and ships only for, specific Windows versions with less concern about binary forward compatibility.

I am unaware of any of the NT kernel APIs used by AFIO having ever changed at all since their introduction into the kernel, most of which happened in the early 1990s. Of course just because they have never changed till now doesn't preclude their changing in the future, but it's rather similar to POSIX deciding that the open() function needs to be changed - they're going to add an open2() function instead of changing the spec of the open() function in a breaking way. Too much code hangs on the current NT kernel API that it is ever worth changing existing APIs.

I guess the test here is: How many potential AFIO users would have concerns about consuming AFIO in some cross-platform project when they find it uses internal APIs? (I'm would imagine most would refuse to use it outright if it were using any undocumented APIs). If the answer is "little to none", then you're probably fine. If not, it will probably work against you in the review.

I think this is the wrong question to ask. The right question to ask is: do I mind using a Boost library which skips the win32 layer to use the publicly documented kernel API instead in order to gain: * Much improved POSIX semantics including inodes, deleting and renaming open files, POSIX timestamping, POSIX volume mount points, no longer random fails when deleting directory trees, POSIX symlinks, POSIX stat_t, POSIX statfs_t etc. * Async everything, including metadata fetches, directory enumeration and file byte range locking * No win32 induced weirdnesses with paths (e.g. char limits, magic character conversion, weird substring expansions) * Up to 50% more performance for any operation consuming a path * 10m item directories are not a problem, same as Linux and BSD * Filesystem race freedom and explicitly documented race guarantees per API * Inode instead path item lookups In exchange for these losses: * No DOS shortname compatibility * 10m item directories hang Explorer and the command line * Generated win32 paths from NT kernel paths can confuse users as they don't "look right" If you have ever been surprised by a bug report where your filesystem related code on Windows did not behave as you expected from POSIX, I think AFIO is worth examining. BTW while we're on the topic AFIO also directly uses the syscall interface on Linux, BSD and OS X for directory enumeration and skips the C library. This allows AFIO to handle > 10m item directories with ease, plus it gives snapshot race freedom with the same guarantees as on Windows. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 22 Jul 2015 at 7:58, glenfernandes wrote:

...

Really with async file i/o it's always more a question of control rather than performance. In the naïve case performance will usually be higher by going synchronous.

I might be misunderstanding you, so I will try harder to get on the same page. Even though English is the only language I know, I know it poorly. :-)

No problem.

Why would anyone interested in reviewing AFIO care about getting more performance by going synchronous? The reason they're interested in an asynchronous file I/O library is because they need asynchronous file I/O, right?

Most people who think they need async file i/o don't need it and shouldn't use it. Conversely, most people who write code which uses a filesystem path don't realise they've just written buggy racy code which could destroy other people's data and open security holes.

Control over performance sounds great, but it's not control over performance if it comes at a cost of performance, right? [Example: I see the support of the C++ allocator model as something which can sometimes offer control over performance, but in no way does it make things any slower at runtime when the allocator supplied is std::allocator than if the code just used 'new' and 'delete'. --end example]

You've got it exactly right: you sacrifice some average performance in exchange for control over worst case performance. Same goes for the race free filesystem code - they come with a performance cost due to POSIX not providing a race free sibling file open API, so AFIO must iterate parent directory opens with inode lookups and back off and retry until success. Windows doesn't have this problem. It's been raised with the AWG, and there was some sympathy for the omission.

I thought motivation to use your library would be one or more of: - Simplicity (makes it easier to write maintainable file I/O code) - Portability (saves me time from writing platform specific code) - Performance (it is faster than code I would write by hand)

I would say all three of these yes.

On simplicity: If someone does not care about portability, can they write smaller/cleaner/more-maintainable code if they choose to use AFIO versus using overlapped I/O with IOCPs or KAIO? Does it sacrifice any simplicity for portability?

If you didn't care about portability, if you write your code in WinRT then all your i/o is async and that is probably the nicest way of writing 100% async i/o code using a mainstream language that I am aware of. Once C++ 1z coroutines are in there, I believe AFIO will let you come close to that clarity and simplicity of coding async as on WinRT, except it's portable. My long term goal here is that C++ becomes like Erlang and your apparently synchronous C++ code magically coroutinises at any point it could block because under the bonnet it's using ASIO for networking and AFIO for file and filesystem, so whenever your legacy C++ codebase "blocks" it is actually off executing other stuff and it'll correctly resume when the i/o completes. That's a long way away though.

On performance: Is it faster or at least no slower than any other libraries? (e.g. libuv) Does it sacrifice any performance for portability?

I haven't compared it to libuv, but libuv does nothing about filesystem races and ought therefore to be quicker. That said, Rust started out with libuv as its i/o layer and they ended up recently dropping it due to poor i/o performance which is why Rust's i/o library is so immature relative to its other standard libraries. Note you can ask AFIO to disable the race freedom code for a specific case, and it does.

On portability: Does it entirely abstract away any platform specific issues?

As much as it can.

(e.g. Do you believe a user of AFIO will be required to write platform-specific code as in your examples?)

I think for any serious use of the filesystem some platform specific code is inevitable. For example, concurrent atomic appends to a file are extremely quick on NTFS and ext4 but exceptionally slow on ZFS. Conversely, fsyncing or especially O_DIRECT on ZFS is lightening quick compared to NTFS or ext4. I can't abstract out those sorts of differences because I can't know if they're important to the end user or not. In a future AFIO I'll provide a high level abstracted API for locking ranges in many files at once, and you won't need to care how that is implemented under the bonnet where it'll use very different solutions depending on your situation. If you look at https://boostgsoc13.github.io/boost.afio/doc/html/afio/quickstart/atom ic_logging.html you'll see lots of benchmarks for various multi-file locking solutions on many platforms, this was me testing the waters for a future high level solution. I'm not against similar high level abstractions like that in the future, but I suspect I won't be writing any I won't be using myself personally. This stuff is very hard to write, much harder than acquire-release atomics for memory race freedom which I used to once think were hard and tricky. They aren't relative to filesystem based algorithms.

...

Now with the race free filesystem extensions [...] things have changed. If you were wanting to write portable code capable of working under a changing filesystem you get no choice but AFIO right now in any language (that I am aware of).

Does the documentation show (with examples) how AFIO helps here?

A good point. It's why I submitted that topic to CppCon. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 22 Jul 2015 at 11:12, TONGARI J wrote:

...

Quite a few people have disliked (a) the choice of future continuations over ASIO's async_result pattern and (b) the batch API. Those two observations have come up repeatedly - Bjorn and Robert on this list have both publicly found issue there, and neither was alone in their opinion.

I'm not dropping futures in favour of async_result - I don't think that helps ease of use because in file i/o you really do want strong i/o ordering, and you usually don't care about ordering much for network i/o. Forthcoming C++ 1z coroutines are also futures based, and that decision is not going to be reversed now. Futures are our future as it were.

async_result is more flexible and it allows the use of future, e.g. `s.async_xxx(..., asio::use_future)`, and you can customize your afio::use_future to use your lightweight futures. What's your real concern of async_result? The verbosity of specifying afio::use_future? Since you mentioned C++ 1z coroutines, the asio way actually allows zero-overhead adaption without using a future, e.g. see: https://github.com/jamboree/act

A good summary of why those who prefer async_result do so, thank you.

Is any performance reason in preferring strict future API to the callback style?

Historically futures were slower, and Chris said so repeatedly in N-papers to WG21. I believe lightweight futures have eliminated the performance gap. I also have an extension to the Concurrency TS in that you can add as many const lvalue ref continuations as you like to lightweight futures, not just a single rvalue ref continuation as per the TS. This effectively implements a single shot signals and slots implementation - you can install as many completion handlers as you like per future. None of this answers your question of course. I will say that file i/o is often 10x-1,000,000x slower than network i/o, and the performance of handlers vs continuations is correspondingly unimportant. Futures gives you the option of monadic programming, and they enforce strict operation ordering very succintly. They are the right choice for file i/o, just as async_result is the right choice for network i/o. Though, to be honest, I suspect lightweight futures will make async_result not as useful as previously. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 23 Jul 2015 at 5:37, Ben Pope wrote:

To clarify; I think you're saying that your lightweight futures are close enough to free that you would prefer to implement your library with them as opposed to callback hell; and so, presenting the nicer future API poses no significant further performance penalty whilst presenting a nicer syntactic model.

That's a much better way of saying it yes. The counter argument is that async_result allows third party supplied synchronisation and is therefore much more flexible. I personally think that means that our standard synchronisation primitives need to up their game instead, and I've hopefully played my part in that.

Personally I'm really interested in your lightweight futures for the same reason. Callback hell sucks; without profile information to the contrary, I'd rather just write code the nice way.

Of course you eliminate callback hell with standard futures and coroutines already. I simply replace standard futures with faster ones that do more stuff. I have yet to benchmark Gor's coroutines when combined with lightweight future promise. It's part of why I estimate four months to rewrite the AFIO engine, as I would assume I'll need to pester him with code examples of pathological performance corner cases :) I also would like optional Boost.Fiber support in lightweight future promise, that gains us coroutines on all platforms. But that's probably later again right now as the lack of ability to relocate fibers across threads is going to require some further reflection and pondering from me. Let me put this another way: I'm aiming to drop the dependency on ASIO in the medium term because the coroutine reactor in the C++ language takes over, and therefore the ASIO reactor is no longer necessary. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 22 Jul 2015 at 15:16, Michael Caisse wrote:

On 07/22/2015 09:01 AM, Niall Douglas wrote:

...

Futures gives you the option of monadic programming, and they enforce strict operation ordering very succintly. They are the right choice for file i/o, just as async_result is the right choice for network i/o.

Is there somewhere in the documentation/rational that you explain this (right choice) comparison further?

There once was, but I deleted it as it generated emails to me telling me why I was wrong :) I don't mind restoring such a section now that I have lightweight futures which do I think address most of the problems that the async_result camp have with futures. Do you think I should incorporate this rationale into the design rationale, the tutorial, or the FAQ? After all, the choice of futures is strictly speaking not the controversial choice. They are the STL choice, and if anything it should be controversial to not be choosing the STL choice rather than the other way round. I also feel a bit wary of getting too deeply into futures because I know the monadic programming camp (let's call them the Bartoz camp) aren't going to be happy with my interpretation of "monad". But if you and the community feel I need to justify the choice of futures as the async operation reference mechanism, I can do that. In any case I will need to discuss the reason I deliberately allow implicit type slicing between afio::future<T> and afio::future<void>, I already can see gnashing of teeth is going to happen from that design decision, and that could be part of a wider discussion. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 22 Jul 2015 at 22:13, Michael Caisse wrote:

...

I don't mind restoring such a section now that I have lightweight futures which do I think address most of the problems that the async_result camp have with futures. Do you think I should incorporate this rationale into the design rationale, the tutorial, or the FAQ?

I personally do not think you need to restore the section. Some people find the async_result clumsy. Some people think it is elegant. I personally like the async_result interface and find it far more flexible; however, I can work with a future interface just fine.

You have mentioned a couple times that futures is the right choice for file I/O and async_result is the right choice for network I/O. I haven't really thought too much about it but I have some ideas on why you might state that. I know you have thought about this problem domain a lot and was interested in how/why you came to that conclusion.

Oh okay. I can try my best to explain so. I assume everyone reading understands futures. For those not aware, ASIO's async_result works by you specialising the async_result trait with your custom completion handler types, and then when you do something like: auto ret=async_write(stream, buffers, handler); ... ret will have whatever type returned by your specialisation of async_result said. ASIO will asynchronously go off and write each of the buffers specified to stream, calling handler once the write has completed (with either success or failure). This pattern works well for socket i/o because your network code invariably does one of two patterns: 1. I am sending messages, so long as each message send is treated atomically I don't care about the send order (i.e. sendmsg()). 2. I am sending a continuous stream, please send as much as you can now and tell me how much you were able to send in the completion handler. That yields a nice simple callback loop, something like: void handler(error_code &ec, size_t bytes) { if((remaining-=bytes)) async_write(socket, advance_buffers(buffers, bytes), handler); } async_write(socket, buffers, handler); In other words, when you finish sending the last thing, send this next thing. File i/o is rather different from socket i/o because if a socket is like a serial port guarded by a mutex, files and filesystem are like random access memory with the same sequential consistency problems: 1. Unlike socket i/o, with file and filesystem i/o usually you are sharing access with other threads and processes i.e. ordering of visibility of changes to others is important. 2. I/o is never partial with file i/o, either you achieve all of a read/write or you don't. This both a blessing and a curse - blessing in that all of a write becomes atomically visible to others or not (unless on a network filing system!), curse in that you must be careful not to issue single 4Gb read/writes for example. 3. File i/o and especially the filesystem is therefore riddled with race conditions and requires special handling just the same way as race free use of memory requires mutexes and atomics etc. i.e. once again, *ordering* of visibility of i/o is vitally important. 4. Also similar to RAM, recent filing systems allocate and deallocate storage using an allocator whose semantics (pathologies!) should be considered, else you'll end up with highly undesirable outcomes, same as with RAM. What this means in practice is that you don't get nice simple single callback loops as with socket i/o. Imagine the find in files regex algorithm: 1. Schedule enumerating directory. 2a. For each directory in enumeration, recurse into 1. 2b. For each file in enumeration, schedule opening file. 3. For each opened file, schedule reading first two 1Mb chunks. 4. For each 1Mb chunk read, stitch onto previous 1Mb chunk and do a regex match. Schedule reading next 1Mb chunk. That's at least four completion handlers, each of which may trigger the others. The above is also highly simplified - a real world solution chokes the parallelism on enumerations are those are particularly expensive when over-parallelised, and that involves even more callbacks. The above is also a 100% read only algorithm. As soon as you bring writing into the picture, you must be very careful to order your writes and your reads perfectly correctly with no chance of racing. Mutual exclusion on the filesystem is particularly fraught, with the only portable method being O_EXCL and even that is known to be broken on some major OSs until very recently. So tl;dr the reason why futures are more appropriate for AFIO is because it's easier to specify ordering of operations with future continuations because they always flow in sequence from start to end. You can do it with ASIO callback handlers too, but it requires more typing, and I think is more error prone and harder for other programmers to understand. The key thing I think those who advocate async_result don't realise is you very rarely are adjusting a single file from a single thread if you are bothering with async file i/o. In the real world, almost certainly multiple threads and processes are all contending on the same shared dataset, and you need fine grained control over exactly when you do reads and writes with respect to one another. Hopefully this answer the question? Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 25 Jul 2015 at 2:32, TONGARI J wrote:

If async_result can only deal with callbacks, I'd agree with you, but as already mentioned, it's also capable of futures, just replace the callback with placeholder like afio::use_future, for example.

I don't know how afio is implemented internally, if using future as its primitive API does provide performance benefit, then I think it's fine to just expose the future API, however, if it adds more performance penalty than callback API, then I think it's conflict with the C++ philosophy - you don't pay for what you don't use.

Using the proposed `await` with the asio style callback can provide zero-overhead synchronization as I shown you before, not a single lock/atomic nor dynamic allocation is needed, while using lightweight future, you still need to pay for atomic op & dynamic allocation when chaining the continuations (correct me if I'm wrong).

On completion, lightweight futures costs exactly two atomic XCHG to signal and execute continuations (about 40 CPU cycles). It only calls deallocation, one per continuation, after all other processing is completed. The distance between ASIO calling the AFIO completion handler and future continuations being invoked is likely less than 150 CPU cycles. The above is a good example of why zero overhead completion handlers is unimportant for file and filesystem where any operation at all is going to cost > 100 microsecond. ASIO maxes out at about 0.365 microseconds per dispatch amortised. Lightweight future-promise costs about 0.051 microseconds, so that's adding a 14% overhead, and that's at least twice as lightweight as STL future-promise. If your operation costs 100 microseconds, the overhead of a lightweight future-promise is therefore 0.05%. If you were talking about merely chaining pure coroutines, or even socket i/o where *reaction* *time* to a recv() is the important factor rather than overall throughput, I'd get your point entirely. However with file and filesystem reaction time is not remotely important at all because it's overall latency and throughput which matters, not the time between when an i/o completes and you do something with it. My personal opinion is that for 99% of anyone doing async file or filesystem that the choice of lightweight future promise as the completion mechanism chosen is just not an issue. Hence the choice to use them over async_result because they are what the STL has chosen as the standard async notiifcation mechanism, and are therefore unsurprising to most C++ programmers. I also hear rumours that the Networking TS may yet enforce futures for all networking, so the choice of futures by AFIO should not be controversial. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 07/24/2015 10:47 AM, Niall Douglas wrote: <snip the explanation>

So tl;dr the reason why futures are more appropriate for AFIO is because it's easier to specify ordering of operations with future continuations because they always flow in sequence from start to end. You can do it with ASIO callback handlers too, but it requires more typing, and I think is more error prone and harder for other programmers to understand.

The key thing I think those who advocate async_result don't realise is you very rarely are adjusting a single file from a single thread if you are bothering with async file i/o. In the real world, almost certainly multiple threads and processes are all contending on the same shared dataset, and you need fine grained control over exactly when you do reads and writes with respect to one another.

Hopefully this answer the question?

Niall

Niall - Thank you for the great explanation. You were saying async_result, I repeated async_result, but I kept thinking completion handlers :-/ Fail on my part. Makes much more sense now. I would highly recommend putting this information somewhere in your documentation. In some ways it is the motivation. Future continuations are fine when the sequence from start to end doesn't branch. I am cautious of their usefulness in the general case still. It isn't uncommon to make choices in a completion handler to queue different async actions with different completion handlers. Thank you for taking the time. michael -- Michael Caisse ciere consulting ciere.com

On 07/27/2015 09:54 AM, Michael Caisse wrote:

Future continuations are fine when the sequence from start to end doesn't branch. I am cautious of their usefulness in the general case still. It isn't uncommon to make choices in a completion handler to queue different async actions with different completion handlers.

You hit the nail on the head. This is one of the main reasons why I seldomly use futures for asynchronous operations. Instead I prefer coroutines (and callbacks for simple sequences) over futures as the use of future continuations quickly becomes unwieldy in reality. async_result gives me the freedom to choose whatever mechanism is appropriate for my application. I suspect that Niall's preference for (his own) futures is due to something that has only been mentioned briefly in this discussion. AFIO operations are launched in a two-step process: 1. An operation is scheduled for execution (e.g. dispatcher::file() for opening a file.) 2. The operation is executed (e.g. dispatcher::call().) Using this two-step approach allows us to schedule batches of dependent operations before they are executed. When we batch operations together we should be able to express dependencies between them. Niall uses his own futures for that.

On 22.07.2015 03:27, Niall Douglas wrote:

On 21 Jul 2015 at 20:07, Glen Fernandes wrote:

...

I was under the impression that you wanted AFIO to be used in production code; i.e. this is intended to be a more practical library than an experimental one. I'm surprised the use of undocumented APIs has not backfired yet in your testing.

The NT kernel API is exceptionally stable as any changes to it cost Microsoft and anyone who writes device drivers dearly.

As a Windows dev and Boost user, I think the use of the NT kernel API is fine as long as it is very, very clearly stated along with the implications that has, especially for the "future-proofness" of resulting applications. For example, will a Windows AFIO binary work under Wine or similar? And if you haven't done so already, you probably should also clearly state in the docs why the Win32 API was insufficient. Cheers - Asbjørn

On 22.07.2015 19:01, Niall Douglas wrote:

On 22 Jul 2015 at 10:27, Asbjørn wrote:

...

As a Windows dev and Boost user, I think the use of the NT kernel API is fine as long as it is very, very clearly stated along with the implications that has, especially for the "future-proofness" of resulting applications.

For example, will a Windows AFIO binary work under Wine or similar?

I would be very surprised if it didn't. I made extensive use of examining Wine and ReactOS sources (at that layer they are the same code) to figure out user mode NT kernel API semantics because most of the code examples on the web use kernel IRP programming, and I needed non-IRP examples to study. Wine and ReactOS have a lovely unit test suite which does exactly what I needed - lots of non-IRP example code patterns to study.

Sounds great, it was my (poor) understanding that Wine was a layer above that (ie just Win32). Still, with temporary solutions/applications having a habbit of becoming permanent solutions[1], it still seems to me that the reliance on NT kernel API rather than just Win32 may stop it to work earlier than a pure Win32 application. Again, I don't view this as a show-stopper at all, just something a new library user needs to be keenly aware of before deciding to use AFIO. [1]: http://it.slashdot.org/comments.pl?sid=7719515&cid=50156045

...

And if you haven't done so already, you probably should also clearly state in the docs why the Win32 API was insufficient.

I think it's on the first page :)

That'll do just fine :P Cheers - Asbjørn

On 22 Jul 2015 at 13:35, Glen Fernandes wrote:

1. Use of documented kernel APIs: I'm torn here; on one hand, I don't condemn use of them and I would agree with Asbjørn that it isn't a show stopper. Yet you have a case of an actual show-stopper (crash) in AFIO because of the use of them.[1]

The show stopper crash is due to how ASIO implements its IOCP reactor which just happens to collide with the WOW64 bug. If I could modify the ASIO source code to special case this it's fixed. This is why Microsoft marked it as wonfix because even though it's a bug in their code, it is really super easy to work around if you modify your IOCP reactor. As I mentioned before, if I don't hand off the async to ASIO I make the problem go away.

2. Use of undocumented or deprecated APIs: Kernel APIs or otherwise, even if apparently innocuous, this practice is alarming to me. I would be surprised if it doesn't taint the review of AFIO.

I can replace ObjectNameInformation with a new routine which uses non-deprecated APIs if the review demands it. Indeed, the ObjectNameInformation use replaced some original code which iterated every mounted volume in the system per path lookup, which was costly. I suppose one _could_ make the assumption that open file handles can never be relocated across mounted volumes and that mounted volumes are never remounted anywhere new while file handles are open within them. With those two assumptions I can use NtQueryFileInformation instead, and reconstruct the same thing NtQueryObject returns. What worried me was the possibility that a volume could be mounted in two or more locations you see, and that the canonical path of a file could change, hence the iteration of all mounted volumes to check for that.

...

I think this is the wrong question to ask. The right question to ask is: [snip]

So the wrong question to ask is, "Is it concerning that AFIO uses these APIs given your use of them is unsupported?" and the right question to ask is "Is it concerning that AFIO use these APIs given your use of them is unsupported but the advantages are [...]?" ? :-)

There is unsupported and there is unsupported. If you want to write a user mode program which executes in the Windows boot sequence before the subsystems start up (i.e. there is no win32 API available yet), then you write that program using the NT kernel API. That's supported, because it's categorised as a DDK user mode application. By unsupported Microsoft have never said "don't use these APIs directly", they actually said "we won't promise we won't change anything inside the header file Winternl.h" and it just happens that the kernel API falls into that header file along with a ton of other stuff. I may have mentioned earlier that Cygwin and Mingw use the same APIs as AFIO, and if you review the Windows bug tracker where those projects report bugs in Windows you'll find a common pattern of the first tier support in Microsoft refusing the fix the bug as its use is "unsupported", then a more senior tier fixes the bug anyway. That's because the single biggest user of the NT kernel API is Microsoft itself, and Microsoft would internally like bugs in one of its most fundamental APIs fixed, so they take these bug reports seriously. If this isn't "supported", it looks awfully like supported. I would actually say the real situation is the Microsoft doesn't _encourage_ direct user mode use of NT kernel APIs. Historically speaking it wouldn't have worked on WinCE and Win95, and that was once a problem. I think we're a long way away from that now. BTW the Visual Studio MSVCRT makes extensive use of NT kernel calls itself, indeed Stephan pointed me at RtlGenRandom which solved a problem for me.

Niall Douglas wrote:

...

I would say all three of these yes.

Sounds like a good enough spot to put yourself out of your misery and take this opportunity to escape the review queue that you've been in for so long. I understand wanting to improve the library, but something tells me you don't want to be in the queue for another two years constantly refactoring AFIO. Just my two cents.

I would certainly prefer review sooner rather than later. It could be if AFIO had been reviewed much earlier on I wouldn't need to do so much refactoring now, but maybe it's too niche for that. I don't know, I haven't even proved the case for futures instead of async_result yet, and I already know the ASIO crowd here is going to be united on that. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 23/07/2015 13:11, Niall Douglas wrote:

On 22 Jul 2015 at 13:35, Glen Fernandes wrote:

...

1. Use of documented kernel APIs: I'm torn here; on one hand, I don't condemn use of them and I would agree with Asbjørn that it isn't a show stopper. Yet you have a case of an actual show-stopper (crash) in AFIO because of the use of them.[1]

The show stopper crash is due to how ASIO implements its IOCP reactor which just happens to collide with the WOW64 bug. If I could modify the ASIO source code to special case this it's fixed. This is why Microsoft marked it as wonfix because even though it's a bug in their code, it is really super easy to work around if you modify your IOCP reactor.

Out of curiosity (and because I have my own non-ASIO IOCP reactor in a WOW64 process and I'd like to know if it's potentially similarly vulnerable), is there more detail on the bug you encountered or what triggers it?

Hi Niall, Did the review commence Friday? (Did the announcement mails only get sent to the -announce list?) Glen -- View this message in context: http://boost.2283326.n4.nabble.com/afio-AFIO-review-postponed-till-Monday-tp... Sent from the Boost - Dev mailing list archive at Nabble.com.

...

Did the review commence Friday? (Did the announcement mails only get sent to the -announce list?)

As far as I am concerned it did :). However until Ahmed announces it I suppose it has not.

If you want to get started, everything is ready at the usual places and the CI dashboard is all green on all supported platforms and targets (the one fail was a timeout due to me pushing the CI too hard, it's not important).

Today I'll be spending swapping my dev workstation hard drive as I killed the poor thing this week during testing the AFIO tutorial workshop examples. I'll then press on with finishing the final key-value store example which is described in detail in the tutorial but is currently without code nor benchmarks. That new code won't be part of this review, but I can link to it here with benchmarks when it's ready.

While I don't really care about your poor hard drive (what has this to do with the review?, why do I need to know this?) I would like to express my deep concern that this review apparently starts before the library has been finalized. I admit that I have not even looked yet - neither at the code nor the documentation - and the review manager most likely thinks it's ready, however the whole 'it will start next Monday' thing (or is it 'Friday' now?) and the constant postponement because of things which have to be added/finished/tested/etc. tell me that this library is not ready for prime time yet and it most likely will not be very useful for Boost users at this point as it will keep changing massively... For this reason, I would like to suggest to wait until the dust has settled and the library has gained at least a handful of users outside of Boost who could report on its usability, and then reschedule the review. Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

On 22 Aug 2015 at 15:21, Glen Fernandes wrote:

On Sat, Aug 22, 2015 at 9:21 AM, Niall Douglas wrote:

...

If you want to get started, everything is ready at the usual places and the CI dashboard is all green on all supported platforms and targets (the one fail was a timeout due to me pushing the CI too hard, it's not important).

In my opinion- I don't think you will need to worry about the tests being all green for a review.

I'm a perfectionist. I wouldn't have considered it ready for review until it was all green across the board on all 60 of its per-commit tested targets. I spent four hours on Friday coaxing wandbox to compile AFIO again so you'd have a working C++ browser playpen you can go play with AFIO in. Little details are important.

...

Today I'll be spending swapping my dev workstation hard drive as I killed the poor thing this week during testing the AFIO tutorial workshop examples. I'll then press on with finishing the final key-value store example which is described in detail in the tutorial but is currently without code nor benchmarks. That new code won't be part of this review, but I can link to it here with benchmarks when it's ready.

One question: Are reviewers also reviewing the "Monad" and "APIBind" libraries? (I see them consumed as sub-modules in the AFIO repository).

Well I asked here about that a few months ago, and I was told that there is precedent for this situation. Apparently historically you review the library being presented for review only, mentioning only the internal sublibraries if there is something catastrophically worrying about them. One then expects the internal sublibraries to be spun out into additional Boost libraries presented for review later if that is appropriate. I can tell you that I personally would not be comfortable sending AFIO into the main Boost distribution until after Monad has been reviewed here, so even if there is a universal unconditional acceptance of AFIO by everyone here, I will personally guarantee I won't send in a final AFIO until after Monad is reviewed here. I think this only fair. Do note I walk you through monad<T> in the tutorial with everything you need to know. And if you find bugs in Monad not already in the Monad issue tracker, please do file them. I know of at least half a dozen myself. Does this sound reasonable? Please do feel free to ask for any clarifications. I have sprinked cautions and notes around the AFIO documentation to explain how the presented AFIO is expected to differ from final AFIO after Monad is reviewed, and indeed there is also a dedicated section listing the expected differences in the release notes. I would *emphasise* very little is expected to change in the public API presented here, indeed I expect all the unit tests and tutorial examples to compile as-is. You or anyone else can base your review on the library as presented. The only real changes in final AFIO are the simplification of some tutorial examples thanks to C++ 1z coroutines, and better performance. The library being presented today is very well tested, and should *not* *lose* *you* *data* which is the most important quality in an i/o library. Even if performance - as you will see in the tutorial's benchmarks - is a bit poor in the current engine in some use cases. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Hartmut Kaiser wrote:

Niall Douglas wrote:

...

Please do feel free to ask for any clarifications. I have sprinked cautions and notes around the AFIO documentation to explain how the presented AFIO is expected to differ from final AFIO after Monad is reviewed, and indeed there is also a dedicated section listing the expected differences in the release notes.

How can you submit a library for review for which you already _know_ it will change?

Hartmut makes a good point. Niall, please correct me if my understanding is lacking here: - You are presenting this library with interface version A - You plan to implement interface version B while also providing A for backward compatibility - Once B is implemented, all documentation will only ever mention B because it is superior - The benefits of B make it superior in every way to A Sounds like A pales in comparison to B. So what does B require? More investment in AFIO? C++17 to be ratified? If the latter, then I don't think anybody cares about B in the context of this review. What may or may not happen in C++1y will not influence acceptance of a library today. If the former, then you are better off waiting to submit AFIO for review when you complete that new interface. Especially if it makes a difference to reviewers: i.e. you can't expect people to accept AFIO on the promise of a better API later. Right?

...

The library being presented today is very well tested, and should *not* *lose* *you* *data* which is the most important quality in an i/o library. Even if performance - as you will see in the tutorial's benchmarks - is a bit poor in the current engine in some use cases.

I'm interested in an asynchronous file I/O library being part of Boost for performance (portably). As long as it is no less reliable than using the native filesystem APIs, reliability isn't a concern or a big selling point for me. Am I to understand that by design, AFIO's implementation implies lower performance than an alternative? (even if that alternative is something like libuv, which has its own flaws) Glen On Sat, Aug 22, 2015 at 5:39 PM, Hartmut Kaiser wrote:

...

...

On Sat, Aug 22, 2015 at 9:21 AM, Niall Douglas wrote:

...

If you want to get started, everything is ready at the usual places and the CI dashboard is all green on all supported platforms and targets (the one fail was a timeout due to me pushing the CI too hard, it's not important).

In my opinion- I don't think you will need to worry about the tests being all green for a review.

I'm a perfectionist. I wouldn't have considered it ready for review until it was all green across the board on all 60 of its per-commit tested targets. I spent four hours on Friday coaxing wandbox to compile AFIO again so you'd have a working C++ browser playpen you can go play with AFIO in. Little details are important.

...

...

Today I'll be spending swapping my dev workstation hard drive as I killed the poor thing this week during testing the AFIO tutorial workshop examples. I'll then press on with finishing the final key-value store example which is described in detail in the tutorial but is currently without code nor benchmarks. That new code won't be part of this review, but I can link to it here with benchmarks when it's ready.

One question: Are reviewers also reviewing the "Monad" and "APIBind" libraries? (I see them consumed as sub-modules in the AFIO repository).

Well I asked here about that a few months ago, and I was told that there is precedent for this situation. Apparently historically you review the library being presented for review only, mentioning only the internal sublibraries if there is something catastrophically worrying about them. One then expects the internal sublibraries to be spun out into additional Boost libraries presented for review later if that is appropriate.

I can tell you that I personally would not be comfortable sending AFIO into the main Boost distribution until after Monad has been reviewed here, so even if there is a universal unconditional acceptance of AFIO by everyone here, I will personally guarantee I won't send in a final AFIO until after Monad is reviewed here.

I think this only fair. Do note I walk you through monad<T> in the tutorial with everything you need to know. And if you find bugs in Monad not already in the Monad issue tracker, please do file them. I know of at least half a dozen myself.

This underlines what I said before. It appears, you submitted a library for review which depends on another (unrelated) library which is not ready yet. How can AFIO be ready for review if Monad is not? Didn't you say you're a perfectionist?

...

Does this sound reasonable?

Not at all. Not to me at least.

...

Please do feel free to ask for any clarifications. I have sprinked cautions and notes around the AFIO documentation to explain how the presented AFIO is expected to differ from final AFIO after Monad is reviewed, and indeed there is also a dedicated section listing the expected differences in the release notes.

How can you submit a library for review for which you already _know_ it will change?

What will happen if AFIO gets accepted now (god forbids!) and Monad later will not be accepted? Will you turn Monad into an implementation detail, in this case? Is that what the community wants? Or is Monad an implementation detail today?

...

I would *emphasise* very little is expected to change in the public API presented here, indeed I expect all the unit tests and tutorial examples to compile as-is. You or anyone else can base your review on the library as presented. The only real changes in final AFIO are the simplification of some tutorial examples thanks to C++ 1z coroutines, and better performance.

But Monad and APIBind are part of the API, aren't they?

...

The library being presented today is very well tested, and should *not* *lose* *you* *data* which is the most important quality in an i/o library. Even if performance - as you will see in the tutorial's benchmarks - is a bit poor in the current engine in some use cases.

My baffling increases with every word you're saying! All of this sounds like from the old quote 'If you can't dazzle them with your brilliance - baffle them with your bullshit!'

Wasn't your whole discussion over months praising your super-duper future design only about exceptional performance? Wasn't I asking for performance benchmarks all along and you told me to hold my breath as I would see it shining once you're done developing? And now you're telling us that your performance is sub-par!

Would you have the decency to stop conveying/implying things which are half-truths, at least on this list, please?

Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Sat, Aug 22, 2015 at 9:38 PM, Niall Douglas wrote:

On 22 Aug 2015 at 19:43, Glen Fernandes wrote:

...

Am I to understand that by design, AFIO's implementation implies lower performance than an alternative? (even if that alternative is something like libuv, which has its own flaws)

Current performance benchmarks can be found throughout the documentation, specifically the tutorials, the release notes and the reference documentation.

Is it lower performance than alternatives? Almost certainly, and the documentation takes pains to present AFIO in a worst possible light in any comparative benchmarks presented so nobody is under any illusions. What matters is correctness, reliability, reusability and portability - and most importantly of all - do NOT lose other people's data.

Sorry, but I have to disagree tremendously. Performance is the only justification I can give clients (and bosses) of having such complex code as asynchronous programming requires. Justifying Boost.ASIO is an exercise in marketing, and the benchmarks are the killer chart that sell complexity. If not by that, Qt would be the chosen path in many non-academic projects. I even see a use case for AFIO in one of my products, where extreme write performance with minimum IO is a first priority, and a big effort was made to implement a subsystem for fast writes under the specific environment. Effort so big that the project froze until the writer code was done. The point of this is that performance should not be low priority. Specially for a boost library, and even more so for an asynchronous library. Citing correctness is not a feature to me. It's just basic principle. You are assumed to have it. As soon as AFIO is really competitive for me to use, I wish to do a full review. As for the general API usage I'm not sure that a full blown review is needed for that. The library is not finished, and you said, and a review now will be just like our past review, where a very interesting library is just not ready yet. And in your case, it doesn't yet perform. Reviewing of incomplete library work is not ideal, IMHO. That being said, I have some questions: Benchmarks, Do you have usable coroutines examples now? The web sample uses futures. Do you have better performance numbers when using them? What good measures did you employ to prevent caches from contaminating benchmarks? Do you believe that performance will improve? Do you know what your bottleneck is? Why do other libraries do better? Can you imitate that while still leaving your superior API? About coroutines, Do you really need C++1z? Why not Boost.Coroutine emulation for backwards support? Why do you use C++11 at all? C++03 is still widespread and a requirement for most projects still. Thank you for your time, Rodrigo Madera

On Sat, Aug 22, 2015, Niall Douglas wrote:

v1.3 AFIO is the old, very well tested, mature engine with the original API from GSoC 2013. It is reliable, won't lose you data, exposes the POSIX race freedom guarantees, and is a good solid base for any file system application. It is not particularly friendly to program with, or at least that's what everyone has told me.

v1.4 AFIO is the library being presented here. It has a new, much more friendly API which should address all the usability concerns people have expressed over the past two years.

The *internal* *implementation* of the v1.4 AFIO you are being presented with is currently implemented internally using the v1.3 engine. That makes it just as reliable, just as well tested, just as solid a base for a database application as v1.3 AFIO is, except it is now easier to program against. A later *internal* implementation will be completely new, but that isn't important to this review because little external changes. Code written now still works later, just faster.

I see where I was not on the same page. You meant AFIO's implementation is going to change (get faster) later, but AFIO's interface is complete now. I think you're better off not mentioning the different versions, and possibly even future implementation details; it is only going to confuse us. Unless you intend for the latter to impact the review (i.e. for anyone who finds AFIO not fast enough now, vote differently based on the promise of a faster engine later) which you've already stated in your e-mail above that you do not wish to do.

Again, instead of jumping to conclusions based on ranting by a person with a well known personal vendetta against me, I'd suggest go read the documentation.

Will do. I was at Table 1.4 which prompted my recent questions about performance. Glen

On Sun, Aug 23, 2015, Niall Douglas wrote:

I couldn't find a Table 1.4.

Did you mean Table 1.5, the one comparing NoSQL databases with the naïve iostreams and AFIO key-value store and where AFIO comes off looking particularly badly?

"Table 1.4. This solution will perform reasonably well under these conditions" http://boostgsoc13.github.io/boost.afio/doc/html/afio/quickstart/workshop/na... That "Maximum performance isn't important to you" was crossed implying that this solution satisfied the desire for performance. Unless you meant "is" instead of "isn't", but then the benchmarks don't reflect that. Glen

Niall,

I would have just loved to do as you say, but I was trapped. I knew that those people who have openly declared war on everything I do would do their absolute best to FUD this review as a way of point scoring and settling grudges. They have been quite literally waiting for this moment with anticipation of "taking me down a peg", and you can see that happening right now.

So I decided that absolute transparency, even if that hurt my chances, was the best policy. I think I have better chances this route than the alternative which would allow the FUD to succeed more easily because it could then be implied that I was hiding something or lying to people.

Clever move! You not only try to push an apparently not-quite-ready library into Boost, but now you're using the criticism voiced against this attempt as an argument in favor of doing it. Please note that I made every effort for my criticism voiced here to be in relation to the code, not the person. I'm not sure why you take all of this as a personal 'vendetta' as you put it. I have no personal matter to resolve with you Niall, I have not even met you personally. All I'm concerned with is the library you're presenting. Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

...

...

Again, instead of jumping to conclusions based on ranting by a person with a well known personal vendetta against me, I'd suggest go read the documentation.

Will do. I was at Table 1.4 which prompted my recent questions about performance.

I couldn't find a Table 1.4.

Did you mean Table 1.5, the one comparing NoSQL databases with the naïve iostreams and AFIO key-value store and where AFIO comes off looking particularly badly?

Niall

-- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Robert,

...

...

I can tell you that I personally would not be comfortable sending AFIO into the main Boost distribution until after Monad has been reviewed here, so even if there is a universal unconditional acceptance of AFIO by everyone here, I will personally guarantee I won't send in a final AFIO until after Monad is reviewed here.

To put things in perspective, I might relate my experience along these lines with the serialization library.

The serialization library is pretty large and complicated. Along the way I needed a few items which weren't already in boost such as Singleton, (still not in boost), a generalization of io_state_saver, composible iterators to implement filtering (I called these "dataflow" iterators, strong_typedef (now winding it's way through standard library process as "opaque typedef", and others. I made versions of the components and documented as if they were separate libraries and put them in the Boost namespace. Documentation of these components can be found within the documentation of the serialization library under the section titled "Other Classes". My intention was to get the serialization library working and accepted. Then at my leisure, I could get them "promoted" into separate boost libraries. That never happened as once the serialization library got accepted, I didn't have the energy to promote them as separate libraries - and it didn't matter much anyway as people do use them as if the were separate libraries. In only a couple of cases have official boost libraries been added which replace their functionality.

There was one bit of fallout from all this. I caught hell for putting them in the boost namespace. So I moved then into the boost::serialization namespace and that was pretty much the end of it.

In summary, these ancillary components were treated as implementation detail by reviewers of the serialization library but were conceived, packaged and documented as separate libraries. So they remain today.

So I don't see a huge problem that part of the implementation of AFIO is packaged as an orthogonal component. It only has to be reviewed in so far as the implementation of AFIO is reviewed.

Having said that, I did follow he discussion of Monad and was not convinced it is suitable as a real library. No need to go in details here.

I saw a little bit of the discussion and attended Niall's presentation on API Bind. As far as I can tell, this addresses the issue of library API versioning. This is a subject we've never explicitly addressed and is much bigger than one library. To make sense it really has to be considered as more than just an implementation detail of the AFIO library. But for now, the only way forward is to consider it as only an implementation detail of the AFIO library which offers facilities for library version control that other libraries don't offer. I think it was a mistake for Niall to include this - it's going to be tough enough to get AFIO over the bar without carrying any extra conceptual baggage. A typical programmer problem - scope creep - I plead guilty.

Your insights are very much appreciated and I fully agree with your assessment. We have had similar situations with Spirit. This however is not my point. I have no problems with sub-libraries being created/used as part of the implementation of a Boost library. My concern is related to the fact that AFIO is apparently not ready for review: it is under heavy development still, constant changes and postponements show that; even the author says that he knows of 'at least half a dozen' issues he needs to fix; the library is known to have inadequate performance (the author's own words!); not even the public API has settled (the author mentioned it may change if some other libraries are accepted); etc. All of this makes the impression of rushing a not-quite-ready (possibly very useful) library into Boost just for the sake of it. Why do we even review a library which is not ready yet? Isn't that a huge waste of everybody's valuable family time? Could I get some answers from the review manager, please? Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

On 8/29/15 9:33 AM, Ahmed Charles wrote:

On 8/23/2015 6:54 AM, Hartmut Kaiser wrote:

...

Could I get some answers from the review manager, please?

I'm unsure, exactly what questions do you want answered?

The specific question is at the end of Hartmut's post "Why do we even review a library which is not ready yet? Isn't that a huge waste of everybody's valuable family time? Could I get some answers from the review manager, please? " So I guess he's questioning your judgment about agreeing to be review manager. I don't think you have to answer this - you made your decision and we're all going to make it work. I'm sure you didn't expect that this would turn into such a large amount of work to do right. Maybe we should make a minor update on some page or another which advises potential review managers to make a pre-check of the library to get an idea as to what they are really signing up for. Other than this, I think the process is working pretty much as it was intended to.

On 8/29/2015 11:27 AM, Agustín K-ballo Bergé wrote:

On 8/29/2015 3:13 PM, Robert Ramey wrote:

...

Maybe we should make a minor update on some page or another which advises potential review managers to make a pre-check of the library to get an idea as to what they are really signing up for.

This would be "Boost Formal Review Process, Notes for Review Managers":

http://www.boost.org/community/reviews.html#Review_Manager

Other than not having the time to read every post during the week, I believe the review is going fine (from a process perspective). I'm not the sort to want to moderate what people say anyways. And a review being contentious is not really something I can/should pre-check for.

On 22 Aug 2015 at 23:18, Robert Ramey wrote:

To put things in perspective, I might relate my experience along these lines with the serialization library.

Firstly, thanks for the story. I found it very supportive.

Having said that, I did follow he discussion of Monad and was not convinced it is suitable as a real library. No need to go in details here.

I will say I find monad<T> an enormous time saver. I find myself writing most of my functions since Monad as noexcept, and letting monad<T> convert exception throws into returned monads. That reduces the time to implement exception safety hugely by constraining the paths of execution flow inversion, and has led to a substantial speed up in my development productivity.

I saw a little bit of the discussion and attended Niall's presentation on API Bind. As far as I can tell, this addresses the issue of library API versioning. This is a subject we've never explicitly addressed and is much bigger than one library. To make sense it really has to be considered as more than just an implementation detail of the AFIO library. But for now, the only way forward is to consider it as only an implementation detail of the AFIO library which offers facilities for library version control that other libraries don't offer. I think it was a mistake for Niall to include this - it's going to be tough enough to get AFIO over the bar without carrying any extra conceptual baggage. A typical programmer problem - scope creep - I plead guilty.

I unfortunately suspect you are right. I implemented APIBind to solve a local problem I had and tried proposing it as a bigger solution for wider ills, but few if nobody here was interested. But that's okay. It still lets me switch between STL implementations and API versions very easily, and I make heavy use of that facility in my automated testing. It's a big win for me here locally. I expect to be using APIBind in any new C++ library I write into the future. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 23 Aug 2015 at 9:55, Robert Ramey wrote:

Reading the other posts I see trouble ahead. There is a strong current that suggests that the library is not ready to be reviewed.

Well, on that point: 1. AFIO has been in the review queue since 2013. 2. I am fairly confident you can write reliable, performant file system applications with AFIO right now. It certainly has seen plenty of testing. 3. All the "heavy development" people refer to are, for the most part, me going off and implementing feedback from people such as yourself by wrapping the essentially unchanged core engine with "ease of use" convenience APIs. I think you'll agree it looks a very different library now than when you first reviewed it last year right Robert? That's me responding to your feedback that you thought the old API was unintuitive. However the internal engine last saw much change a good six months ago, and that was to add race free filesystem which wasn't anything fundamental. People are blowing way out of proportion how much serious change has been made to AFIO recently. The core engine has been stable since 2013/2014. How much more stable does a Boost library need to be before being considered production ready? 4. I think there is a BIG difference between "is it ready to be reviewed?" and "is it ready to enter Boost?". I think it's an emphatic yes to the first question: here is a well tested, stable Boost library with high quality documentation and high quality testing which has already undergone multiple rounds of feedback and review, most of which has been fed back into changes and reforms to the user facing library API. In the end, you have got to ask "when is enough enough?" before it's review ready? Is a well designed API, high quality documentation, high quality testing and a core implementation stable for over six months no longer sufficient for a library to enter Boost? All the new APIs are thin wrappers over the old APIs, all of which are mostly unchanged since 2013. The core implementation is essentially the same as end of GSoC 2013. And as I mentioned, I only added the new APIs wrapping the old APIs due to feedback from yourself and others. If there are severe design flaws in AFIO, then it absolutely should be rejected. But thanks to those multiple rounds of feedback, I think most of the rusty nails have been hammered down by now. In which case, what showstoppers remain between AFIO as presented and acceptance? Is acceptance of AFIO into Boost going to be judged on technical reasons, or politics, or "I don't understand the point of this library?"

And there are strong concerns that you've rolled up too much in to one package.

I think this is entirely a valid concern, and I would absolutely accept acceptance conditions listing what needs to be dropped.

There is also some antipathy regarding they way you've gone about this and used the list so there is less inclination than there might otherwise be to cut you some slack. All this raises the bar to acceptance higher than it might otherwise be - which is already quite high.

As soon as I challenged the authority of certain people with an alternative vision I was going to be heckled and hasseled every step of the way onwards, both publicly and behind my back. That much became very evident very early on, and for the record I wish it weren't so, but you play the hand you're dealt and in every case it has been someone who attacked me first even if that wasn't publicly known, and I responded because nobody else was going to and unlike other internet communities, there is not a strong central leadership here so you are on your own to fight Darwinian style on every little thing, so you get all this conflict and bitterness and unpleasantness resulting. Which is a shame: it's wasted energy better spent on writing code. But I also don't like being bullied, I *particularly* don't like other people being bullied, and there was a lot of bullying here back in 2012-2013 (much of which I think was unintentional and just a product of then Boost culture). That happens far less frequently now, and I would like to hope some of that less adversarial more supportive culture is due to my efforts and responses, even if it made me enemies along the way. I prefer a quiet harmonious life. My life away from dealing with Boost politics is extremely quiet and harmonious, I think I remember telling you some of how boring I am in the Aspen bar at C++ Now.

I propose that:

a) this review be postponed.

I'll abide by whatever the review manager decides. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 8/23/15 12:05 PM, Glen Fernandes wrote:

On Sun, Aug 23, 2015, Niall Douglas wrote: I feel compelled to say: I started contributing to Boost in 2012 when I was still at Microsoft and I observed none of the above: No conflict, no bitterness, and no unpleasantness.

Well, it hasn't always been this way. Generally things here have been more civilized than on other projects/lists. The boost discussion guidlines have been a real help here. But, occasionally we've gotten into nasty personal disputes. Not too often but it has happened. Robert Ramey

Niall Douglas wrote:

Glen Fernandes wrote:

...

Isn't "I don't understand the point of this library" a valid reason for rejection?

In itself, no I don't think so. There are at least ten libraries in Boost I have no understanding of the point of

If almost nobody the understands point of a proposed library to the extent that almost nobody recommend its acceptance: I would think that it isn't unjustly rejected. I believe niche use case libraries have a place in Boost. I suspect, though, that an asynchronous file I/O library falls into the category of something that most people want in Boost. I haven't figured out entirely where the disconnect is. It seems like you're saying "This is the async file I/O library that you need; not the async file I/O library that you want." You also say that only a tiny fraction of developers have those needs. Are any of them going to be reviewing this library?

Can you explain what is not straightforward more precisely please?

Sure. With regards to the examples: - Be more concise, - Have less standard out statements, - Have less comments, - Have no conditional compilation * BOOST_AFIO_USE_LEGACY_FILESYSTEM_SEMANTICS? How can parts of AFIO be legacy? * #if 0? In an example? * No platform specifics The other advice I have is that you may want to omit a comment like "This section was not finished in time for the beginning of the Boost peer review due a hard drive failure induced by the testing of AFIO-based key-value stores in this workshop tutorial (sigh!)" in the documentation. You don't want prospective reviewers to wonder if they should back up their hard drive before trying AFIO. :-) Glen -- View this message in context: http://boost.2283326.n4.nabble.com/afio-AFIO-review-postponed-till-Monday-tp... Sent from the Boost - Dev mailing list archive at Nabble.com.

Hi Niall, On 24 Aug 2015 5:15 pm, "Niall Douglas" wrote:

Lock free filesystem programming is like lock free atomic programming

"You keep using that word, I do not think it means what you think it means". Lock-free is a Word Of Power. Unless you implemented the whole stack (down to issuing disk ops) in user space or audited every supported OS kernel, I do not think you can claim any lock freedom property. BTW, Sorry for the snark, to make amend I'll try to write a review. -- gpd

Niall Douglas wrote:

And I have no problem if AFIO never enters Boost. I would think it a shame and a wasted opportunity as it solves a ton of pain points for those with such needs

Whatever happened to... > Niall Douglas wrote: > > Firstly, you don't invest the multi-year effort to get a library into > > Boost because it's superior, as it probably isn't. You do it because > > of what you will become after you succeed: a Boost library author > > who has passed the judgement of his or her peers. On a more serious note:

Totally separate to the async file i/o is the race free filesystem stuff. I would like to believe that people understand why a race free filesystem API is important.

Your documentation would be the best vehicle to convey (and prove) this. It doesn't do it for me right now. I may be the 0.0001% who doesn't need AFIO as you've currently designed it but ideally the documentation should help me understand why. Glen -- View this message in context: http://boost.2283326.n4.nabble.com/afio-AFIO-review-postponed-till-Monday-tp... Sent from the Boost - Dev mailing list archive at Nabble.com.

On 24 Aug 2015 at 11:52, Glen Fernandes wrote:

Niall Douglas wrote:

...

And I have no problem if AFIO never enters Boost. I would think it a shame and a wasted opportunity as it solves a ton of pain points for those with such needs

Whatever happened to...

> Niall Douglas wrote: > > Firstly, you don't invest the multi-year effort to get a library into > > Boost because it's superior, as it probably isn't. You do it because > > of what you will become after you succeed: a Boost library author > > who has passed the judgement of his or her peers.

Remember I have more than one Boost library coming :) AFIO will be the hardest to get in because it's so niche, and I've known that since the beginning. Monad will be hard to get in because it's too fundamental and everyone will have their own opinion and think they know better despite not having written one themselves, so we won't reach consensus. It'll just descend into argument. My final planned Boost library before I move on from Boost to warmer climes is a reliable (a)synchronous transactional key-value store implemented using AFIO and Monad. That will get into Boost without issue because *everybody* needs to persist state especially on unreliable hardware. That will be the only unequivocably useful library I propose for Boost which very little argument can be made against, and I expect it will be unconditionally accepted. Once it's in, it also becomes much harder to reject AFIO and Monad.

On a more serious note:

...

Totally separate to the async file i/o is the race free filesystem stuff. I would like to believe that people understand why a race free filesystem API is important.

Your documentation would be the best vehicle to convey (and prove) this. It doesn't do it for me right now. I may be the 0.0001% who doesn't need AFIO as you've currently designed it but ideally the documentation should help me understand why.

Ideally yes. But it's like writing long essays on why choose future over async_result. About 20% find it very useful and approve. About 20% think it clutter, confusing and annoying because it's just noise to them. You've seen people already say the AFIO documentation goes into too much detail about things you don't need to know. The remaining 60% don't care, and that suggests it has a poor cost benefit when I could do other much more valuable things with the same time than write essays on stuff more than half of people are indifferent about. In the end you're trapped. I take the view you don't ask uBLAS to explain from first principles matrix mathematics in its documentation. Why therefore ask the same of AFIO regarding file system theory? Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On Sat, Aug 22, 2015 at 4:28 PM, Niall Douglas wrote:

Well I asked here about that a few months ago, and I was told that there is precedent for this situation. Apparently historically you review the library being presented for review only, mentioning only the internal sublibraries if there is something catastrophically worrying about them. One then expects the internal sublibraries to be spun out into additional Boost libraries presented for review later if that is appropriate.

I can tell you that I personally would not be comfortable sending AFIO into the main Boost distribution until after Monad has been reviewed here, so even if there is a universal unconditional acceptance of AFIO by everyone here, I will personally guarantee I won't send in a final AFIO until after Monad is reviewed here.

So if AFIO is accepted, but "Monad" and "BindLib" are not, does that mean: a) They instead become an internal detail of libs/afio? (i.e. not libs/monad but libs/afio/monad) b) You wouldn't want AFIO to become part of Boost? I mean, reviewers might have a different opinion if accepting AFIO requires those two libraries must also become part of Boost. Glen

FWIW, what Monday are we talking about? Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Glen Fernandes Sent: Saturday, August 22, 2015 8:08 AM To: boost@lists.boost.org Subject: Re: [boost] [afio] AFIO review postponed till Monday

Hi Niall,

Did the review commence Friday? (Did the announcement mails only get sent to the -announce list?)

Glen

-- View this message in context: http://boost.2283326.n4.nabble.com/afio- AFIO-review-postponed-till-Monday-tp4678123p4679093.html Sent from the Boost - Dev mailing list archive at Nabble.com.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 22 Aug 2015 at 15:14, Glen Fernandes wrote:

Hartmut Kaiser wrote:

...

FWIW, what Monday are we talking about?

Monday on July-20. But the [new] review start date was Friday August-21.

I delivered a review ready AFIO on Friday. Ahmed is currently busy, and I am sure once he becomes unbusy he will announce the review in the usual way. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 23 Aug 2015 at 9:37, Roland Bock wrote:

...

...

Monday on July-20. But the [new] review start date was Friday August-21. I delivered a review ready AFIO on Friday. Ahmed is currently busy, and I am sure once he becomes unbusy he will announce the review in the usual way. Out of curiosity:

What's the big rush? Just a month ago, the library was in sucha state, that you canceled the review. Now you just "finished" a ready-for-review library on the day the review was supposed to start?

There is no rush. The library was finished in its present form around August 8th (you can check the commit log). Since then it's been almost entirely dotting i's and crossing t's and writing the new tutorial.

And the review is supposed to have started even though the review manager was so busy that he could not even send a single mail.

We all have unexpected things turn up when we don't expect. Ahmed is doing his best, same as all of us.

You also mentioned:

...

A later *internal* implementation will be completely new, but that isn't important to this review because little external changes. To me this is another indicator that maybe it is a bit early.

One of the aspects of the review is to look at the implementation.

Please DO look at the implementation. Any code which calls a filing system API won't be changing. The stuff which will be (internally) changing is how the ASIO reactor is used. That's all.

Sounds like we can drop that item for this review, since it won't last too long anyway?

The four APIs scheduled for deprecation listed at https://boostgsoc13.github.io/boost.afio/doc/html/afio/release_notes.h tml can be skipped. Otherwise all other implementation is up for review.

Don't get me wrong, maybe you are just the victim of your own honesty and full disclosure policy (admirable traits, btw). But judging by the mails you wrote in the last few days, if it were my library, I would kindly ask to treat this more like a pre-review of the new design. The full review would then be done once the implementation is as you want it.

I think some are minded to blow out of proportion how "unfinished" AFIO is for various personal reasons or out of ignorance. I would not be submitting AFIO for review here if I did not consider it production ready. And I should emphasise that the choice to implement the new API on top of the mature engine was something I came here to boost-dev to ask for advice upon many months ago, and what you have here today is what was wanted then. I have delivered what was asked of me - admittedly a month late, but three weeks of that went on writing a new tutorial which was asked of me by people here on boost-dev last month. If I hadn't have written the new tutorial due to the feedback last month, I would have been just a week late, not bad for a ~400 man hour project. I am happy to continue to receive feedback from people here on what further work needs to be done, and implement solutions to problems found by reviewers as I have been doing since 2013. That's why the library is up for review. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 22 Jul 2015 at 22:01, Asbjørn wrote:

Still, with temporary solutions/applications having a habbit of becoming permanent solutions[1], it still seems to me that the reliance on NT kernel API rather than just Win32 may stop it to work earlier than a pure Win32 application.

Ah sorry, you may not be aware that once upon a time the NT kernel had a POSIX subsystem which sat parallel to the Win32 and OS/2 and DOS subsystems [1]. This is how AFIO achieves its much closer to POSIX semantics than is possible in Win32 - by skipping Win32, and using the same NT kernel API as the subsystems use. Mingw, and Cygwin, do exactly the same trick BTW and it's how they achieve better POSIX conformance than MSVCRT can. In other words, AFIO's choice of using the NT kernel API was never a temporary solution. It was an original design choice from the very beginning because Mingw forces you to buy further into their ecosystem than many users want or need just to gain the improved POSIX filesystem semantics. Niall [1]: http://brianreiter.org/2010/08/24/the-sad-history-of-the-microsoft-pos ix-subsystem/ -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 22 Jul 2015 at 23:47, Asbjørn wrote:

On 22.07.2015 19:01, Niall Douglas wrote:

...

On 22 Jul 2015 at 10:27, Asbjørn wrote:

...

And if you haven't done so already, you probably should also clearly state in the docs why the Win32 API was insufficient.

I think it's on the first page :)

Maybe I'm blind as a bat, not an uncommon occurrence, but reading the BoostDoc[1] docs I cannot find any _clear_ mention of AFIO relying on the NT kernel api (it's mentioned in passing on a few unrelated pages). Nor can I find any rationale for this.

Feel free to hit me with a cluebat :)

You're right actually. I forget sometimes that what a text means to me does not mean the same to anybody else. I've logged this at https://github.com/BoostGSoC13/boost.afio/issues/81. Thanks for the observation. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Bjorn Reese wrote:

Do you intend this front-page to be the front-page of Boost.AFIO in the official Boost documentation?

I just assumed the front page he intended to show would be: https://boostgsoc13.github.io/boost.afio/doc/html/afio.html I hope it isn't the aforementioned one; I don't think test results make for a good documentation introduction. Glen -- View this message in context: http://boost.2283326.n4.nabble.com/afio-AFIO-review-postponed-till-Monday-tp... Sent from the Boost - Dev mailing list archive at Nabble.com.