Questions about upcoming review of Boost.AFIO
Dear list,
Ahmed has volunteered to review manage AFIO (thanks Ahmed!), however
I wanted to get the list's feedback on some things first.
1. After collecting feedback at C++ Now from members of WG21 on the
shape and preferred future of a race free Filesystem TS, the next
iteration of AFIO will have a radically different API consisting of
the form:
std::vector
On 05/20/2015 09:56 AM, Niall Douglas wrote:
This is not a problem. It simply gets treated as an implementation detail of the AFIO library under review. There is precedence for this type of thing in which sometimes a useful detail becomes a stand-alone library (after an independent review) at a later time. We have also seen in the past that an implementation detail is too clever or deemed unnecessary by the community. That causes the review to bog down in territory that is not salient to the library core; however, must be resolved. michael -- Michael Caisse ciere consulting ciere.com
On 5/20/15 9:56 AM, Niall Douglas wrote:
Given the above, I would very strongly recommend that you postpone the review until you feel that the submission reflects what you want the library to look like. How would you feel if it was rejected - wouldn't you just kick yourself for not delaying for the submission of the presumably best version you can submit. As far as I know in the whole history of Boost there is only one time that a library was reject only to be accepted after a subsequent submission. Save yourself the agony of doing this more than once. BTW if AFIO requires APIBInd then that should be reviewed first or the two have to be reviewed together. This latter would be agony as the whole discussion of two nominally separate things would become intertwined. Just my 2 cents. Robert Ramey
On Wednesday 20 May 2015 17:56:48 Niall Douglas wrote:
IMO, the library should be in a more or less finalized state to be reviewed. After all, the review verdict shows acceptance or rejection of the library in its form during the review, not something entirely different that it will become in the future.
If it's just an implementation detail then it will be evaluated during the review as a part of the library implementation. I see no problem here.
On 20.05.2015 18:56, Niall Douglas wrote:
As a humble Boost user: please wait. The OSS project I worked on had enough issues with Boost.Filesystem v2 to v3 transition for it to be somewhat of a pain, and something to be avoided if possible. If AFIO was included into Boost with an already obsolete API I'd stay away anyway until the new API was released. Cheers - Asbjørn
Niall Douglas wrote:
What is your contingency plan if people loudly demand C++11 support (or even C++03 support)? Would it be possible to for the new API to degrade more or less gracefully to C++11, or is it C++14 or nothing? The answers to these questions may prove useful for your final decision.
review until you feel that the submission reflects what you want
Many replies, thank you everyone, replied to each in a single email to avoid repeating myself: On 20 May 2015 at 10:55, Robert Ramey wrote: the the
I feel the current API *is* the best version I can submit given the constraints (VS2013, ABI stability) imposed. I also have done *three* full release iterations now to get AFIO ready for peer review. They take weeks, each one, with dozens of hours of unit and stability soak testing done each time. I am kicking myself for having effectively prepared a new Boost library for review three times now, with arguably three times the work of others in the review queue. I'd really like to see the ball start rolling. I have been waiting in the queue two years now. I'd like to see a review so I know what I am doing wrong, and what I should be doing instead.
I would expect a conditional acceptance. And I'll tell you why: I am unaware of anyone using AFIO in production code. Until at least one person is, I myself would not recommend unconditional acceptance. The problem here really is chicken and egg, plus asynchronous file and filesystem is a very, very niche use case. I would wonder if any Boost library is needed by so few users in fact. However for that tiny number of users, async file and filesystem is a god send. I have some benchmarks in the tutorial which show just how much control async can give the programmer - find in files regex is quite literally warm cache double the speed of a naïve multithreaded algorithm with almost no cold cache penalty over single threaded, unlike the -50% cold cache penalty of a naive multithreaded algorithm. The race freedom guarantees on filesystem are also a god send for that tiny minority who care. On 20 May 2015 at 22:33, Andrey Semashev wrote:
Be aware that even with the next release after the next release new API variants are not finished, as I intend to add a Microsoft COM API with C API wrappers in a still further away release. That will mean no less than four separate APIs with which to work with AFIO: (i) ABI stable (the current one, will remain more or less in the future) (ii) Monadic programming (next release) (iii) Fiber programming (via proposed Boost.Fiber) which also doubles as the synchronous API WG21 want (next release). (iv) Microsoft COM API with C bindings (release after next release). I would also see any review being that of the library in its present form. Any future changes are, well, in the future and not in the scope of any review now. I would certainly hope that any review now might find problems very useful to know about now before any API refactor begins. On 20 May 2015 at 21:35, Asbjørn wrote:
Firstly, the current API isn't obsolete. It's fast, ABI stable, and very portable being deliberately designed to be usable from C. Future editions of AFIO will retain a mostly API compatible level with the only breaking change being std::future turning into afio::future and that error_code is now returnable from the afio::future in addition to exceptions. The monadic and fiber APIs almost certainly will wrap the current API, and therefore will be slightly slower. Boost libraries already undergo substantial API evolution. Look at Boost.Thread - it provides four API versions, all with breaking changes. I am unaware that any of those underwent a formal review, rather a post was made here informing people of any change in the default version provided (which indeed was rolled back last time). Also AFIO in its present form is 100% compatible with any future AFIO implementation even in the same translation unit. I can actually *guarantee* you that your code will not break in future Boost releases, unlike almost any maintainer here because I can ship N separate versions of AFIO and none will conflict. Do also remember AFIO is completely independent of Boost. You can use AFIO, including any past version, at any time including in combination with any future version with no ODR violation. This makes it very unlike Filesystem, or any other library here. On 21 May 2015 at 1:30, Peter Dimov wrote:
Most of the upcoming C++ 11 mandatory libraries require C++ 14. One already conditionally approved (Fiber) requires C++ 14. I think people also think in terms of compilers supported rather than C++ editions used.
In the refactor I'm aiming for the subset of C++ 14 provided by VS2015 RTM. That's about clang 3.4 levels minus variable templates and relaxed constexpr. This new API refactor won't happen till 2016 BTW. No time between now and then to go any faster. Plus I need Expression SFINAE, and VS2015 SP1 may have that so no point jumping before MSVC is ready. Right now AFIO v1.3 demands >= clang 3.2, >= GCC 4.7, >= VS2013. The -std=c++11 is sufficient, though -std=c++14 will produce faster code. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
Dear Niall, these are my 2 cents:
I am unaware of anyone using AFIO in production code.
This is how I see it: your library might be included in a Boost release almost in 2016. Those who need asynchronous file i/o on already existing VS2013 projects are already doing "their own thing". I doubt that these projects will be switching to a Boost 2016 release, and really doubt that these projects are going to switch "their own thing" for AFIO. Two years ago this might have made sense to some people (I know it did not make sense to some), but right now I think it is worth asking: Do you happen to know of any particular VS2013 project willing to do this? To me the single most important thing is that _you_ are going to be maintaining this library for the next 10 years if it gets accepted. Having to maintain a library filled with macros and hacks for the sake of old compiler support is painful enough. Having to do this twice for 2 different APIs is a lot of work. So I wish that you would focus on the API you want to have with the hope that this will keep you motivated longer and the whole community will benefit more in the long term.
TL;DR: cut out all of the C++11 legacy code and save your future self a lot of time: require clang 3.7, C++14, and submit a minimal library that you are willing to maintain for the next 10 years. Best regards and keep up the good work, I can't wait to see those constexpr futures in action!
On 21 May 2015 at 9:57, Gonzalo BG wrote:
More like 2017 assuming a conditional acceptance. I have very little free time.
Those who need asynchronous file i/o on already existing VS2013 projects are already doing "their own thing".
To be honest, VS2013 is usually far too new for most in this niche. They need more like VS2005 support, or earlier. They also generally want a C API, not a C++ API. This is my motivation for Microsoft COM bindings, and thence a C API wrapper. libuv is great, but I believe AFIO is better.
I'm not sure I understand the question? Willing to do what?
I'm actually pretty happy with the lack of macros and hacks in the v1.3 engine. All that portability layer stuff is abstracted into APIBind. I also fully expect to support the present API, or a close variant, permanently as it's the ABI stable API. In other words it's the only API in AFIO ever marked up with BOOST_AFIO_DECL for DLL export.
I did an informal poll at C++ Now of those working with MSVC at their day jobs, some deploying at a very large scale. A very large number currently target VS2005, indeed many said it was the most stable MSVC in the past decade which is why they settled on it (and why not, it's a very good C++ 03 compiler). Their plans, overwhelmingly, are a jump from VS2005 straight to VS2015 with no intervening MSVC. Their tests on VS2015 RC have had excellent stability and often a substantial improvement in performance, and in most cases their clang/GCC C++ 11/14 code paths work well on VS2015 without much work. As much as VS2013 is a fine compiler, VS2015 is enormously better again.
The only expected major breaking change in the current API is the replacement of standard futures with the custom future. The custom future can return a T, an error_code, or an exception_ptr. With standard futures the error_code is encapsulated as a system_error. I certainly can see that the community might demand a "finished" API. But firstly when is anything ever "finished"? And secondly, I believe I can convert the async_io_op struct over to an afio::future<T> standin type now before review begins, the afio::future<T> won't be the eventual implementation, but it would mean that the API is "finished" and ready for review. That's currently where I am heading mentally. And thank you community for your feedback. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On 5/20/15 3:30 PM, Peter Dimov wrote:
Boost has never required support of any specific compiler - just compatibility to the current standard - which I guess would mean C++14. In practice however, authors want to see their stuff used so will make efforts to support less conforming but popular compilers. And of course it depends on the type of package - The latest/greatest functional programming implementation would be find with supporting only the standard which users of some "lazy_sort" routine would expect support of older compilers - because it's easily possible and likely more widely used. Robert Ramey
participants (8)
-
Andrey Semashev -
Asbjørn -
Gonzalo BG -
John Maddock -
Michael Caisse -
Niall Douglas -
Peter Dimov -
Robert Ramey