On 6 Feb 2015 at 13:52, Olaf van der Spek wrote:
So if I try to write a std::vector that is too large to fit in a scatter/gather buffer, my write will fail?
AFIO observes the IOV_MAX batch limit, so on POSIX with pwritev() support no it should never fail, though of course you lose atomicity between IOV_MAX batches. On POSIX without pwritev() support AFIO issues each buffer singly anyway, so your atomicity is per buffer.
On Windows if buffered i/o is on then there is no limit and atomicity is per buffer (Windows has no scatter gather file i/o functions for buffered files). If buffered i/o is off, the WriteFileGather() API currently has an unofficial limit of 32Mb on x64 operating systems due to NT kernel structure limits. Because this limit is not documented and not stable even across 32 bit vs 64 bit systems never mind between Intel and ARM, AFIO passes through your request as-is, and returns an error if the WriteFileGather() API does.
What's the definition of atomicity here?
Atomicity on filing systems is that the whole of a write operation will be atomically seen as a single unit by all readers of that file whether in other processes or other machines [1]. So if you write 64Kb of data, either none of the 64Kb will appear to readers, or all of it will. You can't see it mid write. This feature lets you do fun stuff like distributed mutual exclusion algorithms using atomic appends as the message channel, and extent zeroing on the front of the file to prevent the file growing in physical allocation. You can see an example of such an algorithm at https://boostgsoc13.github.io/boost.afio/doc/html/afio/quickstart/asyn c_file_io/atomic_logging.html where performance, except on ZFS, is very respectable. Plus the code is completely platform independent. [1]: Excluding mmaps. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/