Re: [boost] NuDB: A fast key/value insert-only database for SSD drives in C++11

On Thu, 30 Mar 2017 08:34:07 -0400 Lee Clagett wrote:

On Wed, 29 Mar 2017 17:18:23 -0400 Vinnie Falco via Boost wrote:

...

On Wed, Mar 29, 2017 at 5:04 PM, Niall Douglas via Boost wrote:

...

as that paper Lee linked to points out, everybody writing storage [...snip...]

This still has implementation/system defined behavior, but overwriting a single sector is more likely to be "atomic" from the perspective of the filesystem (but not necessarily the hard-drive). And it didn't require massive structural changes. Writing out a cryptographic hash of the header would leave a single assumption - fsync is a proper write barrier in the OS/filesystem and in the hard-drive. Niall has been particularly harsh on fsync, but I do not think its all bad. With the exception of OSX, it seems that many filesystems implement it properly (might regret saying this), and a user can purchase an "enterprise" hard-drive that is not trying to artificial boost benchmarks stats. At the very least the number of assumptions has been decreased.

I forgot that even with the cryptographic hash this algorithm is assuming that the filesystem does not change sectors during a single sector overwrite. Otherwise it could point back to a prior log header that was never actually overwritten. Crap. So a rare but still somewhat crummy non-portable issue. Lee