On Sun, Mar 26, 2017 at 7:11 AM, Niall Douglas via Boost
If you want durability across multiple OSs and filing systems, you need to assume that fsync's are reordered with respect to one another.
Does SQLite assume this? I make no claims that NuDB works on all filing systems. It will definitely not do well on network shares - it wasn't designed for that. Nor will it perform well on spinning disks. It is specifically designed for directly connected SSD drives, on mainstream operating systems. As I said in the original post, its use-cases are niche (hence, why I am probing for the level of interest).
NuDB needs to remove all claims regarding durability of any kind. In fact, you might as well remove the fsync code path entirely
No, I don't think that I'll be doing that. If you feel that this is important, you have my blessing to clone the repository and continue with your own fork.
fsync() performs pathologically awful on copy-on-write filing systems
The library is not designed for exotic file systems like the one you describe. Its meant for simple commodity hardware and operating systems such as what you might find on a bare metal amazon web instance. There is no need for a copy on write file system, as long as the invariants are met (that fsyncs aren't reordered).
In which case you did not make a great choice.
Much, much better would be Blake2b. 2 cycles/byte, cryptographically secure, collision probability exceeds life of the universe.
Hmm, no, it seems that you are the one who "did not make a great choice." The requirements of Hasher do not include cryptographic security. Blake2b is a cryptographically secure hash function which computes digests up to 32 bytes, while xxhasher is a non cryptographically secure hash function which computes a 64-bit digest. NuDB requires a Hasher more like std::hash and less like SHA-1. Blake2b can achieve almost 1Gb/s while xxhash can achieve 110Gb/s. I think I'll be sticking with xxhash but again you have my full support if you want to instantiate nudb::store objects with your own user defined Hasher that uses Blake2b - here's a link to the type requirements so you can get started with it: http://vinniefalco.github.io/nudb/nudb/types/Hasher.html