Zitat von Dean Michael Berris <mikhailberis@gmail.com>:
512 | 2048 | 512 | 512 | 512 | ...
performance is much much worse than:
512 | [1536] | 2048 | 512 | 512 | ...
with 1536 bytes of garbage inserted so the 2048-bytes block can be written aligned to its own size. and yes, the sync after writing the garbage is necessary.
One other thing you can do is to pack the data into 512-byte divisible chunks. If you know you're writing 2048 in one go, then you might have to align to 512-byte chunks. Instead of writing "garbage" you can pack null bytes past the 2048 bytes you originally wanted to store.
I do have sector chunks, that was required for the reason alone that a used sector needs to be distinguishable from an unused sector, because of the zeroed out data at the end of the log. I still insert the "garbage" before the data, I don't think I understood why inserting at the end would be preferable, if that was what you're trying to explain? if cache locality was your only concern that doesn't really matter here, since CPU usage is way below 100%. there can be about 40000 un-flushed small transactions/s, so this is really about optimizing the syncs. in a mode that is only safe in case of an application crash, but not in case of a system failure (flush, but no sync) there are about 30000 transactions/s for the same test case.
I'm not sure if this is part of the requirements in your design or whether you want client threads to really block on transaction commits (even though the syncing is lock-free).
the worker threads are the user threads, so they need to block until the transaction is recorded committed (-> log sync). you could only avoid some syncs when 2 or more independent transactions run concurrently and they both need to sync, those could be combined. but that's not my focus right now.