On 11/13/2013 12:11 PM, Niall Douglas wrote:
On 12 Nov 2013 at 10:52, foster brereton wrote:
Would these algorithms be of general use to the Boost community? If so I would be willing to submit them formally.
Be aware that AFIO (currently in the peer review queue) is slowly gaining an asynchronous batch hash engine. It's a bit different from your normal hash engine, because it can do things like use SIMD to process four SHA256 streams in parallel, and then using AFIO's closure engine to parallelise that 4-SHA256 processing across multiple cores i.e. achieve sixteen parallel SHA256 processing streams. This lets one drop the normal 14.9 cycles/byte down to around 1.4 cycles/byte amortised for SHA256 [1], a big win. The batch hash engine uses a compile-time plugin system, so it can be arbitrarily extended with other hash implementations if suitably rewritten to fit.
Of course AFIO may not pass peer review, so any work done here may be moot. That therefore should not inhibit your efforts.
In case anyone wants to know "when will it be ready?", well being unemployed and having to relocate back to Europe has punched a huge hole in my productivity. Once I'm back (January) I would expect things to move far more swiftly. I should have something which passes basic unit tests this month though - I already have a proof of concept working, it's just converting it into something not so hacked together.
[1] These figures are for an Intel i7-3770K running x64. Other CPUs vary hugely (e.g. ARM, whose NEON unit is not well scheduled by GCC or clang).
0Sounds interesting. Is the hash engine customizable to composite algorithms? Particularly MD5/SHA1? I'm blocked from using AFIO(on Windows) for my file processing as I need to go thru BackupRead/BackupWrite api's which are explicitly incompatible with overlapped io. I haven't looked if AFIO could offer benefits with that restriction. Thanks, Jeff