Concurrency support in iostream mapped_file_source
Hi, Does boost iostream mapped_file_source support concurrent read from multiple threads? What is the best way to implement concurrent read on a performance sensitive application? The file is a read only one and its size is in gigabytes/terabytes Thanks, Lloyd
On 3/07/2020 18:01, Lloyd wrote:
Does boost iostream mapped_file_source support concurrent read from multiple threads? What is the best way to implement concurrent read on a performance sensitive application?
I haven't used it myself, but as far as I can tell from skimming the docs, as long as you open it beforehand and don't close it while the threads are reading, then the actual read operations are immutable and should be safe from any number of concurrent threads, since it's just const memory read access. If you construct a stream object around it, however, the stream will definitely not be thread-safe. So each thread would need its own. Or just access the mapped_file_source directly.
On 03/07/2020 07:01, Lloyd via Boost-users wrote:
Does boost iostream mapped_file_source support concurrent read from multiple threads? What is the best way to implement concurrent read on a performance sensitive application?
The file is a read only one and its size is in gigabytes/terabytes
For memory mapped files in general, reads served from kernel cache will parallelise at CPU concurrency level, whereas reads served from the storage layer will parallelise at device concurrency level. This means that i/o which is not served from cache will synchronise CPUs during i/o submission and completion. Niall
participants (3)
-
Gavin Lambert -
Lloyd -
Niall Douglas