Re: [Boost-users] circular buffer in a multithreaded program

Am 22.04.2014 20:02, schrieb Antony Polukhin:

2014-04-22 15:47 GMT+04:00 Sebastian Gesemann : <...>

...

- locking a mutex - calling cb.erase_first(n); - calling cb.resize(cb.capacity()); - notify writer that there is new free space to write stuff into - unlocking the mutex

This is what was happening in the reader thread. What was happening in the writer thread?

Let me back up a little ... I wrapped the circular_buffer into another object that offers this interface: range allocate(int n); // for writer void commit(int n); // for writer const_range read(int n); // for reader void consume(int n); // for reader Apart from the container that backs up the storage (which used to be a circular_buffer), it also contains - a mutex - two condition variables: - enough_writable (which allocate blocks on) - enough_readable (which read blocks on) All of these four actions include locking the mutex. The first function of a pair returns a range object (think of it as a pair of iterators) with which a thread actually reads or overwrites the data _without_ owning a lock. The second function of a pair updates the state to indicate that either new data is available for a reader or that new free space is available for the writer. The only interesting bit is the consume function which I tried to describe in my first email. Other than that, the only things happening with circular_buffer was invoking begin() and/or end() and doing some "iterator arithmetic" in allocate/read while having a lock. So, essentially, begin/end/erase_first/resize was only executed when having a lock where the last ones did not invalidate any iterators that were in use somewhere. Deref'ing of the iterators was not locked. My guess is that "working" with an iterator while concurrently doing an erase_first/resize creates a data race -- possibly because an iterator tries to read some of circular_buffer's state which is modified via erase_first and/or resize. But that's just a guess. I did not check the circular_buffer implementation too closely.

...

But the program did not work. The first thing that came to my attention was that the checked iterators of circular_buffer don't support this kind of multithreading as they register and unregister themselves into a linked list without any synchronization.

I had exactly the same thoughts...

This was definitely an issue. But it's not the only one apparently. Cheers! sg