[statechart] Asynchronous Machines
From the documentation it seems like passing zero to FifoWorker::operator() (via fifo_scheduler::operator()) will do the
If I have an asynchronous state machine running in another thread and I send it an event, is there any way to have the sending thread block until the event is processed? I need to use an asynchoronous machine because the rest of my app is threaded but yet there are points where I must synchronize to know the machine is in a certain state before moving on. trick, but I'm not sure. I've created a non-blocking asynchronous machine because the thread calling fifo_scheduler::operator() should not block if the queue is empty. It should process all events in the queue and return. Have I got that all correct? -Dave
David Greene wrote:
If I have an asynchronous state machine running in another thread and I send it an event, is there any way to have the sending thread block until the event is processed? I need to use an asynchoronous machine because the rest of my app is threaded but yet there are points where I must synchronize to know the machine is in a certain state before moving on.
From the documentation it seems like passing zero to FifoWorker::operator() (via fifo_scheduler::operator()) will do the trick, but I'm not sure.
I've created a non-blocking asynchronous machine because the thread calling fifo_scheduler::operator() should not block if the queue is empty. It should process all events in the queue and return.
Have I got that all correct?
Quick follow-up: when the documentation says that FifoWorker::operator() "must only be called from exactly one thread," does that mean it must be called only from one thread at any given time (i.e. it must be protected with mutexes or other concurrency protection) or that literally only one thread may ever call it and it must always be the same thread that calls it? If it's the latter then that changes things quite dramatically and I probably want a blocking scheduler and will need to create another thread just to wait on empty and call operator() when full. -Dave
Hi Dave, David Greene wrote:
David Greene wrote:
If I have an asynchronous state machine running in another thread and I send it an event, is there any way to have the sending thread block until the event is processed?
Yes: Add a boost::function member to the event. Have the member point to a function which calls notify_one() on a boost::condition. Have the sending thread call wait() on the condition. When processing the event inside the state_machine, call the boost::function member.
I need to use an asynchoronous machine
because the rest of my app is threaded but yet there are points where I must synchronize to know the machine is in a certain state before moving on.
From the documentation it seems like passing zero to FifoWorker::operator() (via fifo_scheduler::operator()) will do the trick, but I'm not sure.
See below.
Quick follow-up: when the documentation says that FifoWorker::operator() "must only be called from exactly one thread," does that mean it must be called only from one thread at any given time (i.e. it must be protected with mutexes or other concurrency protection) or that literally only one thread may ever call it and it must always be the same thread that calls it?
The latter. fifo_worker::operator() calls terminated(), which in turn returns the bool member terminated_. This call chain is not protected by mutexes. Picture the following scenario with one fifo_worker object: 1. Thread A, which is executed on CPU 1, calls operator(). Inside operator(), terminated_ is modified and operator() returns. 2. Thread B, which is executed on CPU 2, calls operator(), after thread A has returned from operator(). Nowadays both CPUs tend to have caches. Whether or not the cache contents is guaranteed to be written back to the main memory when thread A returns depends on the architecture of your hardware. IIRC, on X86 architectures there is such a guarantee. On other architectures you might need to use mutexes or a similar concept to guarantee that thread B sees the updates of thread A.
If it's the latter then that changes things quite dramatically and I probably want a blocking scheduler and will need to create another thread just to wait on empty and call operator() when full.
I'm not sure I understand that. You only need to create a blocking scheduler and call operator() on it, that's it. operator() will return when you call terminate() from any other thread. HTH & Regards, -- Andreas Huber When replying by private email, please remove the words spam and trap from the address shown in the header.
Andreas Huber wrote:
Hi Dave,
Hi Andreas, thanks for your help.
David Greene wrote:
David Greene wrote:
If I have an asynchronous state machine running in another thread and I send it an event, is there any way to have the sending thread block until the event is processed?
Yes: Add a boost::function member to the event. Have the member point to a function which calls notify_one() on a boost::condition. Have the sending thread call wait() on the condition. When processing the event inside the state_machine, call the boost::function member.
Hmm...that's rather complex, but it makes sense.
Quick follow-up: when the documentation says that FifoWorker::operator() "must only be called from exactly one thread," does that mean it must be called only from one thread at any given time (i.e. it must be protected with mutexes or other concurrency protection) or that literally only one thread may ever call it and it must always be the same thread that calls it?
The latter. fifo_worker::operator() calls terminated(), which in turn returns the bool member terminated_. This call chain is not protected by mutexes. Picture the following scenario with one fifo_worker object: 1. Thread A, which is executed on CPU 1, calls operator(). Inside operator(), terminated_ is modified and operator() returns. 2. Thread B, which is executed on CPU 2, calls operator(), after thread A has returned from operator().
Nowadays both CPUs tend to have caches. Whether or not the cache contents is guaranteed to be written back to the main memory when thread A returns depends on the architecture of your hardware. IIRC, on X86 architectures there is such a guarantee. On other architectures you might need to use mutexes or a similar concept to guarantee that thread B sees the updates of thread A.
Mutexes don't effect cache coherence. Likely there will have to be calls to special intrinsics depending on the architecture.
If it's the latter then that changes things quite dramatically and I probably want a blocking scheduler and will need to create another thread just to wait on empty and call operator() when full.
I'm not sure I understand that. You only need to create a blocking scheduler and call operator() on it, that's it. operator() will return when you call terminate() from any other thread.
Yes, what I said doesn't make sense given your solution outlined above. Hmm...I was going to use a synchronous machine due to the state access problems I outlined in another message, but now I see that won't work if terminated_ is in cacheable memory and there is no hardware coherence. For my purposes that's probably ok because we'll only run on machines with hardware coherence. Therefore if I understand you correctly, it's ok if different threads calls process_event() as long as the call (and any call to statechart routines) are guarded by mutexes to avoid the non-reentrancy problems. -Dave
David Greene
Mutexes don't effect cache coherence. Likely there will have to be calls to special intrinsics depending on the architecture.
See Gottlob's response..
Hmm...I was going to use a synchronous machine due to the state access problems I outlined in another message, but now I see that won't work if terminated_ is in cacheable memory and there is no hardware coherence. For my purposes that's probably ok because we'll only run on machines with hardware coherence. Therefore if I understand you correctly, it's ok if different threads calls process_event() as long as the call (and any call to statechart routines) are guarded by mutexes to avoid the non-reentrancy problems.
That'll probably work, given that you use a single mutex to protect scheduler & machine. However, this is rather ugly as you end up locking two mutextes whenever you queue an event. Regards, -- Andreas Huber When replying by private email, please remove the words spam and trap from the address shown in the header.
Andreas Huber wrote:
Hmm...I was going to use a synchronous machine due to the state access problems I outlined in another message, but now I see that won't work if terminated_ is in cacheable memory and there is no hardware coherence. For my purposes that's probably ok because we'll only run on machines with hardware coherence. Therefore if I understand you correctly, it's ok if different threads calls process_event() as long as the call (and any call to statechart routines) are guarded by mutexes to avoid the non-reentrancy problems.
That'll probably work, given that you use a single mutex to protect scheduler & machine. However, this is rather ugly as you end up locking two mutextes whenever you queue an event.
Does this imply that there's a mutex in state_machine somewhere that gets locked on some operations? That could definitely be problematic and I'll have to take extra care to avoid deadlock. -Dave
David Greene
That'll probably work, given that you use a single mutex to protect scheduler & machine. However, this is rather ugly as you end up locking two mutextes whenever you queue an event.
Does this imply that there's a mutex in state_machine somewhere that gets locked on some operations? That could definitely be problematic and I'll have to take extra care to avoid deadlock.
No, state_machine is not thread-safe and non-reentrant. It does not make any calls to synchronization primitives. IIRC, the only functions that make such calls are fifo_worker::terminate, fifo_worker::queue_work_item, fifo_worker::operator(). So, AFAICS, deadlock should not be an issue. Regards, -- Andreas Huber When replying by private email, please remove the words spam and trap from the address shown in the header.
David Greene wrote:
If I have an asynchronous state machine running in another thread and I send it an event, is there any way to have the sending thread block until the event is processed? I need to use an asynchoronous machine because the rest of my app is threaded but yet there are points where I must synchronize to know the machine is in a certain state before moving on.
Can't you use something like the half-sync/half-async design pattern to solve this ? Regards, Leon Mergen
Leon Mergen wrote:
David Greene wrote:
If I have an asynchronous state machine running in another thread and I send it an event, is there any way to have the sending thread block until the event is processed? I need to use an asynchoronous machine because the rest of my app is threaded but yet there are points where I must synchronize to know the machine is in a certain state before moving on.
Can't you use something like the half-sync/half-async design pattern to solve this ?
Do you have a pointer? I'm not familiar with that pattern. -Dave
David Greene wrote:
If I have an asynchronous state machine running in another thread and I send it an event, is there any way to have the sending thread block until the event is processed? I need to use an asynchoronous machine because the rest of my app is threaded but yet there are points where I must synchronize to know the machine is in a certain state before moving on.
Moreover, how do I actually extract information from the state machine? Since fifo_scheduler<>::processor_handle is a weak_ptr<> I can't use it to query the state of the machine or extract any machine-specific information (such as elapsed time in the StopWatch example, if it were asynchronous). -Dave
David Greene wrote:
Moreover, how do I actually extract information from the state machine? Since fifo_scheduler<>::processor_handle is a weak_ptr<> I can't use it to query the state of the machine or extract any machine-specific information (such as elapsed time in the StopWatch example, if it were asynchronous).
The fact that it's a weak_ptr is an implementation detail. You don't need to know that as a user. Besides creation and destruction of FSMs, the fifo_scheduler interface only allows you to send events. So, the only means of data exchange is through callbacks and events (as outlined in the other post). It is not possible to access the state_machine subclass objects directly because that would mean accessing data shared between threads. This in turn would mean that every state_machine member function needs to be protected by mutexes, which is a recipe for disaster, in my experience. HTH & Regards, -- Andreas Huber When replying by private email, please remove the words spam and trap from the address shown in the header.
Andreas Huber wrote:
David Greene wrote:
Moreover, how do I actually extract information from the state machine? Since fifo_scheduler<>::processor_handle is a weak_ptr<> I can't use it to query the state of the machine or extract any machine-specific information (such as elapsed time in the StopWatch example, if it were asynchronous).
The fact that it's a weak_ptr is an implementation detail. You don't need to know that as a user. Besides creation and destruction of FSMs, the fifo_scheduler interface only allows you to send events. So, the only means of data exchange is through callbacks and events (as outlined in the other post). It is not possible to access the state_machine subclass objects directly because that would mean accessing data shared between threads. This in turn would mean that every state_machine member function needs to be protected by mutexes, which is a recipe for disaster, in my experience.
I understand your reasoning, but it makes it _really_ difficult to incorporate Boost.Statechart into an existing application. I'd have to create all kinds of callbacks and special events just to get some state out of the machine and make decisions based on that information. It seems as though with asynchoronous machines, it's almost an all-or-nothing affair. Either everything interacts via Boost.Statechart objects or there is quite a bit of painful interface code to coax out information when needed to drive some other piece of the application. Hopefully I can get by with a synchronous state machine. It's a very tiny piece of my application but it is the most critical piece. Before it was lots of hand-written ugly code and I was hoping to formalize it with Boost.Statechart. I think I can but it's just turned out to be more challenging than I thought. I'm not meaning to bash your work. I think it's quite wonderful, actually. I'm just relating one user's experience and pointing out some things that might be interesting to discuss. -Dave
David Greene
The fact that it's a weak_ptr is an implementation detail. You don't need to know that as a user. Besides creation and destruction of FSMs, the fifo_scheduler interface only allows you to send events. So, the only means of data exchange is through callbacks and events (as outlined in the other post). It is not possible to access the state_machine subclass objects directly because that would mean accessing data shared between threads. This in turn would mean that every state_machine member function needs to be protected by mutexes, which is a recipe for disaster, in my experience.
I understand your reasoning, but it makes it _really_ difficult to incorporate Boost.Statechart into an existing application. I'd have to create all kinds of callbacks and special events just to get some state out of the machine and make decisions based on that information.
Multi-threaded programming is difficult ;-). Seriously, I'm aware that the asychronous_state_machine & scheduler combo has a very restrictive interface, which somewhat enforces the event-based programming rule that accessing shared state from different threads is best avoided.
It seems as though with asynchoronous machines, it's almost an all-or-nothing affair. Either everything interacts via Boost.Statechart objects or there is quite a bit of painful interface code to coax out information when needed to drive some other piece of the application.
I do understand that there are scenarios where shared access is unavoidable, asynchronous_state_machine was simply not designed to support them. However, you can always use the thread-unsafe state_machine directly and appropriately protect it with mutextes.
Hopefully I can get by with a synchronous state machine. It's a very tiny piece of my application but it is the most critical piece. Before it was lots of hand-written ugly code and I was hoping to formalize it with Boost.Statechart. I think I can but it's just turned out to be more challenging than I thought.
I'm not meaning to bash your work.
No bashing taken. Maybe I should have documented the fact that the asynchronous part of the library only exists because there's currently no library in Boost that supports multi-threaded event-based programming. As such it pursues (and always will be) a minimal approach. HTH & Regards, -- Andreas Huber When replying by private email, please remove the words spam and trap from the address shown in the header.
participants (3)
-
Andreas Huber -
David Greene -
Leon Mergen