[MSM] Comparison with ad-hoc FSM implementation

Hi Phil,
switch ((state<<8) | event) { case (Stopped<<8)|play: start_playback(); state=Playing; break; case (Stopped<<8)|open_close: open_drawer(); state=Open; break; etc. etc.
The greatest advantage of this ad-hoc style is that it is readable by anyone who has even the most basic understanding of C++ without the need to learn a new library, and its exact behaviour is immediately evident. It is also likely to compile and run quickly.
I have no doubt that MSM has many benefits compared to this. Perhaps it would be useful to spell them out?
The question is of course slighty provocative but still valid. The answer will be quite long, sorry about that. I won't say that your solution is not good. Actually if someone is happy with such a solution, great for him. If you check the review from Franz Alt, you'll see the source code generated by Rhapsody, which is not very different from this one, the O(1) double-dispatch excluded. And it's at least way better than non-formal state machines which I happen to see in production code again and again. A state machine library, including MSM, is made for people who are not happy with such an approach. I wrote MSM out of sheer frustration with Statechart's inability to provide me with what I was looking for, and out of admiration for the solution presented in the MPL book. I only hoped that others felt like me and would like an alternative. Concretely, what I was looking for: - a descriptive, expressive and declarative state machine syntax. - most of UML features which are hard to code with a switch/case dispatch. - a library with the explicit goal of supporting a Model-Driven-Development process - enough speed so that it'd be hard to reject a state machine simply on speed grounds (usual excuse). Next, you wil ask me what UML features I deem so important that it justifies a library. My personal favorites (order by what I use most): - events containing data. In your case, you are more or less forced to choose between the O(1) (which you did) and forgo event data or the O(n) (chosen by Rhapsody) at a speed cost. Metaprogramming techniques allow us a O(1) with event data. - actions/guards. Luckily, they are easy to implement in a switch/case. - orthogonal regions. I often use them for error handling and they greatly make the machine more readable by reducing the number of transitions. Also hard to get with a switch/case. - submachines (makes the code cleaner). Implementing them inside a switch case is simply horrible. Soon, you need to hide them in subclasses so we are already going in the direction of a library. - flags. Will make the calling code easier - exit points. Not my favorite, but always needed. Will quickly make your switch/case implementation hard to follow. - anonymous events. Hard to write with a switch/case dispatch. Luckily for your case, not everybody likes them. - history. Easy to implement in a switch/case (as long as you don't need them event-dependent) but doesn't make your code any more readable Of course, like always, developers use only 20% of all UML features. But it's never the same 20%, so what you need depends on your "coding" style. And if you want to implement all these features in a switch/case, your code will quickly become terrible. You'll have to issue guidelines on how to implement them, each developper will have to redo the same again and again with the corresponding flow of bugs (what if I forget a "break"? Uh oh) and low productivity (not even talking about documentation). Do we agree that this proves the need of a library? The next valid question would be, why MSM? It doesn't have to. If you're happy with Statechart or another library and don't need the speed, it's really ok for me, I promise ;-) MSM is for people who, like me, want a declarative syntax. People who want to have a look at the transition table and see the whole structure of a machine. Not as clearly as on a diagram but better than just code. That's why there is a transition table, and that's why there is eUML, to make it even clearer. This will greatly help us developers communicate with domain experts who will not understand code, but quite fast understand a state machine (I tried an it works). The second reason is the descriptive syntax of MSM. I wanted a library which would help a Model-Driven-Development. I wanted a library allowing us to switch from a graphical model to code and code to model. And I wanted the mapping to be complete. The goal would be to, at some point, never have to edit the code. I don't know how far I'll push it but you can be sure that I will push it as far as I can. This is where the functors added in eUML and the reusable states will help. I hope I managed to answer your question. Christophe

Christophe Henry wrote:
Hi Phil,
switch ((state<<8) | event) { case (Stopped<<8)|play: start_playback(); state=Playing; break; case (Stopped<<8)|open_close: open_drawer(); state=Open; break; etc. etc.
The greatest advantage of this ad-hoc style is that it is readable by anyone who has even the most basic understanding of C++ without the need to learn a new library, and its exact behaviour is immediately evident. It is also likely to compile and run quickly.
I used to do that, but found someone reordered the state assignment before the function calls, and one of the functions was dependent on the value of state. Having a truly declarative State Transition Table disallows accidentally changing the above procedural statements sequencing. It also helps reinforce the FSM mindset, which showed the aforementioned function should have been represented as additional states and events. As to whether the above is "immediately evident" it is not so evident to me. I need to interpret a very generic "switch" and "enum" facilities and intuit the semantics from the variable names that it's implementing an STT. Whereas in MSM and it's predecessor the TMP book's STT example declare a type ala: struct transition_table : boost::mpl::vector41 ... To me this is much more evident, and is analogous to using std::copy, or std::accumulate instead of a for-loop... the 1st word of the statement tells me the full semantics of whats to come. [snip]
- events containing data. In your case, you are more or less forced to choose between the O(1) (which you did) and forgo event data or the O(n) (chosen by Rhapsody) at a speed cost. Metaprogramming techniques allow us a O(1) with event data.
This was an important one for me in handling UI selection with an FSM that replaced lots of client code with a single shared state machine processing events such as pressed_item, shift_pressed_item, over_item, release_item... Since these are types, they can have multiple constructors to handle specific needs and be polymorphic for additional design expression. I've also found that using enums in the OP's fashion quickly degrades into a redundant pseudo-type-system increasing cognitive and maintenance overhead while decreasing type safety. [snip]
Of course, like always, developers use only 20% of all UML features. But it's never the same 20%, so what you need depends on your "coding" style.
Exactly!!! So now we'll have a standard place(or two, including the StateChart library) where all developers can look for documentation on how to address FSM issues in an accepted and consistent way. [snip] Jeff

An interesting post, make sure this information is included in the tutorial/introduction in the documentation. Robert Ramey Christophe Henry wrote:
Hi Phil,
switch ((state<<8) | event) { case (Stopped<<8)|play: start_playback(); state=Playing; break; case (Stopped<<8)|open_close: open_drawer(); state=Open; break; etc. etc.
The greatest advantage of this ad-hoc style is that it is readable by anyone who has even the most basic understanding of C++ without the need to learn a new library, and its exact behaviour is immediately evident. It is also likely to compile and run quickly.
I have no doubt that MSM has many benefits compared to this. Perhaps it would be useful to spell them out?
The question is of course slighty provocative but still valid. The answer will be quite long, sorry about that.
I won't say that your solution is not good. Actually if someone is happy with such a solution, great for him. If you check the review from Franz Alt, you'll see the source code generated by Rhapsody, which is not very different from this one, the O(1) double-dispatch excluded. And it's at least way better than non-formal state machines which I happen to see in production code again and again. A state machine library, including MSM, is made for people who are not happy with such an approach. I wrote MSM out of sheer frustration with Statechart's inability to provide me with what I was looking for, and out of admiration for the solution presented in the MPL book. I only hoped that others felt like me and would like an alternative. Concretely, what I was looking for: - a descriptive, expressive and declarative state machine syntax. - most of UML features which are hard to code with a switch/case dispatch. - a library with the explicit goal of supporting a Model-Driven-Development process - enough speed so that it'd be hard to reject a state machine simply on speed grounds (usual excuse).
Next, you wil ask me what UML features I deem so important that it justifies a library. My personal favorites (order by what I use most): - events containing data. In your case, you are more or less forced to choose between the O(1) (which you did) and forgo event data or the O(n) (chosen by Rhapsody) at a speed cost. Metaprogramming techniques allow us a O(1) with event data. - actions/guards. Luckily, they are easy to implement in a switch/case. - orthogonal regions. I often use them for error handling and they greatly make the machine more readable by reducing the number of transitions. Also hard to get with a switch/case. - submachines (makes the code cleaner). Implementing them inside a switch case is simply horrible. Soon, you need to hide them in subclasses so we are already going in the direction of a library. - flags. Will make the calling code easier - exit points. Not my favorite, but always needed. Will quickly make your switch/case implementation hard to follow. - anonymous events. Hard to write with a switch/case dispatch. Luckily for your case, not everybody likes them. - history. Easy to implement in a switch/case (as long as you don't need them event-dependent) but doesn't make your code any more readable
Of course, like always, developers use only 20% of all UML features. But it's never the same 20%, so what you need depends on your "coding" style. And if you want to implement all these features in a switch/case, your code will quickly become terrible. You'll have to issue guidelines on how to implement them, each developper will have to redo the same again and again with the corresponding flow of bugs (what if I forget a "break"? Uh oh) and low productivity (not even talking about documentation).
Do we agree that this proves the need of a library?
The next valid question would be, why MSM? It doesn't have to. If you're happy with Statechart or another library and don't need the speed, it's really ok for me, I promise ;-) MSM is for people who, like me, want a declarative syntax. People who want to have a look at the transition table and see the whole structure of a machine. Not as clearly as on a diagram but better than just code. That's why there is a transition table, and that's why there is eUML, to make it even clearer. This will greatly help us developers communicate with domain experts who will not understand code, but quite fast understand a state machine (I tried an it works).
The second reason is the descriptive syntax of MSM. I wanted a library which would help a Model-Driven-Development. I wanted a library allowing us to switch from a graphical model to code and code to model. And I wanted the mapping to be complete. The goal would be to, at some point, never have to edit the code. I don't know how far I'll push it but you can be sure that I will push it as far as I can. This is where the functors added in eUML and the reusable states will help.
I hope I managed to answer your question.
Christophe _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Hi Christophe, Thanks for your detailed reply. I think it's important that you add some sort of basic rationale of this sort to the documentation - you need to evangelise what you believe are the advantages of the library from first principles to its potential users. I'm just going to pick up on a couple of points, mainly because I don't know enough about UML to comment on the rest. Christophe Henry wrote:
- events containing data. In your case, you are more or less forced to choose between the O(1) (which you did) and forgo event data or the O(n) (chosen by Rhapsody) at a speed cost. Metaprogramming techniques allow us a O(1) with event data.
I'm not sure how true that is. To make this concrete, let me discuss the terminal emulator FSM that I described in the Boost.FSM review. This has one main event, "character received", that carries the character as data. There are a few other events such as "SIGWICH" (GUI terminal size changed by user). For most "character received" events in most states, the action is to insert the recieved character at the current cursor position and advance the cursor. When the "escape" character is received the FSM changes to another state in which the subsequent characters are processed differently. I would implement this something like: struct terminal_fsm { enum state_t { normal, seen_escape, ... }; state_t state; Screen screen; Position cursor_pos; void character_received(char c) { switch (state) { case normal: switch (c) { case escape: state = seen_escape; break; .... more special characters like tab, newline etc .... default: screen[cursor_pos] = c; ++cursor_pos; break; } break; case seen_escape: switch (c) { case 'H': // say esc H = 'home' cursor_pos = 0; break; .... lots more escape sequences .... } state = normal; break; } } void sigwinch(int rows, int cols) { screen.resize(rows,cols); // it would be a better example if this did something depending on // the state; use your imagination... } }; This is O(1). Of course a difference is that it has different entry points for the different events. This doesn't seem unreasonable to me, but if you wanted a single entry point you could do something like: struct character_received_event { char c; }; struct sigwinch_event { int rows, cols; } void process_event(const character_received_event& e) { .... void process_event(const sigwinch_event& e) { .... Now you can call process_event() with events of either type. How would you do this with MSM? Something like this presumably: row < normal, character_received_event, seen_escape, none, &is_escape_char >, row < normal, character_received_event, normal, &insert_at_cursor_and_advance >, row < seen_escape, character_received_event, normal, &cursor_home, &is_H >, row < seen_escape, character_received_event, normal, none >, row < any, sigwinch_event, unchanged, &resize_screen > (I'm guessing that the un-guarded row applies when none of the guards match for a particular state-event pair, and presumably there is some way of matching any state as in the last row.) When this is extended to include all the other special characters and escape sequences, you'll have lots of rows that have the same state-event pair but different guards, each testing the character in the character_received_event for a particular value. It looks to me as if it will do this in O(N), not O(1). Or is there some other way of doing this?
And if you want to implement all these features in a switch/case, your code will quickly become terrible. You'll have to issue guidelines on how to implement them, each developper will have to redo the same again and again with the corresponding flow of bugs (what if I forget a "break"? Uh oh) and low productivity (not even talking about documentation).
C++ provides many ways to shoot yourself in the foot, as we all know. I am reminded of the section in the C++ FAQ that says, paraphrasing: Q: Java has the "final" keyword that makes it impossible for a subclass to override a method. How do I do that in C++? A: Write a comment that says "Do not override this method in subclasses". If that doesn't work, write a comment that says "If you override this in a subclass you will be fired." That's not to say that anti-foot-shooting is a bad thing. I use boost::noncopyable a lot, for example. In my experience of implementing state machines, I have not encountered the "terrible" code that you imagine (or "spaghetti" and Andrey described it in the FSM review). This may be a matter of taste: one person's spaghetti is another's... err... noodles?
Do we agree that this proves the need of a library?
I can see that some believe they need a library to do this; personally I might choose not to use it, but that shouldn't bother you (and doesn't suggest that it should not be in Boost). You are welcome to try to convince me that it has benefits that I have not yet seen.... Regards, Phil.

Phil Endecott wrote:
Hi Christophe,
Thanks for your detailed reply. I think it's important that you add some sort of basic rationale of this sort to the documentation - you need to evangelise what you believe are the advantages of the library from first principles to its potential users.
I'm just going to pick up on a couple of points, mainly because I don't know enough about UML to comment on the rest.
Hello Phil - You may be familiar with Harel Statecharts. It is the "grand-daddy" of UML state machines and lends a very powerful concept over most standard state machine paradigms: hierarchically nested states. Hierarchical finite state machines provide a richer set of semantics that is well suited for medium to large machines. In UML these are called composite states or sub-state machines as Msm uses them. This concept is difficult (messy) to implement with ad-hoc techniques. I personally find that the richness provided by some of the modeling language constructs produces very simple solutions for what can often be messy problems to solve. It brings an elegance to the solution space that cannot be found in just states and transitions alone. It is the richness of the modeling language that helps one solve more complex problems without complexity. Sure, in these simple toy examples the ad-hoc switch statement looks as good if not better than a complex library. But as soon as we start making something that is industrial proof with all the error cases and whatnots ... well things start getting more complicated and I personally start looking for some additional constructs to help out. I certainly don't think a short little email will convince you differently. Nor is it my intent. I thought I would point out that it is the richness of the modeling language that allows you to attack more complex problems with a better paradigm. A library that implements those language constructs then allows an engineer to simply implement the model without a complicated transform. Take care - michael -- ---------------------------------- Michael Caisse Object Modeling Designs www.objectmodelingdesigns.com

This is O(1). Of course a difference is that it has different entry points for the different events. This doesn't seem unreasonable to me, but if you wanted a single entry point you could do something like:
I'd like to point out that the implementation of switch is generally pretty slow, and we're talking about a code which is generally on the critical path. Switch implies jumping to a computed address (something like jmp [eax*8+ebx]... you get the idea), this is very bad for branch prediction and for the pipeline of your beloved processor - assuming you're working on something close to the IA32 or AMD64. MSM's generated code will most likely contain only fixed jumps which behave much, much better, notwithstanding the fact that the template's context will help the compiler make better decisions about optimizations, resulting in favorable contexts in unconditional code. If you care about performance, you want those jumps out of your ways. -- EA __________ Information from ESET NOD32 Antivirus, version of virus signature database 4674 (20091209) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com

AMDG Edouard A. wrote:
This is O(1). Of course a difference is that it has different entry points for the different events. This doesn't seem unreasonable to me, but if you wanted a single entry point you could do something like:
I'd like to point out that the implementation of switch is generally pretty slow, and we're talking about a code which is generally on the critical path.
Switch implies jumping to a computed address (something like jmp [eax*8+ebx]... you get the idea), this is very bad for branch prediction and for the pipeline of your beloved processor - assuming you're working on something close to the IA32 or AMD64.
There are many ways that a compiler can handle a switch statement. a) sequential if/else b) binary search c) jump table Also, a jump table does not necessarily prevent the processor from predicting the branch.
MSM's generated code will most likely contain only fixed jumps which behave much, much better, notwithstanding the fact that the template's context will help the compiler make better decisions about optimizations, resulting in favorable contexts in unconditional code.
If you care about performance, you want those jumps out of your ways.
In Christ, Steven Watanabe

On Thu, 10 Dec 2009 05:50:43 -0800, Steven Watanabe <watanabesj@gmail.com> wrote:
There are many ways that a compiler can handle a switch statement. a) sequential if/else b) binary search c) jump table
And probably other ways, as the cases number rise I've seen only jump tables in binaries. if/else are hidden with the rest of the code making it not trivial to see it's not a switch. I'm not sure I understand what you're referring to when you talk about binary search? What kind of implementation is this and what are the benefits?
Also, a jump table does not necessarily prevent the processor from predicting the branch.
Probably, I guess there is much more intelligence in the branch prediction algorithms than I can imagine. However, AFAIK when the branch prediction fails it needs to wait for all the registers to be loaded and the computation to be done to get the good address. That's a serious stall. Well perhaps it's not a major issue if you consider the penalty you get for any branch fail, but I think this penalty has been greatly reduced since Core 2 (compared to P4). Main point : switches are slower for the same reason a if/then/else maze is slow. -- EA
participants (7)
-
Christophe Henry
-
Edouard A.
-
Jeff Flinn
-
Michael Caisse
-
Phil Endecott
-
Robert Ramey
-
Steven Watanabe