What if such a variant represents a communication protocol which can contain one of enumerated message type instance. In this case the sender can not put there a message (per-compiler) which should not be there (not allowed). And a receiver can be 100% sure that it will not receive a message which should not be handled.

Yeah, sounds cool, until I actually tried as you suggested. Unfortunately, variants can only hold a few types, e.g. 20. For most interfaces I program, there are far more than 20 messages. But this concept is still intriguing. Since variant doesn't cut it for 100 messages, how could one use the typelist concept in this case? terry

"Arkadiy Vertleyb" writes:

"Terry G" wrote

...

I've just finished reading C++ Template Metaprogramming again. I just don't get it.

Should probably not be your first book on template metaprogramming.

Really? We designed the book to serve as a complete introduction.

Really, it's just like trying to learn STL without any idea what a template is.

It's not as though the book starts out assuming you know what a metafunction is. What introductory material do you think we missed and should have covered?

...

Why would someone use a type container?

For example, one could implement a generic tuple class (although I don't think boost::tuple uses this approach, but probably fusion does?). Also see below...

Of course, that begs the question of why someone would want a tuple.

...

I'd really like a reference to an MPL primer or some case studies.

IMO, you should really start with template metaprogramming itself, and then move to MPL. Try Alexandrescu's "Modern C++ design" -- it also contains a lot of examples why you would want to use a typelist.

As great a book as "Modern C++ Design" is, it really surprises me to see it recommended as an introduction to template metaprogramming over "C++ Template Metaprogramming." MC++D is not about specifically TMP, and doesn't even attempt to describe what TMP is. It does contain numerous examples of basic TMP applied to traditional OO design patterns, which I think is the real point of that book, and IMO makes MC++D an ideal follow-on to C++TMP. In C++TMP we tried hard to both * define and clarify the basic concepts of TMP and * shield the reader from messy, low-level C++ details by using the MPL instead of the raw templates employed by MC++D.

Once you have a basic understanding of template metaprogramming, get back to the MPL book -- it's worth it.

Although we do spend considerable breath outlining the Boost facilities in "C++ Template Metaprogramming" (and despite the spelling of certain URLs ;-) ), we hope it won't be viewed as "the MPL book." The whole point was to treat TMP in a way that exposed its general principles and capabilities, without getting tied up in details of any particular technology, be it MPL, Loki, or raw C++ templates. -- Dave Abrahams Boost Consulting www.boost-consulting.com

"Arkadiy Vertleyb" writes:

"David Abrahams" wrote

...

"Arkadiy Vertleyb" writes:

...

"Terry G" wrote

...

I've just finished reading C++ Template Metaprogramming again. I just don't get it.

Should probably not be your first book on template metaprogramming.

Really? We designed the book to serve as a complete introduction.

...

Really, it's just like trying to learn STL without any idea what a template is.

It's not as though the book starts out assuming you know what a metafunction is. What introductory material do you think we missed and should have covered?

Well, it's all about approach to the problem. You present metaprogramming as a systematic discipline. Andrei presents it as a number of tricks. You define metafunction. Andrei just shows how to select between types and implement typelist. For some people your approach may be more natural, but I personaly prefer his as far as the introduction to the subject is concerned. Once I get a good feel of what can be done, and why, then I am ready to talk about systematic approach.

Hmm. Bottom-up vs. top-down is always a tension, and people have different learning styles that are more amenable to one or the other. We do try to show some simple low-level examples right in the first chapter (e.g. binary). Admittedly, that's not as useful an example as some of the things Andrei does with low-level code, but then it's also a *lot* simpler. We then continue with an actual, useful example of low-level type selection using specialization in chapter 2. Although we introduce the general idea of a "metafunction" in the beginning of chapter 2, we don't really try to define it until long after that example is complete. I am not arguing with you; I just want to learn how to serve my readers better, and so far it doesn't make sense to me. What you seem to be saying, by analogy, is that you'd want to see a whole book's worth of material on OO programming in 'C' in order to understand why the OO paradigm is useful, before reading about how to do it in C++ with real classes.

Another thing that IMO doesn't help understanding of the material, either in the book or in the MPL docs, is the extensive usage of lambda. IMO it just adds extra magic to the examples, making one feel that he doesn't really understand how things work. I think examples with regular metafunction objects would be much easier to understand, at least initially.

But we _do_ start with "regular metafunction objects." We don't start using lambda expressions until after we've shown how to work with metafunction classes.

...

...

IMO, you should really start with template metaprogramming itself, and then move to MPL. Try Alexandrescu's "Modern C++ design" -- it also contains a lot of examples why you would want to use a typelist.

As great a book as "Modern C++ Design" is, it really surprises me to see it recommended as an introduction to template metaprogramming over "C++ Template Metaprogramming." MC++D is not about specifically TMP, and doesn't even attempt to describe what TMP is. It does contain numerous examples of basic TMP applied to traditional OO design patterns, which I think is the real point of that book, and IMO makes MC++D an ideal follow-on to C++TMP.

Not really, once I read you book I would like to use MPL, right ;-)

Cute. Yes, it would be nice to see a version of MC++D rewritten to use MPL... but then it would be a much shorter book ;-)

As for me personally, I read Alexandrescu first, then Czarnecki (chapter 10, I think), and then your book. If I had to start now, I would use same order -- it works for me -- but I realize that it can be different for other people.

To summarize -- I think your book is great for the intermediate to advanced metaprogrammers, but as far as beginners are conserned, it may not always work. As in the case with the OP.

Granted, it may not always work. I'm still looking for some insight as to why not. -- Dave Abrahams Boost Consulting www.boost-consulting.com

All, I have been following this discussion and I would like to just add a general point. I have been trying to grok template metaprogramming off and on for a while now, and I think that the leap from object-oriented programming to template metaprogramming is a good deal larger than the leap from procedural programming to object-oriented programming. It could be because I never had Lisp or Scheme in college that makes it more difficult, but I've read 'Modern C++ Design' and 'C++ Template Metaprogramming' a couple of times each (not really cover to cover, but close enough) and I still have a hard time putting my brain in metaprogramming mode. So I guess the long and short of what I'm saying is that template metaprogramming is not something that beginning programmers will understand the utility of or even the mechanics of. Jeremy

FWIW - I'm comfortable with the idea of considering metaprogaming as similar to C++ macros on steriods. This fits in well with the term "metaprogramming" - programming the writing of a program. It also helps your brain from getting too confused between compile time and runtime behavior. When on get's used to this then it never occurs for one to write something like example below. And once your brain gets comfortable with this, MPL seems very, very natural. Robert Ramey

template <typename X> X divide_em(X x, X y) { if (typeid(x) == typeid(std::string)) return x; else return x/y; }

std::cout << divide_emstd::string("one", "two");

"Andrew Holden" writes:

Beth Jacobson wrote:

...

I'm afraid I can't comment on MPL, either the book or the library. I'd

...

assumed that there wasn't much point in even looking at it until I was

...

completely comfortable with metaprogramming. Reading this thread has made me question that assumption. Is MPL appropriate for metaprogramming newbies?

I would say that it has worked / is working for me. I began my study of metaprogramming with David and Aleksey's book and found that it provided a clear understanding of the principles. Using that book, I was able to write a metaprogram to help manage a dialog box's controls,

That sounds cool; can you tell us more?

so I would say it is effective.

Glad to hear it. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Beth Jacobson writes:

I'm afraid I can't comment on MPL, either the book or the library. I'd assumed that there wasn't much point in even looking at it until I was completely comfortable with metaprogramming. Reading this thread has made me question that assumption. Is MPL appropriate for metaprogramming newbies?

It's supposed to be. I'd appreciate hearing how it turns out for you. -- Dave Abrahams Boost Consulting www.boost-consulting.com

"Felipe Magno de Almeida" writes:

On 11/2/06, Arkadiy Vertleyb wrote:

...

"David Abrahams" wrote [snipped - discussion of TMP book]

Hi Arkadiy and David,

FWIW, I agree with Arkadiy about Andrei's book being easier as an introduction for TMP. It shows some metaprogramming tricks which helps building complete solutions for day-to-day problems.

Hmm, most of the problems solved in MC++D don't seem like day-to-day issues to me.

And that really impresses and makes people want to learn a lot more about the subject.

I see. Well, maybe we'll rewrite C++TMP as a 2-3 volume set and include some more of the low-level stuff in the beginning.

Like him I read Andrei's book before C++ Template Programming,

Unfortunately, it isn't really ever possible for them to compare one person's experience of starting with both. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Peter Dimov escreveu:

Terry G wrote:

...

I've just finished reading C++ Template Metaprogramming again. I just don't get it.

Why would someone use a type container?

Imagine a case where for some reason you would like to call

register();

for all combinations of X1..Xm and Y1..Yn. Writing the calls by hand gets tedious. If you put Xi and Yi in type sequences, you can write a metaprogram that generates the calls.

I have to say this is probably the most convincing example I've seen since the last time I read that tutorial thing about representing physical quantities. -- Pedro Lamarão

...

I've just finished reading C++ Template Metaprogramming again. I just don't get it.

Why would someone use a type container?

I have used enable_if, along with some of the logic functions for managing an overload set. Otherwise, I'm clueless.

I'd really like a reference to an MPL primer or some case studies.

The book contains a case study in which type sequences are used to describe the rows of a state machine's transition table. Was that not a sufficient motivating case?

FSMs are not something I'm familiar with. I didn't really notice the vector of rows until recently. How is the FSM approach in your book better than arrays of structs? I didn't see any advantage, and the supporting code is too complex. There must be an advantage, otherwise, why would you work so hard to write these things? I'll try to implement a reusable FSM framework, without MPL. Then, perhaps, advantages/disadvantages of MPL would be more apparent to me. Is there a good way to post hundreds of SLOCs for discussion? I don't have a website. BTW, on the boost website, there are three versions of the FSM player example. A ReadMe.txt to explain the differences would be helpful.

...

Too much pain with very little gain.

What, specifically, do you find painful?

Well, you asked... I write stuff that I think should work, but the compiler issues tons of error messages. Pitfalls like when to use typename. Brain cramps trying to comprehend the compile-time/run-time boundary. Recently, I tried to avoid using a large 150-case switch statement using metaprogramming. Each MessageNN has a unique static const int TAG member. They all derive from a common Message base class that has the common message header, including the tag. The only thing each case did was something like. switch (msg.tag()) { case Message26::TAG: cout << static_cast(msg) << endl; break; case Message27::TAG: cout << static_cast(msg) << endl; break; ... } That is, upcast a received message (over a wire) and print it. Virtual functions cannot be used because the message can have no vtbl. I eventually just did the big switch like I have for twenty years. The pain (frustration) comes from believing there's a "better" way, but not knowing how to realize it. Pain, too, because without a mentor to guide me, I feel like I'm walking around trying to drive in screws with my shiny new hammer. Painful because after you get a nifty mpl::vector of MessageTypes, you realize after the 50th message, that it won't scale up to 150. Painful because compile times take much longer. (VC8 takes a lot longer, not sure why). Painful because I don't understand the runtime and memory trade-offs anymore. i.e. painful because it makes me realize how stupid I still am, in spite of all my past successes. Printf and void* have served me well.

...

Time to ask for help. Rereading this, it sounds negative. Really, I think the MPL is really cool!

Cool, how can we help?

Do know how to do a mind-meld? (Spock recoiled, his face white with shock. He had never experienced such chaos before.) I need to be patient and ask one question at a time. There is a lot of really difficult material packed into a deceptively slim book. So, back to that switch statement... is there a better way? I thought of using something like: template <Tag> struct DerefTag { typedef Message type; } And then specializing it like this. template <> struct DerefTagMessage26::TAG { typedef Message26 type; } But that can only be used with compile-time constants, limiting its usefulness. Now I can write each case statement like this: case Message26::TAG: cout << static_castMessage26::TAG::type&>(msg) << endl; break; That's not better, its worse. Sigh. Should I give up? Is there a better way? BTW, how do compilers handle big switches? How do they find the matching case? Would an stl::map or sorted stl::vector of callback functors yield better portable performance? Hmmm... (<--- the sound I make before wasting even more time). terry

"Terry G" wrote

Recently, I tried to avoid using a large 150-case switch statement using metaprogramming.

This is not a very good case for applying TMP, IMO. If all you want is to avoid code repetition, you are better off using the Boost Preprocessor library. Regards, Arkadiy

"Terry G" writes:

...

...

I've just finished reading C++ Template Metaprogramming again. I just don't get it.

Why would someone use a type container?

I have used enable_if, along with some of the logic functions for managing an overload set. Otherwise, I'm clueless.

I'd really like a reference to an MPL primer or some case studies.

The book contains a case study in which type sequences are used to describe the rows of a state machine's transition table. Was that not a sufficient motivating case?

FSMs are not something I'm familiar with. I didn't really notice the vector of rows until recently. How is the FSM approach in your book better than arrays of structs?

Containing what?

I didn't see any advantage, and the supporting code is too complex.

It's actually quite simple (just a few pages long). If you have to write lots of FSMs, a little bit of complexity in a framework pays off as long as the framework makes defining FSMs very simple and understandable, as ours does. You have to separate the supporting code from the usage code. You'd never use TMP if you only needed to produce a single FSM. As the book points out in the first chapter, template metaprogramming is primarily for library writers targeting a particular problem domain (in this case, FSMs): it can allow their users to write maximally efficient, declarative code that expresses their intention in terms of the domain's own abstractions (in this case, state transition tables).

There must be an advantage, otherwise, why would you work so hard to write these things?

As the book points out, the FSMs generated by the framework as shown have efficiency comparable to that of handwritten FSMs that use case statements, and the declaration of the FSM resembles what one might write down when designing the state machine on a blackboard.

I'll try to implement a reusable FSM framework, without MPL. Then, perhaps, advantages/disadvantages of MPL would be more apparent to me.

Perhaps.

Is there a good way to post hundreds of SLOCs for discussion? I don't have a website.

You could use the Boost vault, I suppose.

BTW, on the boost website, there are three versions of the FSM player example.

Where?

A ReadMe.txt to explain the differences would be helpful.

...

...

Too much pain with very little gain.

What, specifically, do you find painful?

Well, you asked... I write stuff that I think should work, but the compiler issues tons of error messages.

Did you read the chapter on diagnostics and apply the advice therein?

Pitfalls like when to use typename.

Did you read the appendix on the typename and template keywords?

Brain cramps trying to comprehend the compile-time/run-time boundary. Recently, I tried to avoid using a large 150-case switch statement using metaprogramming. Each MessageNN has a unique static const int TAG member. They all derive from a common Message base class that has the common message header, including the tag. The only thing each case did was something like. switch (msg.tag()) { case Message26::TAG: cout << static_cast(msg) << endl; break; case Message27::TAG: cout << static_cast(msg) << endl; break; ... } That is, upcast a received message (over a wire) and print it.

Looks like a good candidate for TMP.

Virtual functions cannot be used because the message can have no vtbl. I eventually just did the big switch like I have for twenty years. The pain (frustration) comes from believing there's a "better" way, but not knowing how to realize it.

The FSM example shows the basic technique for "generating a switch" with TMP... although in this particular case you *might* have been better off using the preprocessor library (also covered in the book).

Pain, too, because without a mentor to guide me, I feel like I'm walking around trying to drive in screws with my shiny new hammer. Painful because after you get a nifty mpl::vector of MessageTypes, you realize after the 50th message, that it won't scale up to 150. Painful because compile times take much longer. (VC8 takes a lot longer, not sure why). Painful because I don't understand the runtime and memory trade-offs anymore.

i.e. painful because it makes me realize how stupid I still am, in spite of all my past successes. Printf and void* have served me well.

I feel your pain.

...

...

Time to ask for help. Rereading this, it sounds negative. Really, I think the MPL is really cool!

Cool, how can we help?

Do know how to do a mind-meld? (Spock recoiled, his face white with shock. He had never experienced such chaos before.) I need to be patient and ask one question at a time. There is a lot of really difficult material packed into a deceptively slim book.

So, back to that switch statement... is there a better way?

As in the FSM example: 1. Decompose the cases of the switch into separate functions, something like: template struct case_ { static void execute(BaseMsg const& msg) { if (msg.tag == DerivedMsg::TAG) cout << static_cast(msg) << endl; else NextCase::execute(msg); } } struct DefaultCase { static void execute(BaseMsg const& msg) { ... } }; 2. Now use MPL to generate a type like: typedef case_ > > switch_; you can do that with the "fold" algorithm over your sequence of derived Message types. 3. then invoke switch_(msg)

BTW, how do compilers handle big switches? How do they find the matching case?

It depends on the compiler and the distribution of values. If it's fairly dense they may just do array lookup in a jump table. Otherwise they may use perfect hashing. Otherwise it may just be a chain of ifs. If you want the optimizations that come from the first two schemes, you're better off generating an actual case statement with the preprocessor library, if possible. -- Dave Abrahams Boost Consulting www.boost-consulting.com

"Terry G" writes:

...

...

BTW, on the boost website, there are three versions of the FSM player example. Where?

Its in the Boost 1.33.1 download from the website, in boost/libs/mpl/example/fsm. I have the book CD too, of course. Which is "best"?

Oh, crud. We should have included a README file there, and I hope Aleksey will help me get this sorted out so I can write one. Here's what I think: player1.cpp - this is exactly what's covered in the book; in fact, it was auto-extracted from the examples as shown in the book. player2.cpp - this example demonstrates that the abstraction of the framework is complete by replacing its implementation with one that dispatches using O(1) table lookups, while still using the same code to describe the particular FSM. Look at this one if you want to see how to generate a static lookup table that's initialized at dynamic initialization time. player.cpp, state_machine.hpp - this is something Aleksey wrote. I'm not sure, but I think it predates the book, and is more sophisticated in some ways but probably should be treated as an advanced example.

...

...

I write stuff that I think should work, but the compiler issues tons of error messages.

Did you read the chapter on diagnostics and apply the advice therein?

Read, yes. Apply, mostly no. I use my text editor to process the verbiage, and I use two compilers. I've been reluctant to learn another tool, i.e. the filters.

Once you set up STLFilt, it just works, and it *really* eases the pain.

Are any of the Lints especially good with Boost/MPL?

Not that I know of.

...

...

Pitfalls like when to use typename.

Did you read the appendix on the typename and template keywords?

Read, yes. Understand, sort of. I can kind of tell when I've omitted the typename keyword, because that's what the compilers usually tell me.

Many compilers still don't tell you everyplace you're supposed to use it.

I never think to use it proactively.

That'll come with time. It's not an MPL-specific thing, of course. Anyone programming with templates will have to learn it eventually. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Aleksey Gurtovoy writes:

David Abrahams writes:

...

"Terry G" writes:

...

...

...

BTW, on the boost website, there are three versions of the FSM player example. Where?

Its in the Boost 1.33.1 download from the website, in boost/libs/mpl/example/fsm. I have the book CD too, of course. Which is "best"?

Oh, crud. We should have included a README file there, and I hope Aleksey will help me get this sorted out so I can write one.

<Aleksey helps> Thanks. I'll write a README file for libs/mpl/example/fsm immediately and put it on the HEAD and the RC branches. Would you mind writing one for libs/mpl/example/? Thanks again, -- Dave Abrahams Boost Consulting www.boost-consulting.com