On Tue, 12 Sep 2006 07:06:39 -0400, Brian Allison wrote:

In my experience, if I'm fixing a bug in some code and notice that an object is constructed in multiple phases, then I immediately start redesigning it so that each invariant is a distinct type.

I'm intrigued :-) Could you please provided an example with a minimal complete definition of the invariant types? -- [ Gennaro Prota. C++ developer, Library designer. ] [ For Hire http://gennaro-prota.50webs.com/ ]

At 9:06 PM -0700 9/13/06, Scott Meyers wrote: [ snip ]

I'll note that C++ itself allows "uninitialized" objects with constructors to be created all the time:

std::ofstream ofs; std::vector<int>::iterator i; std::string s;

Just a nit - I think that your third example is not like the others. A std::string, AFAIK, constructed with the default constructor, is perfectly valid - just empty. The other two - those, I'll give you ;-) -- -- Marshall Marshall Clow Idio Software mailto:marshall@idio.com It is by caffeine alone I set my mind in motion. It is by the beans of Java that thoughts acquire speed, the hands acquire shaking, the shaking becomes a warning. It is by caffeine alone I set my mind in motion.

Scott Meyers wrote:

Marshall Clow wrote:

...

...

I'll note that C++ itself allows "uninitialized" objects with constructors to be created all the time:

std::ofstream ofs; std::vector<int>::iterator i; std::string s; Just a nit - I think that your third example is not like the others. A std::string, AFAIK, constructed with the default constructor, is

At 9:06 PM -0700 9/13/06, Scott Meyers wrote: [ snip ] perfectly valid - just empty.

In each case, a default constructor is invoked. They're all valid objects that presumably fulfill their invariants, it's just that you can't safely invoke very many operations on them. On the string s, for example, invoking size is fine, but invoking operator[] is not fine at all.

But that's true for string even with valid non-default construction: std::string s(""); And also true, in possibly invoking erroneous behavior, of all C++ arrays and array like objects. It's not a "defect" of the default construction of string. But a "defect" of the string interface. Given a differently designed interface, that did not have the operator[], would not have that "defect". Note, I'm not suggesting that this would, by inference, be extended to two phase construction designs. But then again, this is a matter of perspective. If one considers the construction to determine the allowable interface, then your string example is apropos. To me that suggest implementing, when required, two phase designs with double construction. For example: const_string s0; string s1(s0+"something"); But that looks hellishly complicated to pull off design wise. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo

Scott Meyers wrote:

Marshall Clow wrote:

...

At 9:06 PM -0700 9/13/06, Scott Meyers wrote: [ snip ]

...

I'll note that C++ itself allows "uninitialized" objects with constructors to be created all the time:

std::ofstream ofs; std::vector<int>::iterator i; std::string s;

Just a nit - I think that your third example is not like the others. A std::string, AFAIK, constructed with the default constructor, is perfectly valid - just empty.

In each case, a default constructor is invoked. They're all valid objects that presumably fulfill their invariants, it's just that you can't safely invoke very many operations on them.

No, a singular iterator is not a valid object and it fulfills no invariants. You can't even copy it without UB. The ofstream can be made to fulfill its invariant if we define its invariant as "true", I guess, or something with a similar utility. A default-constructed std::string is fine. You are not breaking the invariant with invoking op[], you are supplying an invalid index to it. BTW, did you know that the const version of op[] is required to return 0 for s[0] :-) How exactly is it supposed to return 0 when the return type is a char const& is left as an exercise for the implementer.

David Abrahams wrote:

"Peter Dimov" writes:

...

No, a singular iterator is not a valid object and it fulfills no invariants.

That's arguable. From my POV, if it's in a state that a legal program can create, it's within the invariant by definition.

Then (if I read you correctly) even undefined behavior is within the invariant? Or have I been misunderstanding that legal programs can cause UB? I've always considered that UB is to be treated as "not maintaining the invariant but it's not my fault". But then, whether we consider UB within the invariant (and hence that an iterator is by definition always within the invariant) or whether we consider UB outisde of the invariant but one which doesn't break correctness (and hence that unassigned iterators are not capable of being within/outside of an invariant).... ... is there any practical difference between those two points of view? thanks, Brian

David Abrahams wrote:

Brian Allison writes:

...

David Abrahams wrote:

...

"Peter Dimov" writes:

...

No, a singular iterator is not a valid object and it fulfills no invariants.

That's arguable. From my POV, if it's in a state that a legal program can create, it's within the invariant by definition.

Then (if I read you correctly) even undefined behavior is within the invariant?

No, invariants are about state and UB is about behavior. Behaviors don't fall inside or outside of states.

...

Or have I been misunderstanding that legal programs can cause UB?

I don't understand the question, sorry.

When a program causes undefined behavior, that falls into the category I'm calling "illegal program." I don't just mean those programs that can be diagnosed as illegal by the compiler.

I misunderstood you - I thought you meant "illegal program" in the sense of any program which run afoul of the standard. Thanks for the clarification.

David Abrahams wrote:

Brian Allison writes:

...

...

When a program causes undefined behavior, that falls into the category I'm calling "illegal program." I don't just mean those programs that can be diagnosed as illegal by the compiler.

I misunderstood you - I thought you meant "illegal program" in the sense of any program which run afoul of the standard.

I also consider invoking undefined behavior to be "running afoul of the standard." :)

I realized that with your definition of "illegal program". That would seem to imply that you don't in fact only consider state as an indication of whether or not a program is illegal. On the one hand, it sometimes appears that something which goes outside of the algebra of invariants (forgive me for my imprecision) is your 'illegal program', then you throw in behaviorisms - which have nothing to do with invariants (if I did read you correctly ... <shrug>). Yet you don't allow for the behaviorisms that are specified by the standard in your definition of 'illegal program'. Hm. Perhaps my curiosity->desire to grok Your Standard is no longer on-topic with Library Interface Design, and may even be off-topic for the mailing list. :/

Scott Meyers writes:

I'll note that C++ itself allows "uninitialized" objects with constructors to be created all the time:

That doesn't make it a good practice and general, and...

std::ofstream ofs; std::vector<int>::iterator i;

...these two are examples (although at least the documentation about what that means is rigorous and complete)

std::string s;

...but this is not an example of it

In each case, there are a few operations that can legitimately be performed on such objects, but many operations lead to UB. Is this fundamentally different from the EventLog example? For example, replace EventLog in my example with ofstream, and you have

std::ofstream ofs; ofs << "Hello World";

Trouble ensues, just as it did in the EventLog example.

Yeah, nobody these days claims that the iostreams are an example of stellar, state-of-the-art C++ library design. -- Dave Abrahams Boost Consulting www.boost-consulting.com

- i Scott Meyers wrote:

Brian Allison wrote:

...

As for the advice of the designers of an particular API.... it sounds to me as if your description of their API indicates that you don't consider their API very highly. It sounds (again, from your description) to be hard to use, easy to user to create a bunch of badly working objects, and hard to debug. I do hope that you've got an alternative to that kind of suffering. Or perhaps I misread your description?

I find my intuitions and preferences in tension with advice from others who I believe are worth listening to. This suggests that my intuitions and preferences need refining or that I've misunderstood or misevaluated the advice I find in tension.

Or that the belief you hold about them being worth listening to is not, in fact, properly applied when considering good C++ library design from the user's pouint of view. Remember: you posted this to Boost-users, putting the question of "how important is..." into the context of the users of a library. If I were using a library and I were so easily able to create broken objects, then the first issue I would address is whether or not the library was well-designed. It wouldn't matter how much I liked the guys who wrote the library.

I'm suspicious of a design for a EventLog that seems to require a stream to be useful, yet still allows a EventLog to be created without one, but this seems to be contrary to the advice of a book on how to design an application framework from the people who are responsible for designing one of the most used APIs in existence (.NET). I respect the people in that position, and for those who care about Amazon ratings, it's been well received there (http://www.amazon.com/Framework-Design-Guidelines-Conventions-Development/dp/0321246756/sr=8-1/qid=1158206783/ref=pd_bbs_1/104-6620534-4167149?ie=UTF8&s=books).

Proof by popularity? [The #1 book since Gutenberg has always been The Bible.]

I'll note that C++ itself allows "uninitialized" objects with constructors to be created all the time:

std::ofstream ofs; std::vector<int>::iterator i; std::string s;

Uninitialized objects? The first two are examples of things which don't represent invariants but are interfaces to something else - rarely is it that an interface layer should be used in isolation. [I go a step further and disbelieve that one should ever be *created* in isolation.] The last example is a fine object to create. But we weren't talking about uninitialized objects, we were talking about partial construction, which is a different matter. While the first two are not useable in a default state, they're not meant to ever be created in a *partially constructed* state. Either they're placeholders that are empty and useless, or they're fully assigned and useable.

In each case, there are a few operations that can legitimately be performed on such objects, but many operations lead to UB. Is this fundamentally different from the EventLog example? For example, replace EventLog in my example with ofstream, and you have

std::ofstream ofs; ofs << "Hello World";

Trouble ensues, just as it did in the EventLog example.

Each operation which is valid for the ofstream and iterator is... assignment? I note that neither of them has a partial constructor - which is still the core of the original question, yes? Or perhaps I misread your original poll question? I thought it was about partial construction, not about objects which have an unuseable default state. If instead it was "how do you feel about objects which have an unuseable default state?", then I change my answer to a shrug. After all, I don't use char* unless I have assigned it, and if I can't put off its declaration and can't assign it then I assign to 0. But that's not the same as the antipattern of partial construction. My opinions, as ever, may be ignored at your leisure. They usually are. :) However, they were given in response to your question and not offered out of the blue. regards, Brian

"Yuval Ronen" wrote in message news:eekchf$a8f$1@sea.gmane.org... ...

...

Or perhaps I misread your original poll question? I thought it was about partial construction, not about objects which have an unuseable default state.

If instead it was "how do you feel about objects which have an unuseable default state?", then I change my answer to a shrug. After all, I don't use char* unless I have assigned it, and if I can't put off its declaration and can't assign it then I assign to 0.

But that's not the same as the antipattern of partial construction.

But I don't agree with you here about the difference between "objects with uninitialized state" and "partial construction". I believe they are the same. As long the the constructor doesn't leave the object in 100% constructed state, it doesn't matter if it's 0% or 50%. It's not 100% either way. So I think the default constructor for std::ofstream and std::container::iterator are a defect in the standard (which is unfortunately probably too late to fix).

I agree wholeheartedly.

If an uninitialized state is desired (which is sometimes the case, no doubt about it) then we can all thank God (and Fernando) for Boost.Optional. The usage of optional<> is good for two reasons: ...

You might say something like "but the standard doesn't have optional, so it had to resolve to other means". Maybe so, but that doesn't make those default constructor not-a-defect; it only makes the lack of std::optional an additional defect as well as those default constructors.

The Optional(zero/one) concept also modeled by pointers(albeit with some performance implications dues to heap allocation and indirection), has been available for some time, has it not? So even non-boosted C++ has not had a 'need' for partial or 2 phase constuction. I think std::ofstream, the notions proposed in the book Scott referenced, and in fact most of MFC are all the result of mis-guided early optimization. Jeff

Scott Meyers wrote:

I'll note that C++ itself allows "uninitialized" objects with constructors to be created all the time:

std::ofstream ofs; std::vector<int>::iterator i;

The only time I have found it useful to have default-constructed objects that are otherwise unusable is when I need to store them in a map or other container that requires a default constructible object. That allows me to use the easier operator[] syntax instead of using insert and find. However, I've begun to view that as a defect in the interface of std::map rather than a need to have default constructible objects even when they are not usable. David

"David Abrahams" schrieb im Newsbeitrag news:87bqphhxjl.fsf@pereiro.luannocracy.com...

David Walthall writes:

...

The only time I have found it useful to have default-constructed objects that are otherwise unusable is when I need to store them in a map or other container that requires a default constructible object.

No standard containers have that requirement, though.

This seems to be another defect, as the standard also defines the following constructor: explicit vector(size_type n, const T& value = T(), const Allocator& = Allocator()); Now how is the following code supposed to compile: #include <vector> struct X { X( int ) {} }; void f() { // Error: No default constructor for X. std::vector<X> v( 5 ); } -- Matthias Hofmann Anvil-Soft, CEO http://www.anvil-soft.com - The Creators of Toilet Tycoon http://www.anvil-soft.de - Die Macher des Klomanagers

On 9/12/06, loufoque wrote:

Scott Meyers wrote :

...

...

An example: System.Data.SqlClient.SqlParameter is a class that describes a bound parameter used in a database statement. Bound parameters are essential to prevent SQL injection attacks. They should be exceedingly easy to use since the "competition" (string concatenation of parameters into the SQL statement) is easy, well understood, and dangerous.

You can construct safe SQL queries with streams or printf-like syntax easily

id = "2 ; delete from persons ;" sql << "select first_name, last_name, date_of_birth "

"from persons where id = " << id

Someone just deleted your persons table. Oops. No need to put objects everywhere that complexify everything. _______________________________________________

Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Roman Neuhauser wrote:

I'd like to see the rationale. What's the benefit?

This is excerpted from pp. 19-21, where I'm not bothering to include ellipses to show where I've elided information. So this is a set of their words (mostly full sentences) knitted together to try to show their arguments, but it doesn't show all their text, so for the full story, you need to consult the book. [Begin pseudoquote] Many developers expect to learn the basics of a new framework very quickly. By experimenting with the framework on an ad hoc basis. The initial encounter with a badly designed API can leave a lasting impression of complexity and discourage some from using the framework. This is why it is very important for frameworks to provide a very low barrier for developers who just want to experiment. Many developers experiment with an API to discover what it does and then adjust their code to get their program to do what they really want. There are several requirements that APIs must meet to be easy to experiment with: - It has to be easy to start using an API, regardless of whether it does what the developer wants it to do. A framework that requires an extensive initialization or instantiating several types and hooking them together is not easy to experiment with. - It has to be easy to find and fix mistakes resulting from incorrect usage of an API. For example, APIs should throw exceptions clearly describing what needs to be done to fix the problems. Scott

Scott Meyers writes:

- It has to be easy to start using an API, regardless of whether it does what the developer wants it to do. A framework that requires an extensive initialization or instantiating several types and hooking them together is not easy to experiment with.

How hard is it to set up a new stringstream for testing a component. Or, heck, a file stream? -- Dave Abrahams Boost Consulting www.boost-consulting.com

Most large softwares have test event drivers that that allow some functions/interfaces be called only for debug enviromnets. and in the code we have constructs like DBG_ASSERT() regards divyank --- Scott Meyers wrote:

...

In a case like the example, where I know it might be hard to setup the object correctly, I'd rather have a constructor

Rush Manbert wrote: that took some bogus

...

argument type that set the object up in "simulated success" mode, but only in my debug build.

I'm glad you mentioned this, because my recent knowledge of unit testing comes from books like Beck's "Test-Driven Development" and Feathers' "Working Effectively with Legacy Code" as well as countless breathless articles lauding the wonder of unit testing and showing how it's all oh-so-simple. My feeling is that it would often be convenient to have a special "test" interface, but I've never seen any discussion of doing that. I can imagine a couple of ways of doing this, one being to literally have a different interface for debug builds, another to have "test only" interface elements cordoned off somewhere by convention (similar to Boost's use of namespaces named "detail").

...

EventLog::EventLog (bool dummyarg) { // This constructor sets us up in simulated success mode. #ifdef DEBUG // or whatever you use m_simulateSuccess = true; #else throw something useful #endif }

Hmmm, I'd think this entire constructor would exist only in debug builds, e.g.,

class EventLog { public: #ifdef DEBUG EventLog(bool dummyarg); #endif

Unfortunately, conditional compilation is not without its own resultant spaghetti code and concomitant maintenance headaches.

Scott

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org

http://lists.boost.org/mailman/listinfo.cgi/boost-users

__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

David Abrahams wrote:

I'd much rather develop a library of mock objects for testing. Okay a conforming mock stream may be a little work to write, but you write it once and you're done.

I take this to mean that you have not found it necessary/desirable/useful to create separate test and client interfaces. Okay. My understanding is that the ability to drop in mock objects requires programming to an interface that can be either a "real" object or a mock object, which in turn suggests using either a template parameter or a base class. Suppose, for example, you have this: class BigHonkinHairyClass { ... // expensive to construct }; and you want to implement some function f that takes a BigHonkinHairyClass as a parameter. The straightforward declaration would be void f(BigHonkinHairyClass& bhhc); // maybe const, doesn't matter But now it's hard to drop in a mock. So I assume you'd modify the interface to be either template<typename T> void f(T& bhhc); // T must more or less model // BigHonkinHairyClass or this: class BigHonkinHairyBase { ... // interface to program against -- }; // uses virtuals class BigHonkinHairyClass: public BigHonkinHairyBase { ... }; class MockHonkinHairyClass: public BigHonkinHairyBase { ... }; void f(BigHonkinHairyBase& bhhc); Is that correct? In other words, you'd come up with an interface that let clients do what they wanted but that was also mock-friendly for testing purposes? That is, you'd take testability into account when designing your client interface? Thanks, Scott

Bill Lear wrote:

This is essentially the approach I have taken over the years. At some point, the unit tests become hopelessly complex, Mock objects begin to weigh development down (if you change the interface, you have to change your mock objects) and a transition is made to higher-level integrated testing.

I assume you always need integrated testing in addition to unit tests, but it sounds like you're saying that at some point, maintaining the unit test framework stops paying for itself, so you abandon it. Is that correct? Scott

Scott Meyers wrote:

David Abrahams wrote:

...

I'd much rather develop a library of mock objects for testing. Okay a conforming mock stream may be a little work to write, but you write it once and you're done.

I take this to mean that you have not found it necessary/desirable/useful to create separate test and client interfaces. Okay.

My understanding is that the ability to drop in mock objects requires programming to an interface that can be either a "real" object or a mock object, which in turn suggests using either a template parameter or a base class. Suppose, for example, you have this:

class BigHonkinHairyClass { ... // expensive to construct };

and you want to implement some function f that takes a BigHonkinHairyClass as a parameter. The straightforward declaration would be

void f(BigHonkinHairyClass& bhhc); // maybe const, doesn't matter

But now it's hard to drop in a mock. So I assume you'd modify the interface to be either

template<typename T> void f(T& bhhc); // T must more or less model // BigHonkinHairyClass

or this:

class BigHonkinHairyBase { ... // interface to program against -- }; // uses virtuals

class BigHonkinHairyClass: public BigHonkinHairyBase { ... };

class MockHonkinHairyClass: public BigHonkinHairyBase { ... };

void f(BigHonkinHairyBase& bhhc);

Is that correct? In other words, you'd come up with an interface that let clients do what they wanted but that was also mock-friendly for testing purposes? That is, you'd take testability into account when designing your client interface?

That's about what I've found best when building well-tested code, and it fortunately usually also leads to cleaner designs with little coupling that compile nice and quickly even given current slow C++ compilers. -- James

Scott Meyers wrote:

David Abrahams wrote:

...

I'd much rather develop a library of mock objects for testing. Okay a conforming mock stream may be a little work to write, but you write it once and you're done.

I take this to mean that you have not found it necessary/desirable/useful to create separate test and client interfaces. Okay.

My understanding is that the ability to drop in mock objects requires programming to an interface that can be either a "real" object or a mock object, which in turn suggests using either a template parameter or a base class. Suppose, for example, you have this:

class BigHonkinHairyClass { ... // expensive to construct };

and you want to implement some function f that takes a BigHonkinHairyClass as a parameter. The straightforward declaration would be

void f(BigHonkinHairyClass& bhhc); // maybe const, doesn't matter

But now it's hard to drop in a mock. So I assume you'd modify the interface to be either

template<typename T> void f(T& bhhc); // T must more or less model // BigHonkinHairyClass

or this:

class BigHonkinHairyBase { ... // interface to program against -- }; // uses virtuals

class BigHonkinHairyClass: public BigHonkinHairyBase { ... };

class MockHonkinHairyClass: public BigHonkinHairyBase { ... };

void f(BigHonkinHairyBase& bhhc);

Is that correct? In other words, you'd come up with an interface that let clients do what they wanted but that was also mock-friendly for testing purposes? That is, you'd take testability into account when designing your client interface?

This situation should be rare in well-designed code. Either f requires a BigHonkingHairyClass in order to work - by this I mean the precise semantics of BHHC - and it has to be tested using a BHHC; you can't substitute a mock that emulates a BHHC perfectly because it will be a BHHC itself. Or, f doesn't really require a BHHC, and it should be rewritten in one of the two ways above. This allows client A to pass a BHHC and client B to pass something else. From the library design PoV, it doesn't matter whether client B is a test suite or just another module. It is true that in many projects, the lower layers aren't designed as a proper library, since they don't have to serve arbitrary client code. In such a case, having a test suite as a second client can indeed lead to problems like the above. :-) Another angle is that tests should test the behavior that is exercised by the application. If the application uses f with a BHHC, the tests should test f with a BHHC. Testing f with a mock can find some errors, but may easily miss others. This can be substituted by defining a rigorous interface for BHHCs and testing both f and BHHC against that interface, of course, which gets us back to one of the two refactorings given above. (But the f+actual BHHC test should still be part of the suite, IMO.)

Scott Meyers wrote:

Rush Manbert wrote:

...

In a case like the example, where I know it might be hard to setup the object correctly, I'd rather have a constructor that took some bogus argument type that set the object up in "simulated success" mode, but only in my debug build.

I'm glad you mentioned this, because my recent knowledge of unit testing comes from books like Beck's "Test-Driven Development" and Feathers' "Working Effectively with Legacy Code" as well as countless breathless articles lauding the wonder of unit testing and showing how it's all oh-so-simple.

LOL - "oh-so-simple" breaks down pretty quickly in any large system, doesn't it? My feeling is that it would often be convenient to have a

special "test" interface, but I've never seen any discussion of doing that. I can imagine a couple of ways of doing this, one being to literally have a different interface for debug builds, another to have "test only" interface elements cordoned off somewhere by convention (similar to Boost's use of namespaces named "detail").

I have done this in the past. Here's a real world example: The application was for a storage virtualization controller. There were two separate processors running different software. One handled the virtualization and client services, while the other actually knew how to do I/O to the storage devices. My subsystem was a service that did copies of various flavors (entire logical units, etc.). Since it couldn't actually do any I/O, it could send messages to the other processor that created, started, stopped, etc. a copy utility task. In the real case, the utility would send events to my service to let it know how many blocks had been copied, if an error occurred, etc. There could be many instances of copy processes/utilities running concurrently and independently. I had to test the top level service without any support from the copy utility task on the other processor. (You may wonder why. Let's just say that there were two separate groups that developed software for the different processors, and we had somewhat different ideas regarding testability. My group also had a version of our code that ran on Windows and I needed to be able to test it there.) This meant that I needed to simulate the event stream from the utility task. I also had to be able to force errors (again by simulating events). Needless to say, this required a fairly extensive testing API, plus a notion within the objects that implemented the service that it could be running in "test mode". I think it took as much or more work to develop the test interface and the test drivers as it took to develop the service itself, but it was completely worth it. Especially when there was a problem and we needed to sort out whether it was "our" code or "their" code. ;-) Also, since I had this capability, the Windows version of the service could be driven by our management UI, and the copy processes would appear to make progress, stop, start, and complete. The management UI couldn't tell that they were "fake", so that group could use it to test their software. It was really cool, but the reasons that it was even possible was that a testing subsystem was built into our code, and we were required to have complete tests for our subsystems, and I had to consider how to test my subsystem from day one. In fact, our system shipped with the test subsystem included. It was not readily accessible, of course, but was really useful in some cases where we needed to test something on an installed system. This sort of capability in the field can be a real saving grace in an embedded system.

...

EventLog::EventLog (bool dummyarg) { // This constructor sets us up in simulated success mode. #ifdef DEBUG // or whatever you use m_simulateSuccess = true; #else throw something useful #endif }

Hmmm, I'd think this entire constructor would exist only in debug builds, e.g.,

class EventLog { public: #ifdef DEBUG EventLog(bool dummyarg); #endif

Unfortunately, conditional compilation is not without its own resultant spaghetti code and concomitant maintenance headaches.

Oh, sure. Get the error at compile time if you're going to go this way. The conditional compilation stuff is a definite problem to be avoided if possible. It's hard to avoid if you want to implement a really comprehensive test interface, but does seem less desirable in order to support "exploratory" programming (which I think was the original context here). - Rush

Scott Meyers wrote:

Rush Manbert wrote:

...

Needless to say, this required a fairly extensive testing API, plus a notion within the objects that implemented the service that it could be running in "test mode".

Was the ability to put an object into test mode part of the client-visible API, or did you somehow have a testing API that clients could not access?

In this case, the only way to interact with the service was through a message interface. In essence, the service was passed a message object. It was also an embedded system, so we were not only the service provider, but we also wrote the client. (I realize that we have strayed from the original question of library design. I'm happy to take this off list if anyone is offended.) The messages that could put the object in test mode were part of the client visible API. Additionally, the service always operated in test mode when it was built as part of the Windows executable, because then there was no choice but to simulate the events that were normally generated by the lower level code. This is why the management UI (our client) always thought that a copy process was real. It couldn't tell the difference between the real thing and the test mode behavior.

...

In fact, our system shipped with the test subsystem included. It was not readily accessible, of course, but was really useful in some cases where we needed to test something on an installed system. This sort of capability in the field can be a real saving grace in an embedded system.

This makes it sound like the testing API was in fact visible to clients, but, by convention, it was never used for non-test apps. Is this correct? If so, that's different from, for example, test-only APIs that exist only for debug builds, i.e., a truly separate API that non-test clients can't get at. (I'm not arguing that such an approach is better than what you did, I'm simply trying to understand what you did.)

To clarify my interest, I recently sent this to somebody who send me private mail on this thread:

...

My interest here is not in testing, it's in good design, and good designs facilitate testing. Which means we need to be able to describe how testability affects other design desiderata, such as compile-time error detection, encapsulation, and overly general interfaces. There is a ton of recent literature on testing and testability, but virtually none of it addresses how making something more testable may be in tension with other characteristics we'd like. This thread in Boost is part of my attempt to figure out how the various pieces of the puzzle fit together -- or if they do at all.

I'm not sure how applicable this is to design of a library such as Boost, but my experience has been that having test interfaces available at the subsystem level in the release code version is a very useful thing, even if they are visible to clients. You need to be careful with this, and you really need to protect clients from getting into test mode accidentally, but when you are debugging a large, complex system that may have very limited debugging capabilities (I spent 25 years developing embedded systems, so that's where this viewpoint comes from) it can be very very useful to be able to isolate your subsystems. You can usually only do that if they have been designed so that they can be tested in isolation. I also believe that you often need these sorts of interfaces so that you can force error conditions, especially in a heavily layered system. This lets me test that my subsystem handles errors correctly, but it also allows me to test the error propagation paths out of my subsystem and into the layers above it. I made a little Xcode project that illustrates one way you could approach making objects that have a test mode. I have attached the code and header files to this email. In this case, MyObject has two constructors, each of which takes an initialization object as an argument. One of them takes a "normal" initializer object, while the other takes a "test" initializer. The object is in test mode if you construct it with the second form. There is also a public method that can force a test mode behavior, but only for a test mode object. So my test API is visible to clients. However, if I don't distribute MyObjectInitializerForTest.h, then clients cannot construct a test initializer object, and therefore can't put the object in test mode. Of course, they can see how MyObjectInitializer was declared, so they could figure out how to declare and define a MyObjectInitializerForTest object, but it seems to me that the barrier is sufficiently high that there won't be much of that going on. I took this approach in order to mimic the original case that I described. In that case, the initializer objects were the "normal" or "test" messages. I know that there are slicker ways to do this sort of thing, but this illustrates the basic idea. - Rush /* * MyObjectInitializerForTest.h * */ #ifndef MyObjectInitializerForTest_H #define MyObjectInitializerForTest_H #include "MyObjectInitializerBase.h" class MyObjectInitializerForTest: public MyObjectInitializerBase { public: MyObjectInitializerForTest (int a, bool b) : MyObjectInitializerBase (a,b) {}; ~MyObjectInitializerForTest (void) {}; }; #endif //MyObjectInitializerForTest_H /* * MyObject.cpp * */ #include <iostream> #include "myObject.h" #include "myObjectInitializerForTest.h" MyObject::MyObject (MyObjectInitializer const initializer) : m_testMode (false) , m_a (initializer.m_a) , m_b (initializer.m_b) { testingMemberDataInit (); } MyObject::MyObject (MyObjectInitializerForTest const initializer) : m_testMode (true) , m_a (initializer.m_a) , m_b (initializer.m_b) { testingMemberDataInit (); } MyObject::~MyObject (void) { return; } bool MyObject::methodWithTestModeBehavior (void) { if (m_testMode) { std::cout << "MyObject::methodWithTestModeBehavior: test mode is enabled!"; if (m_doForceResultOfCallToMethodWithTestModeBehavior) { // The return value for this call is forced this time only std::cout << " Forced return value: " << (m_forcedResultOfCallToMethodWithTestModeBehavior ? "true" : "false") << "\n"; m_doForceResultOfCallToMethodWithTestModeBehavior = false; return m_forcedResultOfCallToMethodWithTestModeBehavior; } std::cout << "\n"; } else { std::cout << "MyObject::methodWithTestModeBehavior: test mode is DISABLED.\n"; } // Normal return here - whatever m_b contains return m_b; } void MyObject::forTestingSetResultOfNextCallToMethodWithTestBehavior (bool result) { if (m_testMode) { m_doForceResultOfCallToMethodWithTestModeBehavior = true; m_forcedResultOfCallToMethodWithTestModeBehavior = result; } else { std::cout << "MyObject::forTestingSetResultOfNextCallToMethodWithTestBehavior: Ignored in normal mode!\n"; } } void MyObject::testingMemberDataInit (void) { m_doForceResultOfCallToMethodWithTestModeBehavior = false; m_forcedResultOfCallToMethodWithTestModeBehavior = false; } #include <iostream> #include "MyObject.h" #include "MyObjectInitializerForTest.h" int main (int argc, char * const argv[]) { // insert code here... std::cout << "Hello, World!\n\n"; MyObject normalModeObj (MyObjectInitializer::MyObjectInitializer (1,true)); MyObject testModeObject (MyObjectInitializerForTest::MyObjectInitializerForTest (2, false)); bool result; result = normalModeObj.methodWithTestModeBehavior(); std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n"; result = testModeObject.methodWithTestModeBehavior(); std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n"; // Try to force next return value on normalModeObject (fails) normalModeObj.forTestingSetResultOfNextCallToMethodWithTestBehavior(false); result = normalModeObj.methodWithTestModeBehavior(); std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n"; // Force retuurn value on testModeObject for the next call to methodWithTestModeBehavior() testModeObject.forTestingSetResultOfNextCallToMethodWithTestBehavior(true); result = testModeObject.methodWithTestModeBehavior(); std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n"; result = testModeObject.methodWithTestModeBehavior(); std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n"; return 0; } /* * MyObjectInitializerBase.h * */ #ifndef MyObjectInitializerBase_H #define MyObjectInitializerBase_H class MyObjectInitializerBase { public: MyObjectInitializerBase (int a, bool b) : m_a (a), m_b(b) {}; ~MyObjectInitializerBase (void) {}; int m_a; int m_b; }; #endif //MyObjectInitializerBase_H /* * MyObject.h * */ #ifndef MyObject_H #define MyObject_H #include "MyObjectInitializerBase.h" class MyObjectInitializer: public MyObjectInitializerBase { public: MyObjectInitializer (int a, bool b) : MyObjectInitializerBase (a,b) {}; ~MyObjectInitializer (void) {}; }; class MyObjectInitializerForTest; class MyObject { public: // Normal constructor MyObject (MyObjectInitializer const initializer); // Test Mode constructor MyObject (MyObjectInitializerForTest const initializer); ~MyObject (void); bool methodWithTestModeBehavior (void); // Testing API void forTestingSetResultOfNextCallToMethodWithTestBehavior (bool result); private: void testingMemberDataInit (void); bool m_testMode; int m_a; bool m_b; // Member data used to control testing bool m_doForceResultOfCallToMethodWithTestModeBehavior; bool m_forcedResultOfCallToMethodWithTestModeBehavior; }; #endif //MyObject_H

Jeff Garland wrote:

Testing in total isolation is a myth. To be a little softer -- it's really going to depend on the type of class you are testing whether or not it can be rationally tested in isolation. If you haven't lately, you should re-read Lakos's treatment of this subject in Large Scale C++ Software Design. This book is 10 years old, but he breaks down testability in a way I've not seen anyone else do since doing testing became all the rage. Most of the 'test first' stuff I've seem ignores the inherent untestability of some software.

That's been my impression. One of the things I've been trying to figure out wrt the whole testing hoopla is how well it translates to large projects and how it has to be adjusted when things move beyond toy examples. And yes, I probably should go back and reread Lakos. Scott

Jeff Garland writes:

Well, the testing hoopla 'applies' to the extent that in my experience big systems that *don't* have significant testing discipline never see the light of day. That is, they fail under an avalanche of integration and basic execution problems before ever being fielded. As an aside, I always get a

<snip great post> Great post, Jeff! I learned a lot from it; thanks. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Joel de Guzman writes:

One-phase construction definitely!

I'd like to ask you a question about your parser framework that is related to this. I probably get most of the following wrong, so please be patient. More often than not, a parser constructs objects in a hierarchical manner that reflects the grammar: print(a + b); might be expressed as statement(plus_expression(symbol("a"), symbol("b"))) in some language. The latter objects are better be designed without default constructors, in order to guarantee that we never end up with, say, a meaningless symbol() or plus_expression(). When I write a parser by hand, that's straight forward: boost::optional< plus_expression > parse_plus_expression(input_tokens tokens) { ... } The function will return boost::none if the expression can't be parsed and no default plus_expressions will be necessary. With spirit, there appear to be two approaches: 1) semantic actions, which store away the parsed expression in some way and 2) closures, which implicitly "return" their first member (member1). I don't like 1 for most purposes, because it's too imperative for my taste. I need to keep track of what my actions did and, if something fails to parse at a point and backtracking occurs, need to manually revert the changes. This seems error prone to me. 2 looks better, but, and this is the connection to the OP, it appears that the returned value, as all closure members, must be default constructible. In particular, I risk returning such default constructed values when I failed to assign to them. It's very likely that I missed something, as I didn't seriously tried using spirit. Can you shed some light on this? Jens

Joel de Guzman writes:

Well, the proper mailing list is: https://lists.sourceforge.net/lists/listinfo/spirit-general

It also fits very well into this thread.

Yeah. You did miss something. The closure members can be initialized. See "Initializing closure variables" section in http://tinyurl.com/85hbq

And, to stick with my example, what would be a good initialisation for a plus_expression? You don't want to replace default construction with a copy from what's conceptionally a default. The topic question of this thread is about whether or not you should construct objects prior to when you have enough information to meaningfully do so, and for returning parsed values in spirit, this is neither at closure contstruction time nor at rule/grammar invocation time. I have the impression that spirit either forces you to do exactly that or else use actions, which is even more messy. Am I correct in that observation? Best regards, Jens

David Abrahams writes:

I may totally misinterpreting the question here, but in case I'm not...

And I might be misinterpreting the answer, but we probably mean different things. You're talking about the rules themselves, are you? What I want not to be default-constructed is the value that is constructed by the rule/grammar. The following is taken from the documentation: factor = ureal_p[factor.val = arg1] | '(' >> expression[factor.val = arg1] >> ')' | ('-' >> factor[factor.val = -arg1]) | ('+' >> factor[factor.val = arg1]) ; I'm not complaining that `factor' must have been default constructed. I'm worrying about the return value, which is represented by the `val' closure member. The point where we know what to return is where the assignments are, but by design we must have some value earlier than that. This piece of code is analogous to: optional< value_t > factor(tokens_t tokens) { value_t ret; if(optional< value_t > temp = ureal_p(tokens)) ret = temp; else if(optional< value_t > temp = expression(tokens)) ret = temp; else ... return ret; } where one would like to have optional< value_t > factor(tokens_t tokens) { if(optional< value_t > temp = ureal_p(tokens)) return temp; else if(optional< value_t > temp = expression(tokens)) return temp; else ... } What if value_t is not an int in a calculator example, but, say, a `binary_expression' object for some programming language parser? Any default is clearly completely bogus, and I will have to clutter my program with sanity checks that make sure that I really have a proper object, as I expect. Best regards, Jens

David Abrahams wrote:

Jens Theisen writes:

...

I'm not complaining that `factor' must have been default constructed. I'm worrying about the return value, which is represented by the `val' closure member. The point where we know what to return is where the assignments are, but by design we must have some value earlier than that.

Ah, I understand.

I think Jens got it incorrectly. The var will *not* be created at all on an unsuccessful match. No, the value is not created prior to entering the rule. The value is created *lazily* after, and only after, a successful match is made. That's the beauty of lazy evaluation. Don't be misled by the syntax. On a no-match, the result is an optional<T>(), like in your example. See match class in match.hpp and notice the optional_type val; that's your attribute. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

Jens Theisen wrote:

Joel de Guzman writes:

...

I think Jens got it incorrectly. The var will *not* be created at all on an unsuccessful match.

Which is cool, but misses my point. Take the example again, but this time I have made a mistake:

[...]

How to do it with spirit?

I think this is going off topic from the thread. I suggest we continue this in the Spirit list: https://lists.sourceforge.net/lists/listinfo/spirit-general Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

Joel de Guzman wrote:

One-phase construction definitely! Testing in isolation is a different matter and should not degrade the interfaces. There are ways to allow isolated testing without degrading the interface. It all boils down to decoupling and isolating dependencies. My favorites:

1) Use a template where policies can be replaced by hooks to the testing engine that tests the expected results. In your example, I'd imagine this interface: template <class Printer> EventLog.

Or the more conventional OO approach of subclassing from ostream (in the example) and passing in a null or mock derived class object for testing purposes. Your approach has the drawback that it's now more difficult to have a container of all EventLog objects (because Printer is part of the type) and the OO approach has the drawback of requiring the introduction of a base class and virtuals in cases where they might otherwise not be necessary. To modify the example, suppose the EventLog constructor requires a Widget, and Widget is a large nonpolymorphic object with no virtuals. I'd still pass the Widget by reference, but subclassing it for testing would be ineffective, due to the lack of virtuals. Either way the desire to make the class testable affects the interface seen by users. This is not a complaint, just an observation. In another post, I noted that it seems like it'd be nice to be able to somehow create a "testing only" interface separate from the "normal" client interface.

2) Use callbacks. In the example you provided, I'd imagine EventLog calls logstream to print. So, I'd use a constructor like: EventLog::EventLog(boost::function print) instead. So, instead of calling logstream << stuff directly, I'll call print(stuff). For the testing engine, I'll replace it with something that tests the expected results.

All these falls under the "Hollywood Principle: Don't call us, we'll call you". IMO, with proper design, you can have both single phase construction *and* isolation testing.

But can you also have maximal inlining and, where needed by clients, runtime polymorphism? Templates preserve inlining but tend to sacrifice polymorphism (e.g., it's hard to have a container of (smart) pointers to EventLog<T> objects for all possible Ts), while base class interfaces and callbacks preserve polymorphism at the expense of easy inlining. Scott

Edward Diener writes:

Johan Nilsson wrote:

...

...

and seem to contradict the advice of the designers of the .NET API.

Different platform and philosophy, unless you're talking about C++/CLI.

C++/CLI is not an exception. C++/CLI follows the .Net philosophy and rules regarding the construction of objects, not the C++ philosophy and rules, as I pointed out in another post on this thread. Because of that it is not necessarily apropos to use .Net as a basis for discussion of constructor philosophy in C++, although .Net's reliance on properties and events does have an anology to work that has been done with C++ lately, and does affect how one thinks about constructing and using objects.

C++/CLI's default null-initialization of references doesn't in any way justify designs that pass out half-baked objects to users, any more than we would approve of passing out half-baked objects containing only shared_ptr<>s in C++. It's bad design. It makes the client of a class responsible for things that the class designer should have taken care of. That's a universal no-no, regardless of what language the code is written in. Default null-initialization of references does make it easier to not think about certain exception-safety and object state issues and "get away with it" some of the time, but that doesn't make code written that way correct. Usually, it just means that bugs may be masked or harder to detect. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Edward Diener writes:

Most .Net objects aren't half-baked objects to use, but they are objects which are created with a default constructor, which gives people the idea that they are not ready to use. They rely quite a bit on properties being set in order to use, and those properties are set via the design-time interface in Visual Studio, which then injects code in the default constructor to set the properties. So while it looks like these objects are not ready to use, they really are.

Edward, I know little of .Net, so I'd appreciate it if you could help me out here. When you say "design-time interface," what are you talking about? Some kind of GUI? Are you saying that this GUI modifies the compiled binary, leaving no textual trace of member initialization values in the original source for the class? [for what it's worth, I don't think the answers to this question can affect my stance on Scott's question at all. Either the objects are fully-baked, or not, upon construction. How they come to be baked, and whether this "design-time interface" thing is a good idea, are separate issues] -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:

Edward Diener writes:

...

Most .Net objects aren't half-baked objects to use, but they are objects which are created with a default constructor, which gives people the idea that they are not ready to use. They rely quite a bit on properties being set in order to use, and those properties are set via the design-time interface in Visual Studio, which then injects code in the default constructor to set the properties. So while it looks like these objects are not ready to use, they really are.

Edward, I know little of .Net, so I'd appreciate it if you could help me out here. When you say "design-time interface," what are you talking about? Some kind of GUI?

Yes. The Visual Studio designer allows one to drop components into other components and forms, and modifies the source code constructors which create the component and/or form. A component is also a component container, allowing other components to be nested inside it, and a form is the equivalent of a Windows window which naturally can have components in it. A component is a particular kind of class and controls, which people are used to seeing in forms ( windows ), are just visual components.

Are you saying that this GUI modifies the compiled binary, leaving no textual trace of member initialization values in the original source for the class?

The original source. The Visual Studio designer is actually modifying the constructors to setup the components in the source code and then this gets compiled into the resulting binary. The areas being modified are marked off in the source so that the user does not change them but allows the Visual Studio designer do it instead. The actual way it does this is better than it sounds. When you first create a component and/or form from menus in Visual Studio, the constructors get an InitializeComponents(); call added to them with the appropriate definition later in the source code. It is in this InitializeComponents() definition that the Visual Studio Designer manipulates code based on the properties and events of components added to a component and/or a form. There are two constructors, a default one which takes no parameters and an other one which is passed a container and in which codes gets added automatically which adds the component to the container.

[for what it's worth, I don't think the answers to this question can affect my stance on Scott's question at all. Either the objects are fully-baked, or not, upon construction. How they come to be baked, and whether this "design-time interface" thing is a good idea, are separate issues]

I agree with your analysis. I was only trying to point out that the default constructor ( or container constructor ) of .Net was erroneously causing others, including the OP, to think that .Net creates classes which are not ready to be used upon construction. They are, by and large, ready to use immediately because the necessary code to setup their properties and events has already been injected into the constructors when the programmer manipulated a form or a component in the visual designer. So having someone point out to the OP that .Net is an example of an environment in which two-phase construction is the rule is just plain wrong, and any inference about software design of classes following this idiom should not be made from that erroneous conclusion.

Yuval Ronen wrote:

As I explained in another post in this thread, I think boost::optional is the ultimate solution to the active/inactive state problem, so the class itself can still have single phase construction.

There is a lot about boost::optional that I am not entirely happy about, but we dont really want to get into that on this thread, however the general idea of using optional or a similar facility is still very interesting. I imagine that the idea would be to refactor the current class with active/inactive state into two separate classes. The long lived object (needed for complex structural reasons) that would operate as normal, but all the active-dependent interface would be moved to the new/separate class and accessed through a member function returning an optional<T> (or similar). I can see where this would make excellent sense in several of my applications. To optimise access inside tight loops you could assign the optional<T> to a T& - and so only be checking existence at the start... as opposed to my present problem of having to choose between assertion or exception checking at the top of each top-level class function. Yes, I like the thought very much. Thank you. -- Geoff Worboys Telesis Computing