[variant2] Andrzej's review -- design
Hi Everyone, First of all, I wanted to thank Peter for writing this non-trivial library and sharing it with the community. This is not a full review yet. I just wanted to discuss the design first, especially the never-empty guarantee. And I am going to rehash the old argument. The core of variant2 is its never-empty guarantee and the mechanism to achieve it: a number of different algorithms chosen depending on what the potentially stored types permit. In fact this mechanism is not much different than the one present in boost::variant. The issue to solve is what happens in copy/move assignment of variant<X, Y> when we have to change the stored type from X to Y. X has to be destroyed and Y initialized. But if the initialization of Y throws we would be left without either X (already destroyed) or Y (not successfully constructed). (The same problem applies to member function emplace()). Except for the last step boost::variant and variant2 try to apply the same sequence: 1. If all the subtypes have noexcept initialization, no safety measures need to be taken. 2. If sub-types have noexcept move constructor, a temporary local storage is used and the backup value is moved to and from it. 3. If one of the subtypes is a special type boost::blank or std::monostate respectively, in case of a throw this type is default constructed in the variant. 4. If there is any nothrow-default-constructible subtype in the variant, it gets default-constructed upon exception. If neither of the above can be applied, the two libraries apply different approaches. variant2 is big enough to fit two unions of X and Y within its storage. This way it does not have to destroy the X when initializing the Y. boost::variant, for the additional storage uses heap allocated memory. Interestingly, the double storage as in variant2 has been considered in the initial design of boost::variant but rejected by the community, because it added spacial overhead even for use cases that never invoked variant's assignment. This is described in the Rationale section of boost::variant documentation: https://www.boost.org/doc/libs/1_69_0/doc/html/variant/design.html#variant.d... In boost.variant, only these programs pay for the overhead that really invoke the assignment operator. Neither of the solutions seems strictly superior than the other. The double storage could be thought as better because it seems to be able to cover use cases where heap allocations are banned altogether. On the other hand, in these applications it is difficult to imagine that X and Y would be throwing exceptions in move constructors or during initialization. One inferiority of boost::variant is that it can fail for its own reasons unrelated to failures of X and Y. Even if no operations on X and Y throw during the assignment of variant, the variant can still throw because it allocates memory. Some people believe that an application cannot recover from bad_alloc anyway, but this is not a universally held view. To me, the important question is, do we really need the never-empty guarantee. It was an unquestionable requirement for the initial design of boost::variant. Now, variant2 is advertised as an allegedly superior solution to std::variant, because the latter does not provide the never-empty guarantee, even though the cost of providing it is significant. But I myself fail to see the *practical* value of the never-empty guarantee. The argument I hear is that, we want to make it an invariant that either X or Y is stored in variant. I agree with this goal in general, but the costs and contortions required to satisfy it outweigh the benefit. Now, I hear that if the invariant is weakened everyone will have to check everywhere if any variant is in a valueless_by_exception state. This seems to be confirmed by the semantics of std::variant::visit(), which requires that valueless_by_exception is checked and an exception thrown on valueless. This last part is indeed strange in std::variant, but I do not agree that the fix is to provide a never-empty guarantee at the cost of compromising run-time performance and complicating the design. I can think of the following alternative design. The invariant of a variant (pun intended) allows for a valueless_by_exception state. There is no way to get to this state other than by throwing form the initialization of Y in variant's assignment or .emplace() operation. Calling .visit() on a valueless variant is UB. The destructor and reset functions such as assignment and .emplace() can handle the valueless state. This is almost what std::variant does except for UB in .visit(). Why I find this superior to variant2. 1. It is simpler, faster and smaller. 2. Observing the valueless state can only happen when somebody is handling exceptions incorrectly. And in that case this is exception handling strategy that needs to be fixed: not the variant's guarantees. It should be noted that variant2 DOES NOT PROVIDE A STRONG EXCEPTION SAFETY GUARANTEE. The never-empty guarantee has a certain negative effect of confusing people so that they are led to believe that they have strong exception safety. But they don't. even expert programmers in this forum expressed their surprise that changing from X to Y in variant<X, Y, Z> can get them a variant storing Z. Sometimes it is even trickier: X may be a container that currently stores 10 elements. You assign Y, and after the exception you get an X that stores zero elements. Because of these problems, the only reasonable thing to do is to follow the advice from Dave Abraham's article: https://www.boost.org/community/exception_safety.html To quote the relevant part, "If a component has many valid states, after an exception we have no idea what state the component is in; only that the state is valid. The options for recovery in this case are limited: either destruction or resetting the component to some known state before further use." If this principle is followed no-one will ever observe the valueless_by_exception state. This also applies to variant2. If this principle from Dave Abrahams is followed, no-one will ever benefit from the never-empty guarantee (but people will still pay the costs of providing it). You cannot do anything meaningful with variant that threw other than destroy or reset it. (But it is probably best to destroy it.) If objects are allowed to survive the stack unwinding, they should either offer a strong exception safety guarantee, or you have to have some very detailed knowledge of a given particular type how its state gets modified under exceptions, but this is usually very difficult and best left to experts. I have started a thread in this list a while ago, requesting for an example of code that *correctly handles exceptions* (does not stop stack unwinding at random places), where the programmer would make use of the never-empty guarantee , and chose something else than destroying or resetting the variant. And although I received some generic statements, referring to the purity of the design, strength of the invariants, and the easiness of thinking or "correctness", none of the proponents of the never-empty guarantee gave such an example. Which might be an indication that the problem we are solving may be nonexistent. I am not sure of this claim, but I have not seen any convincing example either. I believe that we are facing one of this problems where the strong invariants of the type need to be weakened due to practical considerations. I can see no correctness issues with valueless state other than in programs that are already exception-unsafe. Also, I have heard claims that if someone is fine with valueless state they should put std::monostate as one of the types and the same effect without double buffering is achieved. But this is not true. Variant with monostate can get the degenerate state in a correct program; often: in the variant constructor. Now I really have to special-case it whenever I inspect the variant object, which is a nightmare: this is really a weak invariant in an allegedly strong-invariant variant. In contrast, for std::variant you only get the valueless state in the program that incorrectly handles exceptions: no need to special-case the valueless_state when doing visitation.. variant2 is not a replacement for std::variant, as the choice to support never-empty guarantee at the cost of more expensive operations and increased object size is not necessarily superior. Even if my view is not generally held, it has to be acknowledged that the issue is controversial. Variant2 cannot replace boost::variant either. This is not only about supporting pre c++11, but also for different trade-offs in never-empty guarantee, which were deemed inferior in boost::variant design. variant2 is ismply a third point in the multi-dimensional design space of different trade offs for variant-like libraries. On a different subject, one thing that is missing from both: boost::variant, and std::variant, is the narrow-contract observers for the specific stored type, useful when I know by other means what is currently stored in the variant object. This is something the third variant implementation has a chance to change, however it chose to follow its predecessors. This seems somewhat counter to the spirit of C++. We requested a similar thing of Niall's Outcome library, and the narrow-contract observers were added. Below is the comparison between boost::variant, std::variant and variant2. I do not know how to make a table in the email discussion interface, so I will have to make it with a list. 1. Works in pre-c++11 dialects * boost::variant: yes * std::variant: no * variant2: no 2. Size of variant<X, Y> (when X and Y are unfriendly) * boost::variant: sizeof(void*) + SXY (SXY is the size of union{X x; Y y;}) * boost::variant: sizeof(int) + SXY * variant2::variant: sizeof(int) + 2 * SXY 3. Uses heap allocations * boost::variant: yes * std::variant: no * variant2: no 4. Throws its own exceptions in assignment * boost::variant: yes * std::variant: no * variant2: no 5. Fast (narrow contract) access to X alternative. * boost::variant: no * std::variant: no * variant2: no 7. Strong exception-safety guarantee * boost::variant: no * std::variant: no * variant2: no 8. Never-empty guarantee * boost::variant: yes * std::variant: no * variant2: yes 9. Throws its own exceptions in visitation * boost::variant: no * std::variant: yes * variant2: no Regards, &rzej;
Andrzej Krzemienski wrote:
It should be noted that variant2 DOES NOT PROVIDE A STRONG EXCEPTION SAFETY GUARANTEE.
Yes, variant2::variant is a never-valueless, basic guarantee variant. There probably does exist a legitimate need for a strong guarantee variant, which would have to make different design tradeoffs. It's certainly possible to add one in the future, if there's demand. (For instance, a "strong" variant wouldn't delegate to T::op= in its assignment, from which it falls out that it would be able to store non-assignable types and still be assignable itself. It would double-buffer much more often though.)
It should be noted that variant2 DOES NOT PROVIDE A STRONG EXCEPTION SAFETY GUARANTEE. The never-empty guarantee has a certain negative effect of confusing people so that they are led to believe that they have strong exception safety. But they don't. even expert programmers in this forum expressed their surprise that changing from X to Y in variant<X, Y, Z> can get them a variant storing Z. Sometimes it is even trickier: X may be a container that currently stores 10 elements. You assign Y, and after the exception you get an X that stores zero elements. Because of these problems, the only reasonable thing to do is to follow the advice from Dave Abraham's article: https://www.boost.org/community/exception_safety.html To quote the relevant part, "If a component has many valid states, after an exception we have no idea what state the component is in; only that the state is valid. The options for recovery in this case are limited: either destruction or resetting the component to some known state before further use."
If this principle is followed no-one will ever observe the valueless_by_exception state. This also applies to variant2. If this principle from Dave Abrahams is followed, no-one will ever benefit from
On Tue, Apr 2, 2019 at 9:28 AM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote: the
never-empty guarantee (but people will still pay the costs of providing it). You cannot do anything meaningful with variant that threw other than destroy or reset it. (But it is probably best to destroy it.) If objects are allowed to survive the stack unwinding, they should either offer a strong exception safety guarantee, or you have to have some very detailed knowledge of a given particular type how its state gets modified under exceptions, but this is usually very difficult and best left to experts.
The argument for a variant type that never ends up in a "valueless" state is not that it is wonderful to deal with objects that change state from X to Y, but that error recovery code is already difficult to write correctly, and while dealing with an object found in a state we didn't leave it in (basic exception safety guarantee) may be annoying, this is preferable than having to deal with objects that explode unless they're destroyed. It's not that programmers "want" to do something with the object other than destroying it, but 1) it is difficult to ensure that this can never ever happen and 2) the way it can be ensured is to sprinkle checks at random places (e.g. where variant asserts are triggered when during error recovery some code happens to touch an object but not destroy it), which is even worse.
wt., 2 kwi 2019 o 19:23 Emil Dotchevski via Boost <boost@lists.boost.org> napisał(a):
It should be noted that variant2 DOES NOT PROVIDE A STRONG EXCEPTION SAFETY GUARANTEE. The never-empty guarantee has a certain negative effect of confusing people so that they are led to believe that they have strong exception safety. But they don't. even expert programmers in this forum expressed their surprise that changing from X to Y in variant<X, Y, Z> can get them a variant storing Z. Sometimes it is even trickier: X may be a container that currently stores 10 elements. You assign Y, and after the exception you get an X that stores zero elements. Because of these problems, the only reasonable thing to do is to follow the advice from Dave Abraham's article: https://www.boost.org/community/exception_safety.html To quote the relevant part, "If a component has many valid states, after an exception we have no idea what state the component is in; only that the state is valid. The options for recovery in this case are limited: either destruction or resetting the component to some known state before further use."
If this principle is followed no-one will ever observe the valueless_by_exception state. This also applies to variant2. If this principle from Dave Abrahams is followed, no-one will ever benefit from
On Tue, Apr 2, 2019 at 9:28 AM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote: the
never-empty guarantee (but people will still pay the costs of providing it). You cannot do anything meaningful with variant that threw other than destroy or reset it. (But it is probably best to destroy it.) If objects are allowed to survive the stack unwinding, they should either offer a strong exception safety guarantee, or you have to have some very detailed knowledge of a given particular type how its state gets modified under exceptions, but this is usually very difficult and best left to experts.
The argument for a variant type that never ends up in a "valueless" state is not that it is wonderful to deal with objects that change state from X to Y, but that error recovery code is already difficult to write correctly, and while dealing with an object found in a state we didn't leave it in (basic exception safety guarantee) may be annoying, this is preferable than having to deal with objects that explode unless they're destroyed. It's not that programmers "want" to do something with the object other than destroying it, but 1) it is difficult to ensure that this can never ever happen and 2) the way it can be ensured is to sprinkle checks at random places (e.g. where variant asserts are triggered when during error recovery some code happens to touch an object but not destroy it), which is even worse.
Please, illustrate it with an example. Maybe you have some good scenario in mind, but to me, the necessity of sprinkling checks at various places is an indication of a bad approach to exception handling. If I just unwind the stack and let all my objects die, there is no point where I would need any checks. Regards, &rzej;
On Tue, Apr 2, 2019 at 11:53 AM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:
wt., 2 kwi 2019 o 19:23 Emil Dotchevski via Boost <boost@lists.boost.org> napisał(a):
On Tue, Apr 2, 2019 at 9:28 AM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:
It should be noted that variant2 DOES NOT PROVIDE A STRONG EXCEPTION SAFETY GUARANTEE. The never-empty guarantee has a certain negative effect of confusing people so that they are led to believe that they have strong exception safety. But they don't. even expert programmers in this
expressed their surprise that changing from X to Y in variant<X, Y, Z> can get them a variant storing Z. Sometimes it is even trickier: X may be a container that currently stores 10 elements. You assign Y, and after
exception you get an X that stores zero elements. Because of these problems, the only reasonable thing to do is to follow the advice from Dave Abraham's article: https://www.boost.org/community/exception_safety.html To quote the relevant part, "If a component has many valid states, after an exception we have no idea what state the component is in; only that
state is valid. The options for recovery in this case are limited: either destruction or resetting the component to some known state before further use."
If this principle is followed no-one will ever observe the valueless_by_exception state. This also applies to variant2. If this principle from Dave Abrahams is followed, no-one will ever benefit from the never-empty guarantee (but people will still pay the costs of
it). You cannot do anything meaningful with variant that threw other
destroy or reset it. (But it is probably best to destroy it.) If objects are allowed to survive the stack unwinding, they should either offer a strong exception safety guarantee, or you have to have some very detailed knowledge of a given particular type how its state gets modified under exceptions, but this is usually very difficult and best left to experts.
The argument for a variant type that never ends up in a "valueless" state is not that it is wonderful to deal with objects that change state from X to Y, but that error recovery code is already difficult to write correctly, and while dealing with an object found in a state we didn't leave it in (basic exception safety guarantee) may be annoying, this is preferable
forum the the providing than than
having to deal with objects that explode unless they're destroyed. It's not that programmers "want" to do something with the object other than destroying it, but 1) it is difficult to ensure that this can never ever happen and 2) the way it can be ensured is to sprinkle checks at random places (e.g. where variant asserts are triggered when during error recovery some code happens to touch an object but not destroy it), which is even worse.
Please, illustrate it with an example. Maybe you have some good scenario in mind, but to me, the necessity of sprinkling checks at various places is an indication of a bad approach to exception handling.
Aren't you asking for an example of a destructor that does something other than call other destructors?
wt., 2 kwi 2019 o 21:31 Emil Dotchevski via Boost <boost@lists.boost.org> napisał(a):
On Tue, Apr 2, 2019 at 11:53 AM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:
wt., 2 kwi 2019 o 19:23 Emil Dotchevski via Boost <boost@lists.boost.org
napisał(a):
On Tue, Apr 2, 2019 at 9:28 AM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:
It should be noted that variant2 DOES NOT PROVIDE A STRONG EXCEPTION SAFETY GUARANTEE. The never-empty guarantee has a certain negative effect of confusing people so that they are led to believe that they have
exception safety. But they don't. even expert programmers in this forum expressed their surprise that changing from X to Y in variant<X, Y, Z> can get them a variant storing Z. Sometimes it is even trickier: X may be a container that currently stores 10 elements. You assign Y, and after
exception you get an X that stores zero elements. Because of these problems, the only reasonable thing to do is to follow the advice from Dave Abraham's article: https://www.boost.org/community/exception_safety.html To quote the relevant part, "If a component has many valid states, after an exception we have no idea what state the component is in; only that
state is valid. The options for recovery in this case are limited: either destruction or resetting the component to some known state before further use."
If this principle is followed no-one will ever observe the valueless_by_exception state. This also applies to variant2. If this principle from Dave Abrahams is followed, no-one will ever benefit from the never-empty guarantee (but people will still pay the costs of
it). You cannot do anything meaningful with variant that threw other
destroy or reset it. (But it is probably best to destroy it.) If objects are allowed to survive the stack unwinding, they should either offer a strong exception safety guarantee, or you have to have some very detailed knowledge of a given particular type how its state gets modified under exceptions, but this is usually very difficult and best left to experts.
The argument for a variant type that never ends up in a "valueless" state is not that it is wonderful to deal with objects that change state from X to Y, but that error recovery code is already difficult to write correctly, and while dealing with an object found in a state we didn't leave it in (basic exception safety guarantee) may be annoying, this is preferable
strong the the providing than than
having to deal with objects that explode unless they're destroyed. It's not that programmers "want" to do something with the object other than destroying it, but 1) it is difficult to ensure that this can never ever happen and 2) the way it can be ensured is to sprinkle checks at random places (e.g. where variant asserts are triggered when during error recovery some code happens to touch an object but not destroy it), which is even worse.
Please, illustrate it with an example. Maybe you have some good scenario in mind, but to me, the necessity of sprinkling checks at various places is an indication of a bad approach to exception handling.
Aren't you asking for an example of a destructor that does something other than call other destructors?
Can you show a destructor like that, does a visitatoin on a variant, and that you wouldn't be ashamed to put in your program? Regards, &rzej;
Andrzej Krzemienski wrote:
Can you show a destructor like that, does a visitatoin on a variant, and that you wouldn't be ashamed to put in your program?
The destructor doesn't have to do visitation. It merely needs to call a function f1, which calls a function f2, which calls a function f3, which calls a function f4, which does visitation. What we've been telling you from the start is that this forces you to partition your functions into two classes, one allowed to do visitation, the other not, then keep track of which is which, never calling the wrong class in a destructor, or from a catch clause. And, of course, this has nothing to do with visitation, specifically. The exception safety guarantee is a global thing. Once you lose basic, you lose it everywhere, not just for visit.
wt., 2 kwi 2019 o 22:59 Peter Dimov via Boost <boost@lists.boost.org> napisał(a):
Andrzej Krzemienski wrote:
Can you show a destructor like that, does a visitatoin on a variant, and that you wouldn't be ashamed to put in your program?
The destructor doesn't have to do visitation. It merely needs to call a function f1, which calls a function f2, which calls a function f3, which calls a function f4, which does visitation.
What we've been telling you from the start is that this forces you to partition your functions into two classes, one allowed to do visitation, the other not, then keep track of which is which, never calling the wrong class in a destructor, or from a catch clause.
And, of course, this has nothing to do with visitation, specifically. The exception safety guarantee is a global thing. Once you lose basic, you lose it everywhere, not just for visit.
I think I will take it to a separate thread in order not to obfuscate the review. One other remark I forgot to mention in the initial message is that the library upon assignment and emplacement prefers to first create a "temporary" object and then move from into the desired location. This works under the implicit assumption that the move constructor is cheaper that the said initialization. But I do not think it is the case in general. Moves are faster compared to other initialization only in the cases where the type is implemented as a handle to parts that are remote: allocated on heap, or stored somewhere in the kernel, and who often have weak invariants: in the moved from state they do not represent any resource. But other types like std::array<>, static_vector, even if they provide noexcept move are as expensive as making another copy. This is another price to be paid for the never-empty guarantee. There are also the cases where some type has move operations that can obviously never fail, but the author forgot to annotate them with noexcept (or noexcept was not available at the time when the type was designed). This last problem could be addressed if variant2 did not use std::is_nothrow_move_constructible directly, but a custom type trait that users would be able to specialize, in order to control the behavior of variant2. Regards, &rzej;
Andrzej Krzemienski wrote:
I have started a thread in this list a while ago, requesting for an example of code that *correctly handles exceptions* (does not stop stack unwinding at random places), where the programmer would make use of the never-empty guarantee , and chose something else than destroying or resetting the variant. And although I received some generic statements, referring to the purity of the design, strength of the invariants, and the easiness of thinking or "correctness", none of the proponents of the never-empty guarantee gave such an example.
template<class T> class monitor { private: char const* file_; int line_; T const& v_; T old_; public: explicit monitor( char const* file, int line, T const& v ): file_( file ), line_( line ), v_( v ), old_( v ) {} ~monitor() { if( v_ != old_ ) { std::clog << file_ << ":" << line_ << ": monitored value changed from " << old_ << " to " << v_ << std::endl; } } }; Or in general, in a destroy-only world, you can never read any values in a destructor or in a function called from a catch clause. It might be an interesting experiment to make Clang warn on such a use and then try it on some real codebases. These reads of a potentially-destroy-only values are invisible to you because we don't live in such a world.
wt., 2 kwi 2019 o 19:54 Peter Dimov via Boost <boost@lists.boost.org> napisał(a):
Andrzej Krzemienski wrote:
I have started a thread in this list a while ago, requesting for an example of code that *correctly handles exceptions* (does not stop stack unwinding at random places), where the programmer would make use of the never-empty guarantee , and chose something else than destroying or resetting the variant. And although I received some generic statements, referring to the purity of the design, strength of the invariants, and the easiness of thinking or "correctness", none of the proponents of the never-empty guarantee gave such an example.
template<class T> class monitor { private:
char const* file_; int line_; T const& v_; T old_;
public:
explicit monitor( char const* file, int line, T const& v ): file_( file ), line_( line ), v_( v ), old_( v ) {}
~monitor() { if( v_ != old_ ) { std::clog << file_ << ":" << line_ << ": monitored value changed from " << old_ << " to " << v_ << std::endl; } } };
Thank you. This is the best example I have seen so far.
Or in general, in a destroy-only world, you can never read any values in a destructor or in a function called from a catch clause.
Yes: this seems reasonable, you cannot just call any non-mutating function on a type you do nto know in a piece of the program that is expected not to fail (such as a destructor or scope guard). Similarly, you do not want to invoke arbitrary non-modifying functions in catch clauses on objects that may have suffered from a failure, whose type you do not know and whose invariants can potentially be very "weak". Logging in destructor is risky: you are doing more than cleanup and this can fail on its own. Your example would not compile because variant2 does not provide the streaming operator. Regards, &rzej; It might be an
interesting experiment to make Clang warn on such a use and then try it on some real codebases.
These reads of a potentially-destroy-only values are invisible to you because we don't live in such a world.
On 3/04/2019 05:27, Andrzej Krzemienski wrote:
This is not a full review yet. I just wanted to discuss the design first, especially the never-empty guarantee. And I am going to rehash the old argument.
The core of variant2 is its never-empty guarantee and the mechanism to achieve it: a number of different algorithms chosen depending on what the potentially stored types permit. In fact this mechanism is not much different than the one present in boost::variant.
I agree with most of the points that Andrzej has made, although I would go a step further and question why people want a never-empty variant in the first place. There is always potential uncertainty about the actual content type of a variant, requiring either assumptions (and UB or exceptions if wrong) or explicitly testing, eg. with visit. Checking if a variant is empty seems no different to me than checking if it contains an int; eliding either check confers exactly the same risks. Requiring that variants always possess an empty state dramatically simplifies implementation, including default construction and move assignment. And it avoids the valueless_by_exception kludge by promoting it to an entirely expected possible state of variant. (Thereby increasing the chance that consumers will consider it.) It also means that (like traditional monads), variant<T> is behaviorally equivalent (bar some naming conventions) to optional<T>, which seems like a consistent conceptual choice. And variant<> is valid and can only hold the empty state. Furthermore, while I can see possible value in a never-empty strong-guarantee variant (although I remain unconvinced that the "never-empty" part provides much benefit, as above); as soon as this guarantee is weakened then I don't see any advantage at all in trying to maintain a "never-empty" property. Unexpectedly changing type and/or content is exactly as bad as unexpectedly becoming empty.
On Tue, Apr 2, 2019 at 4:33 PM Gavin Lambert via Boost < boost@lists.boost.org> wrote:
There is always potential uncertainty about the actual content type of a variant, requiring either assumptions (and UB or exceptions if wrong) or explicitly testing, eg. with visit
I read the above as "UB or error handling if wrong". It does not matter what form of error handling is used. The use of "assumption" is imprecise, because the assumptions we're talking about are a matter of a conscious design choice. If we're assuming that the basic invariants are in place during error recovery (the basic guarantee), it is because we are writing code to ensure that it is true. Of course the point of it is to avoid having to explicitly test. The problem with explicitly testing is that it must be done everywhere because we don't know if maybe we are being called from an error-handling context (note, this doesn't necessarily mean exception handling). For example, every member function must check if the class invariants are in place for *this (assuming UB is not acceptable).
Checking if a variant is empty seems no different to me than checking if it contains an int; eliding either check confers exactly the same risks.
Under the basic guarantee, you'd only check if it contains an int if you care whether it contains an int. Typical use is not to check, but to do something if it contains an int and something else if it contains a different type. There is no check.
Unexpectedly changing type and/or content is exactly as bad as unexpectedly becoming empty.
It is not as bad. Introducing the empty state forces us to define behavior for that state. This is logically equivalent to requiring well defined behavior from any member function even for instances that failed to initialize. It's better to ensure and then assume that this can not happen.
On 3/04/2019 13:38, Emil Dotchevski wrote:
The problem with explicitly testing is that it must be done everywhere because we don't know if maybe we are being called from an error-handling context (note, this doesn't necessarily mean exception handling). For example, every member function must check if the class invariants are in place for *this (assuming UB is not acceptable).
How is this different if the instance can be empty, or can unexpectedly change from type A to type B? If you don't know whether you're handling an errored instance or not, then it could be unexpectedly empty (with one variant implementation) or it could be unexpectedly type B (with another implementation). I don't see any practical difference between these cases.
Under the basic guarantee, you'd only check if it contains an int if you care whether it contains an int. Typical use is not to check, but to do something if it contains an int and something else if it contains a different type. There is no check.
That's still a check, even if the check is inside of visit().
It is not as bad. Introducing the empty state forces us to define behavior for that state. This is logically equivalent to requiring well defined behavior from any member function even for instances that failed to initialize. It's better to ensure and then assume that this can not happen.
The empty state is just another type that the variant can hold. Think of it as std::monostate always implicitly being the first type argument of the variant (as Peter points out). My point is that I don't see any advantages to requiring that people specify this explicitly, and I only see disadvantages to allowing people to not specify it at all. If it's illegal for the first type of the variant to be anything other than std::monostate (which I think should be the case), then it could be implicit rather than explicit. And both implementation and usage of the variant would be dramatically simplified. And then you also get the useful properties that variant<> is valid (which is handy for generic variadic code) and variant<T> is equivalent to optional<T> (and variant<T, U> is similar to optional<union<T, U>>, but better). This just "makes sense" to me.
On Tue, Apr 2, 2019 at 7:07 PM Gavin Lambert via Boost < boost@lists.boost.org> wrote:
On 3/04/2019 13:38, Emil Dotchevski wrote:
The problem with explicitly testing is that it must be done everywhere because we don't know if maybe we are being called from an
error-handling
context (note, this doesn't necessarily mean exception handling). For example, every member function must check if the class invariants are in place for *this (assuming UB is not acceptable).
How is this different if the instance can be empty, or can unexpectedly change from type A to type B?
If you don't know whether you're handling an errored instance or not, then it could be unexpectedly empty (with one variant implementation) or it could be unexpectedly type B (with another implementation). I don't see any practical difference between these cases.
Consistent with the basic guarantee, if you assign one std::vector to another and the copy operation fails, the contents of the target vector is not specified, but it is guaranteed to be a valid vector, so that if you happen to handle it later, you can safely do with it what you do with any other vector: you can insert or delete elements, iterate, whatever. By analogy with your position, a better design for std::vector would be to define a special "error" state that you must check for *every* time you handle a vector, and the justification for that design is that otherwise it might be unexpected to see a vector in a half-copied (but valid, by definition) state. Do you see the practical difference now? Going back to variant, in one case, we are defining an empty state, which forces the user to provide explicit handling for that state, *everywhere*. Otherwise, we are not defining an empty state, and the user doesn't have to provide special handling, *anywhere*, because he doesn't have to care that A "unexpectedly" changed into B. He sees a B, he does what he normally does with B.
On 4/04/2019 13:26, Emil Dotchevski wrote:
Consistent with the basic guarantee, if you assign one std::vector to another and the copy operation fails, the contents of the target vector is not specified, but it is guaranteed to be a valid vector, so that if you happen to handle it later, you can safely do with it what you do with any other vector: you can insert or delete elements, iterate, whatever.
By analogy with your position, a better design for std::vector would be to define a special "error" state that you must check for *every* time you handle a vector, and the justification for that design is that otherwise it might be unexpected to see a vector in a half-copied (but valid, by definition) state. Do you see the practical difference now?
No, because vector already has that state -- it's called "a vector with size zero" (aka "empty"), which is the same state it typically transitions to when successfully moved-from. This is also the case for all the smart pointer types. I am saying that variant should behave exactly the same way. It's what people expect.
On Thu, Apr 4, 2019 at 3:14 PM Gavin Lambert via Boost < boost@lists.boost.org> wrote:
On 4/04/2019 13:26, Emil Dotchevski wrote:
Consistent with the basic guarantee, if you assign one std::vector to another and the copy operation fails, the contents of the target vector
is
not specified, but it is guaranteed to be a valid vector, so that if you happen to handle it later, you can safely do with it what you do with any other vector: you can insert or delete elements, iterate, whatever.
By analogy with your position, a better design for std::vector would be to define a special "error" state that you must check for *every* time you handle a vector, and the justification for that design is that otherwise it might be unexpected to see a vector in a half-copied (but valid, by definition) state. Do you see the practical difference now?
No, because vector already has that state -- it's called "a vector with size zero" (aka "empty"), which is the same state it typically transitions to when successfully moved-from.
template <class T> void f( std::vector<T> & x, std::vector<T> const & y ) { x = y; } Above, if the assignment fails, the content of x is unspecified, but it is guaranteed to be a valid vector, containing valid objects (if not empty). By analogy: if a variant2 assignment fails, its value is unspecified, but it is guaranteed to be valid.
Gavin Lambert wrote:
Requiring that variants always possess an empty state dramatically simplifies implementation, including default construction and move assignment. And it avoids the valueless_by_exception kludge by promoting it to an entirely expected possible state of variant. (Thereby increasing the chance that consumers will consider it.)
That's more or less what using `monostate` as the first alternative achieves.
śr., 3 kwi 2019 o 01:33 Gavin Lambert via Boost <boost@lists.boost.org> napisał(a):
On 3/04/2019 05:27, Andrzej Krzemienski wrote:
This is not a full review yet. I just wanted to discuss the design first, especially the never-empty guarantee. And I am going to rehash the old argument.
The core of variant2 is its never-empty guarantee and the mechanism to achieve it: a number of different algorithms chosen depending on what the potentially stored types permit. In fact this mechanism is not much different than the one present in boost::variant.
I agree with most of the points that Andrzej has made, although I would go a step further and question why people want a never-empty variant in the first place.
There is always potential uncertainty about the actual content type of a variant, requiring either assumptions (and UB or exceptions if wrong) or explicitly testing, eg. with visit. Checking if a variant is empty seems no different to me than checking if it contains an int; eliding either check confers exactly the same risks.
Requiring that variants always possess an empty state dramatically simplifies implementation, including default construction and move assignment. And it avoids the valueless_by_exception kludge by promoting it to an entirely expected possible state of variant. (Thereby increasing the chance that consumers will consider it.)
It also means that (like traditional monads), variant<T> is behaviorally equivalent (bar some naming conventions) to optional<T>, which seems like a consistent conceptual choice. And variant<> is valid and can only hold the empty state.
In principle, the stronger the invariant, the fewer opportunities to make bug. The strive for never-empty variant is conceptually similar to a strive for a pointer that does not have the null-pointer state. We often give advice to people that as long as this is possible you should prefer using references to using pointers, because the former do not have the null state. The significant portion of crashes in the programs I have worked with was when a null pointer was dereferenced. Function f() returns a shared_ptr<> and the implied contract is, it is not null. I do not want to check for null pointer whenever I use it. But at some point some clever developer comes to an idea that if he doesn't know what to do, he will return a null pointer. These problems would not happen if there was a type never_null_shared_ptr. It just wipes a class of bugs from existence. I am myself in favor of never-empty guarantee for variant, as long as it does not cost too much. The notion of how easy/hard it is to implement something should not drive the interface and the contract of a type. I object to the never-empty guarantee for variant because it costs too much and affects the contract of the type. In C++ performance characteristics of a type are part of its contract. Regards, &rzej;
Furthermore, while I can see possible value in a never-empty strong-guarantee variant (although I remain unconvinced that the "never-empty" part provides much benefit, as above); as soon as this guarantee is weakened then I don't see any advantage at all in trying to maintain a "never-empty" property. Unexpectedly changing type and/or content is exactly as bad as unexpectedly becoming empty.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I have to say that I'm with Andrzej on this matter. Mabye I'm blind, but it seems to me that people repeatedly mix up "never empty" and "no error state". Variant2 can still get into an error state, just that that error state is not explicitly named and modeled as part of variant2, but instead, a "regular" state is reused to signal that error: Either monostate or whatever is the first type that has a non throwing default constructor. It is only if neither of those options is available that variant2 falls back to double buffering and does actually avoid ending up in an error state at all. Also, I don't see the problem with valueless_by_exception. Other than nullptr, you don't have to explicitly check for it most of the time (at least not more often than you'd have to check for monostate in variant2). Consider what you can do with a variant: - get: If you are guessing the current type wrong, you get an exception Doesn't matter if the actual type is valueless_by_exception or just an unexpected regular type. - get_if: Again, if you are guessing wrong, you get a nullptr - copy/move: valueless again behaves just like another type - index: Valueless just returns a unique index. The only special thing about it is that it doesn't fall into the regular range between zero and the number of member types. - comparison operators: valueless_by_exception is even more convenient, because you don't have to check first if the left and right hand side are of the same type, but you can still treat it just as any other T if you want. - visit: This is the only case where valueless is really special: You get an exception instead of having to provide an overload for e.g. monostate. Considering how rarely a variant ends up in such a state, throwing an exception is imho the right thing to do most of the time anyway (use an exception to report exceptional errors). But when that is not appropriate, then yes, you have to either check beforehand or catch the exception instead of building the logic into your visitor. Of course I can imagine a scenario, where one or the other approach is simpler, but 99% of the time it really doesn't make a big difference. What I'm trying to say here is not that the design of std::variant is better than boost::variant2, but that they are just two different design choices that may be more or less convenient in a particular scenario, but overall don't make a big difference one way or the other. What I personally do find a bit problematic is that variant2 would be yet another variant type that is not completely compatible to std::variant, without providing significant benefits or a fundamentally different design (e.g. providing strong exception guarantee or dramatically improved compile-time or run-time performance). Mike Dev
On Wed, Apr 3, 2019 at 2:08 AM Mike via Boost <boost@lists.boost.org> wrote:
Mabye I'm blind, but it seems to me that people repeatedly mix up "never empty" and "no error state".
Variant2 can still get into an error state, just that that error state is not explicitly named and modeled as part of variant2, but instead, a "regular" state is reused to signal that error:
What you're missing is that this is a matter of definition. Your definition is: if we enter a state because of an error, then that state is an error state. An alternative definition is: if an assignment operation fails, the object is left in an unspecified but valid state. Under the second definition, it is illogical to call the "unspecified but valid" state an error state that must be checked, because by definition it is a perfectly valid state. Further, this valid state is not "reused to signal that error". It is the result of the error, but the error is communicated by other means, e.g. an exception.
participants (5)
-
Andrzej Krzemienski -
Emil Dotchevski -
Gavin Lambert -
Mike -
Peter Dimov