[contract] move operations and class invariants
Hello all, This is not really a question about Boost.Contract, but more a question on how contract programming interacts with C++ move operations. C++ requires that moved-from objects can still be destructed. Because contract programming requires class invariants to hold at destructor entry, it follows that moved-from objects must still satisfy class invariants. That sounds restrictive... and it might force the class invariants to be empty. For example, for vector a class invariant is size() <= capacity(). Should that sill hold after the vector has been moved? That means I can still call size() and capacity() on a moved-from object, which might not be the case. If some sort of moved() function could be called on a moved-from object, the invariants could be programmed as follow to work around this issue: class vector { void invariant() cont { if(!moved()) BOOST_CONTRACT_ASSERT(size() <= capacity()); ... // Only invariants that are truly needed to execute the destructor. } bool moved() const; public: vector(vector&& other) { boost::contract::check c = boost::contract::constructor(this) .postcondition([&] { BOOST_CONTRACT_ASSERT(!moved()); BOOST_CONTRACT_ASSERT(other.moved()); }) ; ... } ... }; I'm not really sure... What do you think? Do you know if this topic "C++ move & class invariants" has already been discussed somewhere? Thanks. --Lorenzo
Le 2017-11-29 05:39, Lorenzo Caminiti via Boost a écrit :
Hello all,
This is not really a question about Boost.Contract, but more a question on how contract programming interacts with C++ move operations.
C++ requires that moved-from objects can still be destructed. Because contract programming requires class invariants to hold at destructor entry, it follows that moved-from objects must still satisfy class invariants.
This is not only about destructors, but more generally about reusing moved-from objects.
That sounds restrictive... and it might force the class invariants to be empty. For example, for vector a class invariant is size() <= capacity(). Should that sill hold after the vector has been moved? That means I can still call size() and capacity() on a moved-from object, which might not be the case.
If it can’t hold, that should be part of the contract. IMHO moving-from can result in two situations : 1) reverting back to an empty state 2) going to an invalid state The bad thing with 2) is that currently you have no way to compile-time check it, and it will lead to crashes. 1) is safer, but has problems with RAII, since the object no longer hold any resource.
If some sort of moved() function could be called on a moved-from object, the invariants could be programmed as follow to work around this issue:
I'm not really sure... What do you think? Do you know if this topic "C++ move & class invariants" has already been discussed somewhere?
I’m not aware of any litterature on this. At first glance, i’ll separate two things : - value types (such as std::vector), for which move semantic is mainly a performance matter. For those types, 1) (empty state) makes a lot of sense. - entity types (which holds resource), for which move semantic is really an ownership transfer. For those types, 1) doesn’t make a lot of sense, and 2) may lead to hard to debug crashes. In the current state of the language, I’d rather make these types not movable, and use unique_ptr to transfer ownership. Regards, Julien
You could add a `bool moved_from` field and OR it in your invariants. If you can afford the cost of contracts, then maybe you can also afford the overhead of this extra field. On Nov 29, 2017 1:14 AM, "Julien Blanc via Boost" <boost@lists.boost.org> wrote:
Le 2017-11-29 05:39, Lorenzo Caminiti via Boost a écrit :
Hello all,
This is not really a question about Boost.Contract, but more a question on how contract programming interacts with C++ move operations.
C++ requires that moved-from objects can still be destructed. Because contract programming requires class invariants to hold at destructor entry, it follows that moved-from objects must still satisfy class invariants.
This is not only about destructors, but more generally about reusing moved-from objects.
That sounds restrictive... and it might force the class invariants to
be empty. For example, for vector a class invariant is size() <= capacity(). Should that sill hold after the vector has been moved? That means I can still call size() and capacity() on a moved-from object, which might not be the case.
If it can’t hold, that should be part of the contract. IMHO moving-from can result in two situations : 1) reverting back to an empty state 2) going to an invalid state
The bad thing with 2) is that currently you have no way to compile-time check it, and it will lead to crashes. 1) is safer, but has problems with RAII, since the object no longer hold any resource.
If some sort of moved() function could be called on a moved-from
object, the invariants could be programmed as follow to work around this issue:
I'm not really sure... What do you think? Do you know if this topic
"C++ move & class invariants" has already been discussed somewhere?
I’m not aware of any litterature on this.
At first glance, i’ll separate two things :
- value types (such as std::vector), for which move semantic is mainly a performance matter. For those types, 1) (empty state) makes a lot of sense. - entity types (which holds resource), for which move semantic is really an ownership transfer. For those types, 1) doesn’t make a lot of sense, and 2) may lead to hard to debug crashes. In the current state of the language, I’d rather make these types not movable, and use unique_ptr to transfer ownership.
Regards,
Julien
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost
2017-11-29 5:39 GMT+01:00 Lorenzo Caminiti via Boost <boost@lists.boost.org> :
Hello all,
This is not really a question about Boost.Contract, but more a question on how contract programming interacts with C++ move operations.
C++ requires that moved-from objects can still be destructed. Because contract programming requires class invariants to hold at destructor entry, it follows that moved-from objects must still satisfy class invariants.
That sounds restrictive... and it might force the class invariants to be empty. For example, for vector a class invariant is size() <= capacity(). Should that sill hold after the vector has been moved? That means I can still call size() and capacity() on a moved-from object, which might not be the case.
If some sort of moved() function could be called on a moved-from object, the invariants could be programmed as follow to work around this issue:
class vector { void invariant() cont { if(!moved()) BOOST_CONTRACT_ASSERT(size() <= capacity()); ... // Only invariants that are truly needed to execute the destructor. }
bool moved() const;
public: vector(vector&& other) { boost::contract::check c = boost::contract::constructor(this) .postcondition([&] { BOOST_CONTRACT_ASSERT(!moved()); BOOST_CONTRACT_ASSERT(other.moved()); }) ; ... }
... };
I'm not really sure... What do you think? Do you know if this topic "C++ move & class invariants" has already been discussed somewhere?
Sort of. I think that there is a consensus there: if you allow the *special* moved-from state, or "zombie" state (occasionally, but not necessarily, equivalent to default-constructed state), you cannot have strong invariants in your class. Now your invariants will have to be "either in zombie state or the strong invariant holds". Some mention of it in the blog post: https://akrzemi1.wordpress.com/2016/04/07/sessions-and-object-lifetimes/ This is one tiny aspect where C++11 move semantics made the language a bit worse compared to C++98. This can be somewhat mitigated in C++17, where you can return by value non-moveable types. No moves - no weak invariant problems. Proper fix could only be achieved with "destructive move", but I do not know if it is doable in C++. Regards, &rzej;
Lorenzo Caminiti wrote:
C++ requires that moved-from objects can still be destructed. Because contract programming requires class invariants to hold at destructor entry, it follows that moved-from objects must still satisfy class invariants.
No, moved-from objects must be valid objects. Not only can they be destroyed, they must be usable. Their state is unspecified, but invariants hold.
That sounds restrictive... and it might force the class invariants to be empty. For example, for vector a class invariant is size() <= capacity(). Should that sill hold after the vector has been moved?
Yes.
That means I can still call size() and capacity() on a moved-from object, which might not be the case.
No. You must be able to call size() and capacity().
On 30/11/2017 03:15, Peter Dimov wrote:
Lorenzo Caminiti wrote:
C++ requires that moved-from objects can still be destructed. Because contract programming requires class invariants to hold at destructor entry, it follows that moved-from objects must still satisfy class invariants.
No, moved-from objects must be valid objects. Not only can they be destroyed, they must be usable. Their state is unspecified, but invariants hold.
In general you should expect to be able to call any method which is valid on a default-constructed object, *especially* assignment operators (as it's relatively common to reassign a moved-from object). (You cannot, however, actually assume that it will return the same answers as a default-constructed object would.) It *might* also be legal to call other methods with more restrictive preconditions, but that's sort of the point -- you can't assume this either way, so actually doing so isn't really valid. The part about moved-from objects being in an unspecified state is not meant to imply that they could be "zombie" objects or otherwise invalid (because that shouldn't happen), it just means that it's allowed for the class to implement a move as a "true" move (give away storage, so now the instance is empty) or as a copy (both objects now have the same data in separate storages), or somewhere in between (such as a swap). To put it another way, if you move-assign a vector to another vector, it is perfectly legal for the source vector to not be empty afterwards. (For example, instead of a destroy-and-swap it could be implemented as a pure swap. Or even as a copy, although that's less likely with modern STLs.) This is also why this code is perfectly valid: std::vector<int> a { 1, 2, 3 }; std::vector<int> b; b = std::move(a); a.clear(); a.push_back(4); // a == { 4 }, b == { 1, 2, 3 } But omitting the call to clear() would be a bug -- it's still legal, but there's no guarantee what the contents of "a" would be at the end if you didn't explicitly clear it, so it is probably unintended. (At this point language lawyers might jump on me that vector does actually provide somewhat more specific guarantees about how move-assign behaves. But the point stands for generic types.)
On 30/11/2017 11:45, I wrote:
To put it another way, if you move-assign a vector to another vector, it is perfectly legal for the source vector to not be empty afterwards. (For example, instead of a destroy-and-swap it could be implemented as a pure swap. Or even as a copy, although that's less likely with modern STLs.) [...] (At this point language lawyers might jump on me that vector does actually provide somewhat more specific guarantees about how move-assign behaves. But the point stands for generic types.)
Donning my own language-lawyer hat for a moment: there are actually some cases where a modern STL is actually *required* to implement a move-assign like a copy-assign -- notably when containing a type that has a custom copy-constructor without a custom move-constructor and using a non-propagate_on_container_move_assignment allocator which doesn't compare equal. Perhaps this can be argued to be a niche case, but it would indeed be a case where the source vector isn't empty afterwards, thus requiring the clear().
2017-11-29 23:45 GMT+01:00 Gavin Lambert via Boost <boost@lists.boost.org>:
On 30/11/2017 03:15, Peter Dimov wrote:
Lorenzo Caminiti wrote:
C++ requires that moved-from objects can still be destructed. Because
contract programming requires class invariants to hold at destructor entry, it follows that moved-from objects must still satisfy class invariants.
No, moved-from objects must be valid objects. Not only can they be destroyed, they must be usable. Their state is unspecified, but invariants hold.
In general you should expect to be able to call any method which is valid on a default-constructed object, *especially* assignment operators (as it's relatively common to reassign a moved-from object). (You cannot, however, actually assume that it will return the same answers as a default-constructed object would.)
Agreed (assuming you meant "on a moved-from-object" rather than "on a default-constructed object"), but while such an object is "valid", this information is of little use in some cases. And I think it is such cases that are relevant for creating class invariants. Let me give you some context. I would like to create a RAII-like class representing a session with an open file. When I disable all moves and copies and the default constructor (so that it is a guard-like object) I can provide a very useful guarantee: When you have an object of type `File` within its lifetime, it means the file is open and you can write to it, or read from it. This means calling `file.write()` and `file.read()` is *always* valid and always performs the desired IO operation. When it comes to expressing invariant, I can say: ``` bool invariant() const { this->_file_handle != -1; } ``` (assuming that -1 represents "not-a-handle") But my type is not moveable. So I add move operations (and not necessarily the default constructor), but now I have this moved-from state, so my guarantee ("When you have an object of type `File` within its lifetime, it means the file is open and you can write to it, or read from it") is no longer there. You may have an object to which it is invalid to write. Of course, the moved-from-object is still "valid", but now "valid" only means "you can call function `is_valid()` and then decide" (and of course you can destroy, assign, but that's not the point). Now, in turn, every function like `read()` or `write()` has a precondition: `is_valid()`. So object is always "valid" but calling 90% of its interface is invalid (unless you guarantee the precondition manually). The invariant informally is "either in a moved-from-state or you can use write/read", and there may be no way to express it in the code. This is still an "invariant", but it is *weak*, that is, it is less useful in practice. The previous invariant (in the guard-like design) is *strong* it has practical value to the user: I do not have to check anything before calling `read()`. The new invariant is *weak*: you have to "check" something time and again, and the design is more prone to bugs: you can call functions out of contract. The distinction into "weak" and "strong" invariants is not strict or formal, but it does matter in practice. I think this is the problem that Lorenzo is facing. Regards, &rzej;
On 30/11/2017 20:48, Andrzej Krzemienski wrote:
In general you should expect to be able to call any method which is valid on a default-constructed object, *especially* assignment operators (as it's relatively common to reassign a moved-from object). (You cannot, however, actually assume that it will return the same answers as a default-constructed object would.)
Agreed (assuming you meant "on a moved-from-object" rather than "on a default-constructed object"), but while such an object is "valid", this information is of little use in some cases. And I think it is such cases that are relevant for creating class invariants.
Not quite. I meant "you should be able to call any method on a moved-from object that is valid for a default-constructed object", ie. those without strict preconditions, ie. the class invariant should still hold and the object should still be in a valid state -- you just can't assume any particular state (neither empty nor full nor somewhere in between). As such it is usually only reasonable to perform those operations which cause a well-defined postcondition state regardless of the initial state -- ie. assignment, destruction, or explicit clearing or resetting or things of that nature. But it would also be legal to perform other operations and then interrogate the object about its resulting state -- but that's rarely useful in practice as it's a possible source of nondeterminism in different environments, and usually we want our software to be more predictable. :)
Let me give you some context. I would like to create a RAII-like class representing a session with an open file. When I disable all moves and copies and the default constructor (so that it is a guard-like object) I can provide a very useful guarantee: When you have an object of type `File` within its lifetime, it means the file is open and you can write to it, or read from it.
This means calling `file.write()` and `file.read()` is *always* valid and always performs the desired IO operation. When it comes to expressing invariant, I can say:
``` bool invariant() const { this->_file_handle != -1; } ```
(assuming that -1 represents "not-a-handle")
But my type is not moveable. So I add move operations (and not necessarily the default constructor), but now I have this moved-from state, so my guarantee ("When you have an object of type `File` within its lifetime, it means the file is open and you can write to it, or read from it") is no longer there. You may have an object to which it is invalid to write. Of course, the moved-from-object is still "valid", but now "valid" only means "you can call function `is_valid()` and then decide" (and of course you can destroy, assign, but that's not the point).
As soon as you add those move operations which can put the class into a state where the invariant no longer holds, then it's not an invariant any more. At best it becomes preconditions for most of the methods. This should be self-evident. (Move-assignment isn't too bad, as that can be implemented as a pure swap, which will maintain invariants. But move-construction is an invariant-killer, because it's effectively a swap with nothingness.) Any time that you have a class that wants to provide a "no empty guarantee", and you want to add a move operation to it, you have a problem. I recommend not trying to mix these concepts -- while not completely incompatible, they don't play nicely together. (This also applies to default construction -- if you find yourself wanting to make something non-default-constructible because that would make it somehow invalid, then it probably shouldn't be moveable.) If you want to make a file handle that you can move, then you should sacrifice the no-empty guarantee and allow it to default-construct to "no file open", and return to that state when moved-from. And yes, then you need to check *at certain boundaries* and after certain operations that you've been given a non-empty handle. Emptiness is not an unexpected state for a file handle, so this should surprise nobody. (And you then have to decide an appropriate balance between setting preconditions but merely asserting them in debug builds, or verifying them explicitly in all builds and returning errors or throwing exceptions. But that's true for anything.) Another option if you really want to retain both no-empty and moveability is to wrap it in a unique_ptr. Now you're moving the pointer to the object, not the object itself, which remains immobile. It still means you have to check if someone's handed you an empty pointer -- but you can be more explicit at the boundaries, with methods taking a unique_ptr<File> (&& or const&) if they will be checking if it's empty or taking a File (& or const&) if they assume they've been given a non-empty one. Granted that it is *possible* to implement move operations on a no-empty class, but AFAIK this invariably leads to producing a zombie object where any attempt to use it other than for assignment or destruction would produce UB due to violated preconditions (and consequently also weakening the class invariant to become method preconditions). This seems like a really bad idea to me.
Hello Andrzej and all, On Wed, Nov 29, 2017 at 11:48 PM, Andrzej Krzemienski via Boost <boost@lists.boost.org> wrote:
Let me give you some context. I would like to create a RAII-like class representing a session with an open file. When I disable all moves and copies and the default constructor (so that it is a guard-like object) I can provide a very useful guarantee: When you have an object of type `File` within its lifetime, it means the file is open and you can write to it, or read from it.
This means calling `file.write()` and `file.read()` is *always* valid and always performs the desired IO operation. When it comes to expressing invariant, I can say:
``` bool invariant() const { this->_file_handle != -1; } ```
(assuming that -1 represents "not-a-handle")
But my type is not moveable. So I add move operations (and not necessarily the default constructor), but now I have this moved-from state, so my guarantee ("When you have an object of type `File` within its lifetime, it means the file is open and you can write to it, or read from it") is no longer there. You may have an object to which it is invalid to write. Of course, the moved-from-object is still "valid", but now "valid" only means "you can call function `is_valid()` and then decide" (and of course you can destroy, assign, but that's not the point).
Now, in turn, every function like `read()` or `write()` has a precondition: `is_valid()`. So object is always "valid" but calling 90% of its interface is invalid (unless you guarantee the precondition manually).
The invariant informally is "either in a moved-from-state or you can use write/read", and there may be no way to express it in the code. This is still an "invariant", but it is *weak*, that is, it is less useful in practice. The previous invariant (in the guard-like design) is *strong* it has practical value to the user: I do not have to check anything before calling `read()`.
The new invariant is *weak*: you have to "check" something time and again, and the design is more prone to bugs: you can call functions out of contract.
I agree. In code: #include <boost/contract.hpp> class myfile { void invarinat() const { if(is_valid()) BOOST_CONTRACT_ASSERT(handle_ != -1); // Else, only assertions the destructor absolutely needs to be true. } bool valid_; file_handle handle_; public: bool is_valid() const { boost::contract::check c = boost::contract::public_function(this); return valid_; } void read() { boost::contract::check c = boost::contract::public_function(this) .precondition([&] { BOOST_CONTRACT_ASSERT(is_valid()); }) ; /* ... */ } myfile& operator=(myfile&& from) { boost::contract::check c = boost::contract::public_function(this) .precondition([&] { BOOST_CONTRACT_ASSERT(from.is_valid()); }) .postcondition([&] { BOOST_CONTRACT_ASSERT(is_valid()); BOOST_CONTRACT_ASSERT(!from.is_valid()); }) ; /* ... */ } ~myfile() { boost::contract::check c = boost::contract::public_function(this); /* ... */ } /* ... */ }; Two questions: 1. Is it OK to assume I can call is_valid() on a moved-from object (so I can put it to guard invariants, preconditions, etc. and also check it in user code where needed to satisfy preconditions)? Based on the replies to this email thread so far, I think the answers is "yes". 2. How useful is a class like the one above with "crippled" invariants and is_valid() preconditions on all its useful public methods like read()? The answer seems to be: not very useful. I guess that's the price to pay for the performance gain of moving objects around... Thanks, --Lorenzo
Lorenzo Caminiti wrote:
2. How useful is a class like the one above with "crippled" invariants and is_valid() preconditions on all its useful public methods like read()?
A read() member that takes no arguments and returns void is not very realistic. read() can fail, so you can drop its "valid" precondition and just fail when the handle is invalid.
Le 2017-12-01 02:52, Lorenzo Caminiti via Boost a écrit :
1. Is it OK to assume I can call is_valid() on a moved-from object (so I can put it to guard invariants, preconditions, etc. and also check it in user code where needed to satisfy preconditions)? Based on the replies to this email thread so far, I think the answers is "yes".
2. How useful is a class like the one above with "crippled" invariants and is_valid() preconditions on all its useful public methods like read()? The answer seems to be: not very useful. I guess that's the price to pay for the performance gain of moving objects around...
Nobody will care to check is_valid() before every call, especially if validity is assumed in the majority of cases and invalidity a rare case. Worse, people will forget to add it to their own functions preconditions. Another thing to consider is that adding methods to the class just for the sake of expressing the contract looks like there’s something broken in the design first. As for guidelines, I strongly agree with what has been said, that non-default-constructible and move-constructible are somewhat antagonist and should raise a red flag. As for usefulness, in an ideal world you would like the compiler (or a static analysis tool) to check the contracts for you. Regarding move semantics, that’s the choice made by the rust compiler (reusing a moved-from object is a compile time error). In C++, reusing a moved-from object is perfectly valid, so you would have to resort to other mechanisms to forbid it. IMHO a tool-friendly way (which, by the way, is also user-friendly) to do that is using unique_ptr : the rule to check (nullptr dereference) is much more likely to be implemented, the users are much more likely to write x != nullptr preconditions. I’m not sure there are many cases where you, at the same time : - need the object to be not constructible (always valid) - need to transfer the ownership of the object - can’t afford the extra cost of a pointer (which is, by the way, not more expensive than checking an is_valid() function). Regards, Julien
participants (6)
-
Andrzej Krzemienski -
Barrett Adair -
Gavin Lambert -
Julien Blanc -
Lorenzo Caminiti -
Peter Dimov