Re: [boost] Review of a safer memory management approach for C++?
Ingo, Mathias, and David, Sorry I missed your responses. It was not sent to me personally and I am only signed up for boost summaries. Is there some forum where I can present the approach described in: http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf and have an opportunity to discuss the various issues with interested parties and leaders in the C++ community? I am not sure the discussions on this mail list have adequately addressed the issues. My responses are below:
6. Re: Review of a safer memory management approach for C++? (Ingo Loehken) 8. Re: Review of a safer memory management approach for C++? (Mathias Gaunard) 10. Re: Review of a safer memory management approach for C++? (David Abrahams)
Message: 6 Date: Thu, 27 May 2010 20:02:45 +0200 From: Ingo Loehken <Ingo.Loehken@boc-eu.com> To: boost@lists.boost.org Subject: Re: [boost] Review of a safer memory management approach for C++? Message-ID: <OF6B67C15A.1605A772-ONC1257730.00627F93-C1257730.00631B78@boc- eu.com> Content-Type: text/plain; charset="US-ASCII"
if I understand "hard to reason about" in the right why : like there is no need for shared ownership at all, this also means that there is no use for COM Programming - and of course there is. Raw resp. opaque pointers have one huge advantage in productive environments : They work as perfect isolators, removing any compile time dependencies. And from my point of view, compile time is what matters most, if you go productive, otherwise you end up waiting for recompiles.
Ingo, These are exactly the right points to be made. Many complex programs are going to involve sharing objects. We see this in many programming environments. Trying to "design" our way out of the sharing problem entirely will create more problems that it tries to solve. We need to separate the issues of sharing. One set of issues relates to change prorogation that will happen in any programming language (e.g. if objects A and B share object C and if A changes C what happens to B?). This is where we should be focusing our design attention. The other issue is silly memory management problems in C++ that other more modern languages like C#, Python, and Java don't suffer from so much. Objects stay around as long as they are needed. The goal of the approach described in: http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf is to as much as possible make the presence of sharing explicit so that we highlight the first type of "essential complexity" in programs but to minimize as much as possible the "accidental complexity" in trying to figure out when we need to actually delete an object and who's responsibility it is to delete the object. This is a very important issues and is discussed some in Section 6.1 in: http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf Cheers, - Ross
From: Mathias Gaunard <mathias.gaunard@ens-lyon.org> To: boost@lists.boost.org Date: 27.05.2010 14:59 Subject: Re: [boost] Review of a safer memory management approach for C++? Sent by: boost-bounces@lists.boost.org
Bartlett, Roscoe A wrote:
Hello Boost developers,
I am interested in finding one or more individuals who are knowledgeable about memory management in C++ (and especially of the reference- counting approach taken by boost::shared_ptr and boost::weak_ptr) to review an idea for a comprehensive approach to safer memory management in C++ that encapsulates all raw C++ pointers in high-level code.
Never use owning naked pointers and only use RAII (as is required for exception-safe programming anyway) with exclusive ownership and no aliasing, and you have no problems.
[Bartlett, Roscoe A] Mathias, I agree but using raw pointers in any ubiquitous way is a problem. You can't detect dangling references and there are all of the other loosely typed problems described in Sections 2.1 and 2.2 in: http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf The issue is that it is easy to make mistakes with raw pointers related to ownership issues and if you use raw pointers, it is very difficult to detect them, even for tools like Valgrind and Purify. If no-one made mistakes, then we would all do well just to use void* and program in C. I want to eliminate undefined behavior in C++ programs in such a way that a properly configured static analysis tool could help to enforce. I believe that the approach described in http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf could be used with some static analysis tool to largely eliminate undefined behavior related to the usage of memory. There are lots of other areas of undefined behavior in C++ that this approach can't address (like bad type conversions of built-in types) but there are other idioms for addressing that.
Usage of shared_ptr should be an exception, not a widely deployed solution to memory management issues. Shared ownership is hard to reason about, and even if you use a similar cycle-aware solution, cycles remain a real problem (they prevent deterministic ordered destruction of objects, meaning they're only applicable to certain classes of objects).
[Bartlett, Roscoe A] No, if you share an object, you use a shared_ptr (or Teuchos::RCP), period. If sharing is the exception, then fine. Otherwise, don't play games with "I need shared_ptr here but I don't need it there because X, Y, and Z." We are never going to be able to produce programs that eliminate all sharing of objects, and we don't want to. Some types of programs are actually much simpler and more robust if you share some objects. For performance critical applications (e.g. the CSE domains I come from) you can't just willy nilly copy large objects and expect to achieve good performance. This is just not reality.
Message: 10 Date: Thu, 27 May 2010 14:41:09 -0400 From: David Abrahams <dave@boostpro.com> To: boost@lists.boost.org Subject: Re: [boost] Review of a safer memory management approach for C++? Message-ID: <m24ohtgpbu.wl%dave@boostpro.com> Content-Type: text/plain; charset=US-ASCII
At Thu, 27 May 2010 20:02:45 +0200, Ingo Loehken wrote:
if I understand "hard to reason about" in the right why : like there is no need for shared ownership at all, this also means that there is no use for COM Programming - and of course there is.
I think you don't understand it the right way. Shared ownership (at least in the presence of mutation) is hard to reason about because seemingly-local modifications can have non-local effects.
[Bartlett, Roscoe A] Yes, but let's focus on the "essential complexity" of object sharing and not get bogged down in the "accidental complexity" of memory management. The approaches described in http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf make explicit the "essential complexity" of object sharing while trying to eliminate the "accidental complexity" of memory management (see Section 6.1 in http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf). Trying to over design a program to avoid all shared ownership is what make C++ programming so unproductive and has all the negative consequences described in Section 1 in: http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf Designs with object sharing can be much less complex overall than designs with sharing. You just need decent tools to detect circular reference problems (and that is what the Teuchos::RCP class has). We have found that circular reference problems are fairly rare but when they occur they are a bear to track down. That is why I created all of the RCPNode tracing support in the Teuchos::RCP classes. Actually, we have found that non-deleted RCPNode objects are far more likely to occur due to other types of problems (e.g. calling abort(..), typical memory leaks, etc.) and in this role the RCPNode tracing machinery is a very cheap but effective version of memory checking that is automatically built into every debug-mode program and runs order of magnitude faster than tools like Valgrind and Purify. [Bartlett, Roscoe A] Cheers, - Ross Sorry again for not responding sooner.
At Tue, 1 Jun 2010 12:47:33 -0600, Bartlett, Roscoe A wrote:
Ingo, Mathias, and David,
Sorry I missed your responses. It was not sent to me personally and I am only signed up for boost summaries.
Is there some forum where I can present the approach described in:
http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf
and have an opportunity to discuss the various issues with interested parties and leaders in the C++ community? I am not sure the discussions on this mail list have adequately addressed the issues.
The comp.lang.c++.moderated newsgroup seems like a good bet.
From: David Abrahams <dave@boostpro.com> To: boost@lists.boost.org Subject: Re: [boost] Review of a safer memory management approach for C++? Message-ID: <m24ohtgpbu.wl%dave@boostpro.com> Content-Type: text/plain; charset=US-ASCII
At Thu, 27 May 2010 20:02:45 +0200, Ingo Loehken wrote:
if I understand "hard to reason about" in the right why : like there is no need for shared ownership at all, this also means that there is no use for COM Programming - and of course there is.
I think you don't understand it the right way. Shared ownership (at least in the presence of mutation) is hard to reason about because seemingly-local modifications can have non-local effects.
[Bartlett, Roscoe A]
Yes, but let's focus on the "essential complexity" of object sharing
Okay.
and not get bogged down in the "accidental complexity" of memory management. The approaches described in
http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf
make explicit the "essential complexity" of object sharing while trying to eliminate the "accidental complexity" of memory management (see Section 6.1 in http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf).
I'm all for making essential complexity obvious and eliminating accidental complexity, but I'm not sure I see a compelling example that Teuchos does that, at least not when compared with what I consider normal state-of-the-art C++ programming, which is mostly value-based, dynamic allocations are rare, and those that occur are immediately managed, e.g. by appropriate smart pointers.
Trying to over design a program to avoid all shared ownership is what make C++ programming so unproductive and has all the negative consequences described in Section 1 in:
http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf
Designs with object sharing can be much less complex overall than designs with sharing. You just need decent tools to detect circular reference problems (and that is what the Teuchos::RCP class has).
Well, I fundamentally disagree with all of the above. Overuse of runtime polymorphism, thus dynamic allocation, and thus shared ownership (it's almost an inevitable progression) is one of the things that has made C++ programming unproductive, and those people I know who most zealously avoid it tend to be more productive than everyone else. IMO. In my experience, designs with object sharing tend to increase complexity in all kinds of ways beyond the memory management issues that you describe. Perhaps the most important way is, they tend to obscure the overall computation being performed by hiding it in networks of interacting objects. I would like to see an example of a design with shared object ownership that is “much less complex” than all other functionally equivalent designs that eschew shared ownership. I'm not saying such applications don't exist, but IME they are rare, and a concrete example would help to bring this whole discussion back to earth. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Hi David,
Trying to over design a program to avoid all shared ownership is what make C++ programming so unproductive and has all the negative consequences described in Section 1 in:
http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf
Designs with object sharing can be much less complex overall than designs with sharing. You just need decent tools to detect circular reference problems (and that is what the Teuchos::RCP class has).
Well, I fundamentally disagree with all of the above. Overuse of runtime polymorphism, thus dynamic allocation, and thus shared ownership (it's almost an inevitable progression) is one of the things that has made C++ programming unproductive, and those people I know who most zealously avoid it tend to be more productive than everyone else. IMO.
I couldn't agree more. C++ best practices has been evolving from dynamically-polymorphic designs to statically-polymorphic ones, but rather slowly, so I don't expect to see most competent programmers do the mental migration just yet. This requires a mindshift comparable to that of structured -> OO programming. I even wonder if this isn't a new paradigm altogether. For instance, it's been so long since I last wrote a virtual function that recently, I made the silly mistake of forgetting that, in a derived class, an overrided function doesn't have to specify the virtual keyword ;) Anyway. Speaking of value-semantics and moving (as opposed to sharing) objects... I recall a CUJ article in the mid 90's with a smart pointer offering a handle/body solution for value-semantics of dynamically allocated objects. I don't remember the details (such as using COW or what), nor the author, but I do remember that it was, IMO, simply brillant, but way ahead of its time. Compilers at the time could barely template two and two toghether, so the actual utility was rather impractical. I just thought that, maybe, C++0x finally got what it takes to tackle on something like that? IMO this would make for a terrific boost project, to the level of what shared_ptr used to be 15 years ago. Anyone? (I can't get into something like that without a time machine I'm afraid :) Best -- Fernando Cacciola SciSoft Consulting, Founder http://www.scisoft-consulting.com
participants (3)
-
Bartlett, Roscoe A -
David Abrahams -
Fernando Cacciola