Re: [boost] Review of a safer memory management approach for C++?
Hello, This is the last email that I will write and respond to on this tread of emails about runtime vs. compile-time polymorphism (we have beet this dead horse about enough I think). I will summarize my arguments and the baulk responses from people. My perspective comes from having developed and maintained a templated package in Trilinos called Thyra that uses only the most basic type of templating; templating runtime abstract interfaces on the single Scalar type (i.e. double, float, complex<float>, complex<double>, etc). This is the most trivial and basic type of templating that you can possibly imagine (and was the type of templating that templates in C++ was supposed to support in the first place). Templating all of this code was not my first choice but in retrospect it was the right thing to do since you would have to duplicate all of the code (source and binary) multiple times (once for each scalar type) in order to support this variety of scalar types in C++ in a single program. However, this has not been easy by any stretch of the imagination. Now Trilinos is redoing is core linear algebra and linear solvers with the advent of the more heavily templated Tpetra package and will have similar issues. Up until recently, I was just having to compile and link Thyra and client code using the implicit template instantiation mechanism on the *wide* variety of compilers that we are contractually obligated to support including several different versions of GCC, Intel, Sun, PGI, IBM, Pathscale, and Visual C++ on a whole bunch of platforms (Linux, various flavors of Unix, Windows, specialty HPC systems, etc.). In many ways, I think that I have been the unfortunate first adopter of heavily templated code on many of these platforms. Eventually, I figured out how to portably and maintainability manage explicit template instantiation (see GCG 28 in http://www.cs.sandia.gov/~rabartl/ThyraCodingGuideLines.pdf) which has allowed me to port this templated code to several platforms where it would otherwise just segfault or crash the compilers (e.g. Sun and PGI). Going to carefully orchestrated explicit template instantiation has allowed me to recover a lot of the positive properties of non-templated code (i.e. fast recompile times, being able to compile just a single source files to work out compile problems, etc.). However, even with this most trivial type of templating (a single template argument), explicit instantiation is not automatic to support and significantly complicates the design and maintenance of the software. For new more heavily templated packages like Tpetra and the next generation of template software based on Tpetra in Trilinos that is being written, it is even more difficult and more complex to support explicit instantiation. If have fought the good fight for the use templated code in this non-ideal HPC environment for many years and I have earned to right to express my experiences and wisdom from the trenches on this topic. In return, the majority of the comments have basically been: a) Common response from list members: If have had trouble with templated code, it is because I don't know how to design or write software (just read some of the comments below and in some of the other email responses). My response: You can't get more simple templated code than a single Scalar template argument that is used in Thyra. I can't imagine a more basic use of templating and I have no clue how to design this "better". The only way to make this portable is to use explicit template instantiation which is not completely trivial to support, even for the most simplistic type of templating (i.e. just one Scalar template argument). b) Common response from list members: I should not bother supporting troublesome compilers like Sun, IBM, and PGI. My response: As I have explained many times, if our code can't support *all* of these platforms, then our customers can't use any of this code and we need to find another job. I don't know how much more direct I can state this. I have fought many battles against our customers who demand to use bad C++ compilers but we are going to have one or two bad compilers that we need to support for many years to come that throws a monkey wrench into the otherwise "perfect design" of the software we would like to write. You should see the emails that I get from people about "being in an ivory tower" when I complain about having to support non-ANSI C++ compilers. If some of my responses have been testy, it is because I am a little warn out with the above two general responses to problems that I have had with heavily templated code. Please don't just dismiss my opinions based on hard fought experience by assuming that I have no idea what I am doing. This entire thread of many emails and responses was sparked by my original request to review the approach described in: http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf which really has nothing to do with runtime vs. compile-time polymorphism. There has been a lot of responses but has anyone taken the time to even read at least the reduced table of contents described in the Preface? I know this is a long document at over 150 pages in the main text but the reduced table of contents is only something like 45 pages or so. A few selected responses to some other emails below ...
[Bartlett, Roscoe A] As explained above, I fight the battle all the time to avoid having to support such bad non-ANSI compilers but we will have to live with these problems for many years to come. If we can't support *all* of these platforms, then they can't use *any* of the code that does not port to all of these platforms, period. This is not a joke. I suspect that Trilinos and other HPC projects are not the only software projects that have to live this reality.
[Bartlett, Roscoe A] 99% of Thyra-based code should not ever be inlined (see description of Thyra above or just go look at the Thyra Website).
[Bartlett, Roscoe A] If you templated the algorithms instead of using Thyra interfaces, you would have new and different object code for every concrete set of operator/vector subclasses. Just in Trilinos this would result in at least a 3x increase in the size of the object code. There is no way around this.
[Bartlett, Roscoe A] Using Thyra as the example, if you throw away the abstract interfaces, then 99% of your algorithmic code would be templated in order to achieve (compile-time) polymorphic behavior. This is a very simple issue to analyze.
[Bartlett, Roscoe A] The only way to manage large amounts of templated code is to explicitly instantiate it which is not trivial to code and maintain (again, see above).
[Bartlett, Roscoe A] What is this supposed to mean?
[Bartlett, Roscoe A] As described at the top, when the common response to problems is that "you don't know how to design software" one has almost no choice but to respond in such a way. Again, the templating in Thyra could not be more simple.
[Bartlett, Roscoe A] As described at the very top, I fight the battle against bad C++ compilers constantly but we are going to have to live with some bad compilers for many years to come. What do you expect us to do? Should I just quit my job as a protest against bad C++ compilers?
[Bartlett, Roscoe A] Not true. As described in Section 6.1 in: http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf the idioms using the Teuchos MM classes can yield better self documenting code than you can get with languages like Java and Python. Did you bother to read the document? Yes I know it is very long but I think there is some worthwhile material contained in it that should be given a far shake and not just dismissed by uninformed email responses. (By uninformed, I mean you likely did not ready the document, not that you are fundamentally uninformed). - Ross
On 6/7/10 9:49 AM, Bartlett, Roscoe A wrote:
Interesting thread... I think the paragraph above more or less sums up your predicament. Simply, you can't use modern C++ because you are obligated to support poorly conforming compilers. That's reasonable. But any other rationale on top of that (relating to heavy template use, static vs runtime polymorphism, etc.) is really irrelevant because of your primary requirement above. Luckily for most of us here, we are free from such obligations and requirements :-) Cheers, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
participants (2)
-
Bartlett, Roscoe A -
Joel de Guzman