vicente.botet wrote:
----- Original Message ----- From: "Johan Nilsson" <r.johan.nilsson@gmail.com>
"vicente.botet" <vicente.botet@wanadoo.fr> skrev i meddelandet news:3E6A91215D384FF9AD7BFB070DDD16F7@viboes1...
----- Original Message ----- From: "Johan Nilsson" <r.johan.nilsson@gmail.com> To: <boost@lists.boost.org> Sent: Friday, November 21, 2008 1:02 PM Subject: Re: [boost] Breaking existing libraries
Just an idea that popped out of my head - what about running (or at least compiling) the unit/regression tests from the _previous_ major release against the current release as a way to automatically detect and flag interface/behavioral changes?
Sure, there are many potential problems with such an approach:
- Some library tests are likely to also excercise code at the implementation/detail level. - Already limited testing resources (on the other hand, interface changes are likely to be detected by running such tests for a single, reasonably compliant, compiler). - ...
In order to have the better the author can preserve the test of the older releases and run them on the current release.
Sorry, I don't follow you.
Daniel Walker has expresed this better "Once accepted, the tests should be a verification that the library does what the community voted on. The tests are a verification of quality. At that point, I think it might be a good idea to quarantine the tests, take them out of the authors hands, and put them under the stewardship of a benevolent dictator of boost as a whole so that they can be used to assure that the library does what the community voted on. "
Sounds like a good idea in general. Reference, please.
If we need to change while we make evolulion on a library this is a symptom the interface has changed and the same way the test is broken, the user code canbe broken. If we forbid this test changes, we are able to identify breaking changes.
In order to truly make such a thing work, I think it would be neccessary to separate interface and implementation tests. There is almost always a need for testing implementation details, which should be free for the author to change (and improve upon). How do you think extensions to the interface should be handled? / Johan