[BoostCon07][testing] Session reminder.
Boost Community, The Boost Conference will be here in very short order! And I need to get feedback for the "Testing Boost" sprint we'll be doing <http://boostcon.com/program/sessions#rivera-testing-sprint>. If you have any ideas, concerns, off the wall comments, etc. at *minimum* send them directly to me at "rrivera/acm.org". Why that account? Well because that will be the one account I can guarantee I will access to during the conference, and hence people can send me stuff up to the last minute. Not that I'm encouraging lateness, but I know how busy we all are ;-) If you are going to send it to the Boost dev list you will also want to eventually send me a summary follow up, if needed, from the ensuing discussions. Looking forward to seeing some of you at Aspen! -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
on Wed May 02 2007, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
Boost Community,
The Boost Conference will be here in very short order! And I need to get feedback for the "Testing Boost" sprint we'll be doing <http://boostcon.com/program/sessions#rivera-testing-sprint>. If you have any ideas, concerns, off the wall comments, etc. at *minimum* send them directly to me at "rrivera/acm.org". Why that account? Well because that will be the one account I can guarantee I will access to during the conference, and hence people can send me stuff up to the last minute. Not that I'm encouraging lateness, but I know how busy we all are ;-)
If you are going to send it to the Boost dev list you will also want to eventually send me a summary follow up, if needed, from the ensuing discussions.
Okay, here are some issues I think ought to be solved, in no particular order, some (much) more important than others: * inaccurate header dependency tracing impedes useful incremental testing * report usability is poor for most constituencies. This point could be explored in much more detail. * no mechanism to pinpoint the checkin that caused a regression * no mechanism for communicating that tests are known not to work on some platform (toolset+os) to the build system, and so avoid building them * generating XML by parsing the jam log is fragile and prevents the use of multiple build processes (-jN). This one should be almost embarassingly easy to fix. * The smallest granularity of test that any tester can contribute is the entire Boost suite for one toolset, which makes for longer turnaround. * No way for a developer to request testing results for a specific branch * No "nanny system" that bothers the developer who causes a regression until it is fixed. I think I've thought of others from time-to-time; these are just the ones off the top of my head this morning. -- Dave Abrahams Boost Consulting www.boost-consulting.com Don't Miss BoostCon 2007! ==> http://www.boostcon.com
David Abrahams wrote:
Okay, here are some issues I think ought to be solved, in no particular order, some (much) more important than others:
[...] * No way for a test to fail with a "skipped" status (gray box). This is similar to
* no mechanism for communicating that tests are known not to work on some platform (toolset+os) to the build system, and so avoid building them
but it can rely on Boost.Config macros, which may not be easily accessible to the build system.
(sorry, dropped the cc by mistake) David Abrahams wrote:
Okay, here are some issues I think ought to be solved, in no particular order, some (much) more important than others:
[...] * No way for a test to fail with a "skipped" status (gray box). This is similar to
* no mechanism for communicating that tests are known not to work on some platform (toolset+os) to the build system, and so avoid building them
but it can rely on Boost.Config macros, which may not be easily accessible to the build system.
on Fri May 04 2007, "Gennadiy Rozental" <gennadiy.rozental-AT-thomson.com> wrote:
"David Abrahams" <dave@boost-consulting.com> wrote in message news:876478d51y.fsf@valverde.peloton...
* No way for a developer to request testing results for a specific branch
And against particular branch.
How is "for a specific branch" different from "against a particular branch?" it seems as though you're repeating what I meant to say (?)
That's what independent versioning should address.
I don't see a relationship between the two issues. The ability to request test results against a specific branch has obvious utility; you can try out changes and see how portable they are without disturbing general test results. Independent versioning, by it self, does not address that need. You'd still need to implement all the same testing infrastructure, so I don't see the relevance here. -- Dave Abrahams Boost Consulting www.boost-consulting.com Don't Miss BoostCon 2007! ==> http://www.boostcon.com
"David Abrahams" <dave@boost-consulting.com> wrote in message news:87zm4k87rw.fsf@valverde.peloton...
on Fri May 04 2007, "Gennadiy Rozental" <gennadiy.rozental-AT-thomson.com> wrote:
"David Abrahams" <dave@boost-consulting.com> wrote in message news:876478d51y.fsf@valverde.peloton...
* No way for a developer to request testing results for a specific branch
And against particular branch.
How is "for a specific branch" different from "against a particular branch?" it seems as though you're repeating what I meant to say (?)
My library depend in lib A, B, C. I want to test against new_dev branch of A, HEAD branch of B and 1_33 branch of C.
That's what independent versioning should address.
I don't see a relationship between the two issues. The ability to request test results against a specific branch has obvious utility; you can try out changes and see how portable they are without disturbing general test results. Independent versioning, by it self, does not address that need. You'd still need to implement all the same testing infrastructure, so I don't see the relevance here.
Yes. independent versioning has little to do with "testing for a specific branch". But has everything to do with "testing against particular branch" ;) Gennadiy
on Sat May 05 2007, "Gennadiy Rozental" <gennadiy.rozental-AT-thomson.com> wrote:
"David Abrahams" <dave@boost-consulting.com> wrote in message news:87zm4k87rw.fsf@valverde.peloton...
on Fri May 04 2007, "Gennadiy Rozental" <gennadiy.rozental-AT-thomson.com> wrote:
"David Abrahams" <dave@boost-consulting.com> wrote in message news:876478d51y.fsf@valverde.peloton...
* No way for a developer to request testing results for a specific branch
And against particular branch.
How is "for a specific branch" different from "against a particular branch?" it seems as though you're repeating what I meant to say (?)
My library depend in lib A, B, C. I want to test against new_dev branch of A, HEAD branch of B and 1_33 branch of C.
So assemble a branch that contains everything you want, check it in, and request the test. SVN is really good for that sort of thing because of the way it copies "by reference".
That's what independent versioning should address.
I don't see a relationship between the two issues. The ability to request test results against a specific branch has obvious utility; you can try out changes and see how portable they are without disturbing general test results. Independent versioning, by it self, does not address that need. You'd still need to implement all the same testing infrastructure, so I don't see the relevance here.
Yes. independent versioning has little to do with "testing for a specific branch". But has everything to do with "testing against particular branch" ;)
I still don't see it, sorry. -- Dave Abrahams Boost Consulting www.boost-consulting.com Don't Miss BoostCon 2007! ==> http://www.boostcon.com
"David Abrahams" <dave@boost-consulting.com> wrote in message news:87tzur78cq.fsf@valverde.peloton...
on Sat May 05 2007, "Gennadiy Rozental" <gennadiy.rozental-AT-thomson.com> wrote:
My library depend in lib A, B, C. I want to test against new_dev branch of A, HEAD branch of B and 1_33 branch of C.
So assemble a branch that contains everything you want, check it in, and request the test. SVN is really good for that sort of thing because of the way it copies "by reference".
Does it means that my branch will keep track of changes made in referenced branches? If not it's useless in practice. Gennadiy
on Sun May 06 2007, "Gennadiy Rozental" <gennadiy.rozental-AT-thomson.com> wrote:
"David Abrahams" <dave@boost-consulting.com> wrote in message news:87tzur78cq.fsf@valverde.peloton...
on Sat May 05 2007, "Gennadiy Rozental" <gennadiy.rozental-AT-thomson.com> wrote:
My library depend in lib A, B, C. I want to test against new_dev branch of A, HEAD branch of B and 1_33 branch of C.
So assemble a branch that contains everything you want, check it in, and request the test. SVN is really good for that sort of thing because of the way it copies "by reference".
Does it means that my branch will keep track of changes made in referenced branches? If not it's useless in practice.
You could do that with svn:externals.
David Abrahams writes:
Okay, here are some issues I think ought to be solved, in no particular order, some (much) more important than others:
* inaccurate header dependency tracing
... and the fact that Jamfiles/rule files themselves are not included as dependencies (see "Incremental testing is not reliable" on http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?Boost.Testing) ...
impedes useful incremental testing
[...]
* generating XML by parsing the jam log is fragile and prevents the use of multiple build processes (-jN). This one should be almost embarassingly easy to fix.
And enormously beneficial, IMO. -- Aleksey Gurtovoy MetaCommunications Engineering
participants (5)
-
Aleksey Gurtovoy -
David Abrahams -
Gennadiy Rozental -
Peter Dimov -
Rene Rivera