On Thursday 12 February 2004 07:38 am, Martin Wille wrote:
Hartmut Kaiser wrote:
What about generating an email to the maintainers of the corresponding library, which makes sure they will know about a failed test ... :-)
They do this with GCC, and Mark Mitchell has credited it with improved release speed and quality. When something breaks, everyone who has checked in some code since the last successful test run gets a complaining e-mail every day until it gets fixed.
I often had failures due to problems with my local setup. Emails about failed tests should not be sent without being checked by a human before. (Which main- tainer would want to be notified about my disk-full errors?)
Do these failures typically result in a huge number of problems? Perhaps a simple threshold, such as "if there are more than X new failures, e-mail the maintainer of the tests; otherwise, email the people that checked in code last".
I reported some problems several times (sometimes to the list sometimes directly to the maintainer). This didn't always result in the problem being fixed. I'm afraid automatically generated emails would be ignored even more.
The advantage of automatically-generated e-mails is that they would come quickly. For instance, I broken function on GCC 2.95.3 two days ago. I'd looked at a few (but not all) regression results, and hadn't seen any problems, so I forgot about it. Had it not been for David A. telling me directly that I broke something, I would not have known until a release came up and people started looking at regression results more carefully. It's much easier to fix code when you write it the first time, instead of months later in a time crunch. Automatically-generated emails can help that. Doug