On Thu, Oct 30, 2014 at 10:23 AM, Adam Wulkiewicz <adam.wulkiewicz@gmail.com
wrote:
Paul A. Bristow wrote:
So I developed a simple program downloading summaries, test logs and saving the
enhanced result. You may check it out here: https://github.com/awulkiew/summary-enhancer
Neat :-)
+1
I'd certainly find this very helpful when checking the Boost.Math test monster matrix.
Yes, it's big and various kind of errors can be found there.
FYI, it has some new features. It can keep a log about the tests previously detected as failing and generate a report about new failures. It's also possible to send an email containing this report (to many recipients). It uses SMTP without authentication and e.g. Google throws such messages into SPAM, but a filter can be set for them. Alternatively SPF record could be configured in the machine's domain but I haven't played with it. Some local SMTP server could also probably work. Currently the mailing configuration must be placed in the file mail.cfg containing per-line settings (see the example in the repo). So if run periodically the program should notify about new regressions.
That's also very helpful! The big question in my mind is whether we should enhance the current regression testing system or start fresh using one of the continuous integration frameworks? --Beman