Guys,
As you may know, I'm working on automatic regression testing of Boost on Android. I spent several weeks debugging it and now it seems to be close to the finish - i.e. we'll have soon fully automatic 24x7 regression testing with uploading all results to the Boost public FTP. As you can see, some results are already uploading on a regular basis - http://www.boost.org/development/tests/master/developer/summary.html (there are results of just few libraries at the current moment since I've limited testing by only those libraries to save time needed for debugging testing scripts; as far as I finish with my debugging, I'll enable testing of all libraries).
However, I've realized that for thorough testing matrix of the variants become really big. Look, right now there are nine runners for Android, and this number is in fact very limited. They differs by target ABI (three arm variants, x86 and x86_64) and version of Android (API level 19 - Android 4.4 and 21 - Android 5.0). However, it has sense to test also on Android 4.0, 4.1, 4.2 and 4.3 since theirs market share is still large (see https://developer.android.com/about/dashboards/index.html). Add to that MIPS target ABI (not yet included to the testing) and multiple by two to have tests running with both default settings and with -std=c++11 - and you'll get really big total number of runners.
I'm asking for advice from community. It's not the big problem for us to run all such tests in all variants, but I'm unsure if it will be acceptable for Boost community to see such wide table of runners. I'm afraid it will look like a flood. We also publish Android-only results on https://boost.crystax.net/master/developer/summary.html and we'll definitely display all results from all runners there. Please let me know if the same approach would work for http://www.boost.org/development/tests/master/developer/summary.html or should I limit somehow runners to be uploaded to the Boost FTP.
The only thing I would say, is that if the number of runners is too high, there's a danger of developers being drowned in data. For me the most interesting variants would be the machine architechture ones (currently most of the tests are being run on Intel), and the least interesting would be C++ dialect ones as we already have good coverage of that, and I assume there's not much difference between GCC in C++11 mode on Andriod to GCC on Linux or whatever? John.