The only thing I would say, is that if the number of runners is too high, there's a danger of developers being drowned in data. For me the most interesting variants would be the machine architechture ones (currently most of the tests are being run on Intel), and the least interesting would be C++ dialect ones as we already have good coverage of that, and I assume there's not much difference between GCC in C++11 mode on Andriod to GCC on Linux or whatever?
Well, they shouldn't differ; however, how can you know that without thorough testing? Please note Android is not the Linux. Linux is very stable compared to Android. Many (even obvious) things works another way on Android due to highly non-standard Android libc (Bionic). Also, compilers are built from own sources, which are synced with upstream (from time to time), but no one can guarantee there would be the same behavior for GCC in C++11 mode on Android and GCC (the same version) on Linux. Nevertheless, even if C++ dialects doesn't mean so much for you (and may be for others), testing on different Android versions still produce really big matrix of variants. What about that? -- Dmitry Moskalchuk