I have done some investigation and it may be possible for OSL to provide a testing farm for Boost developers. OSL would like to know what kind of hardware, OS, compilers are most needed. Please post your requests in this thread. -- Dave Abrahams Boost Consulting www.boost-consulting.com
On Sun, 20 Mar 2005 18:09:27 -0500, David Abrahams wrote
I have done some investigation and it may be possible for OSL to provide a testing farm for Boost developers. OSL would like to know what kind of hardware, OS, compilers are most needed. Please post your requests in this thread.
This might seem kind of obvious, but our current platform/compiler set excluding compiler/OS versions is: Windows VC, Intel, Borland, Code Warrior Linux GCC, Intel Solaris GCC, (could add Sun compiler if they fix it) MAC GCC, Code Warrior So I'd say by default this is the 'highest priority' set. That's a minimum of 4 machines and 9 compilers to cover our current set. We should probably count on 2 compiler versions for each vendor making 18 compilers. If we were going to get crazy, we might want 2 windows machines to support the long list of compilers (btw, we also seem to have lost our MingW tester -- did we ever have a cygwin tester?) Seems like the real variation here is in OS's. Of course, there are hundreds of Linux distros so Linux isn't just Linux -- not sure which one to pick there. There's other platforms some people might like such as BSD (see http://lists.boost.org/MailArchives/boost/msg77524.php) Not to mention HP, AIX, etc. BTW, maybe someone should ask some of these companies to provide the tools and a platform to OSL -- it's all to thier benefit if Boost works with their platform. Certainly IBM has been a big open source supporter recently... Obviously the C++ community is going to owe a big debt to OSL -- let me revise that -- an even bigger debt! Jeff
Yes you're right -- as I mentioned in the paragraph, however, we seem to have lost our MingW tester currently.
How long do the regressions tests normally take to run? I have an idle computer for 10-12hrs or so each night which could quite happily run the mingw tests. Also is it possible to execute distributed builds? (via distcc or similar) Martin
Martin Slater wrote:
Yes you're right -- as I mentioned in the paragraph, however, we seem to have
lost our MingW tester currently.
How long do the regressions tests normally take to run? I have an idle computer for 10-12hrs or so each night which could quite happily run the mingw tests. Also is it possible to execute distributed builds? (via distcc or similar)
Well, that obviously depends a lot on the speed of the computer. For a single toolset to test with, 10-12h are definitely sufficient, even on a slightly outdated CPU and with rebuilding everything from scratch. Don't run the tests with less than .5 GB of RAM. I don't think anybody runs tests using distcc. However, I know at least Boost 1.31.0 can be built using distcc and with -j4 (someone told me that just yesterday on IRC). IIUC, there's nothing that should prevent distcc from working fine with Boost.Build (of course, you need sufficiently similar boxes). In fact, it would be a good thing to test with distcc. HTH, m
Well, that obviously depends a lot on the speed of the computer.
Well yes;) Ballpark figure was all i was after, the computer is a p4 3.4ghz.
For a single toolset to test with, 10-12h are definitely sufficient, even on a slightly outdated CPU and with rebuilding everything from scratch. Don't run the tests with less than .5 GB of RAM.
Got a gb so that would be fine.
I don't think anybody runs tests using distcc. However, I know at least Boost 1.31.0 can be built using distcc and with -j4 (someone told > me that just yesterday on IRC). IIUC, there's nothing that should prevent distcc from working fine with Boost.Build (of course, you need sufficiently similar boxes). In fact, it would be a good thing to test with distcc.
Good to hear. cheers Martin
Martin Slater writes:
~2-4 hours for a single toolset (i.e. compiler version).
No problem then, if you need another tester
By all means!
(mingw and vc if you need it, we also have intel compiler around) just let us know and i'll try and configure the box during the week.
I think we have "plain" Intel and VC 7.1/8.0 covered, but if you are interested in testing something like VC + STLPort, it'd be a welcome contribution, too. Thanks for the offer! -- Aleksey Gurtovoy MetaCommunications Engineering
I think we have "plain" Intel and VC 7.1/8.0 covered, but if you are interested in testing something like VC + STLPort, it'd be a welcome contribution, too.
Thanks for the offer!
No problem ;) I've not used VC + stlport before but I'll look into setting it up when I've setup the regression scripts for mingw. I'll probably be back with some questions soon. Martin
Martin Slater writes:
I think we have "plain" Intel and VC 7.1/8.0 covered, but if you are interested in testing something like VC + STLPort, it'd be a welcome contribution, too. Thanks for the offer!
No problem ;) I've not used VC + stlport before but I'll look into setting it up when I've setup the regression scripts for mingw. I'll probably be back with some questions soon.
Just to make sure you start at the right place: http://www.meta-comm.com/engineering/regression_setup/instructions.html -- Aleksey Gurtovoy MetaCommunications Engineering
Just to make sure you start at the right place: http://www.meta-comm.com/engineering/regression_setup/instructions.html
Thanks, setting it up now. I'll take any questions I have to the test list. Martin
Martin Wille wrote:
Martin Slater wrote:
Yes you're right -- as I mentioned in the paragraph, however, we seem to have
lost our MingW tester currently.
How long do the regressions tests normally take to run? I have an idle computer for 10-12hrs or so each night which could quite happily run the mingw tests. Also is it possible to execute distributed builds? (via distcc or similar)
I don't think anybody runs tests using distcc. However, I know at least Boost 1.31.0 can be built using distcc and with -j4 (someone told me that just yesterday on IRC). IIUC, there's nothing that should prevent distcc from working fine with Boost.Build (of course, you need sufficiently similar boxes). In fact, it would be a good thing to test with distcc.
There's one thing that prevents using distcc.. Using the -j# option causes the build log to get scrambled, because there's no synchronization for that use case in BBv1. So the processing of the log for the result generation can't deal with it. If I remember correctly Volodya put in handling for this into BBv2. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com - 102708583/icq
Rene Rivera wrote:
Martin Wille wrote:
[...]
In fact, it would be a good thing to test with distcc.
There's one thing that prevents using distcc.. Using the -j# option causes the build log to get scrambled, because there's no synchronization for that use case in BBv1. So the processing of the log for the result generation can't deal with it. If I remember correctly Volodya put in handling for this into BBv2.
Ouch. You're right, of course. A very good reason to switch to BBv2 as soon as possible. Regards, m
Martin Wille <mw8329@yahoo.com.au> writes:
Rene Rivera wrote:
Martin Wille wrote:
[...]
In fact, it would be a good thing to test with distcc.
There's one thing that prevents using distcc.. Using the -j# option causes the build log to get scrambled, because there's no synchronization for that use case in BBv1. So the processing of the log for the result generation can't deal with it. If I remember correctly Volodya put in handling for this into BBv2.
Ouch. You're right, of course.
A very good reason to switch to BBv2 as soon as possible.
Actually, not quite. I do support switching, but BBv2 is assembling the log in the same way as BBv1, at least for the time being. Hopefully it will be easier to build smart log assembly in BBv2, though. -- Dave Abrahams Boost Consulting www.boost-consulting.com
Rene Rivera wrote:
I don't think anybody runs tests using distcc. However, I know at least Boost 1.31.0 can be built using distcc and with -j4 (someone told me that just yesterday on IRC). IIUC, there's nothing that should prevent distcc from working fine with Boost.Build (of course, you need sufficiently similar boxes). In fact, it would be a good thing to test with distcc.
There's one thing that prevents using distcc.. Using the -j# option causes the build log to get scrambled, because there's no synchronization for that use case in BBv1. So the processing of the log for the result generation can't deal with it. If I remember correctly Volodya put in handling for this into BBv2.
No, I did not ;-) The only -j related change I ever made is a mechanism to serialize execution for certain actions. I indent to use it for linking, which is IO bound and can't benefit from multiple CPUs. - Volodya
"Aaron W. LaFramboise" <aaronrabiddog51@aaronwl.com> wrote in message news:423E19C7.1040206@aaronwl.com... | Jeff Garland wrote: | | > Windows VC, Intel, Borland, Code Warrior | | Don't neglect GCC on Windows. :-) and digital mars. -Thorsten
This mail is forwarded to some of my vendor-contacts (in bcc) to see if they would be interested in this effort (hi guys, this is a follow-up on a thread on the boost-ml). I had already similar discussions with them on this topic but if vendors run the regression tests for instance, are they considered to be 'impartial' etc. So this might be a solution. Additionally I must say that we are almost up and running again with the regression tests on IBM and HP (I've been saying that for a long time, I know). I'm leaving on a 10-day ski-vacation though but hope to be back in the regression business again in a month. toon Jeff Garland wrote:
On Sun, 20 Mar 2005 18:09:27 -0500, David Abrahams wrote
I have done some investigation and it may be possible for OSL to provide a testing farm for Boost developers. OSL would like to know what kind of hardware, OS, compilers are most needed. Please post your requests in this thread.
This might seem kind of obvious, but our current platform/compiler set excluding compiler/OS versions is:
Windows VC, Intel, Borland, Code Warrior Linux GCC, Intel Solaris GCC, (could add Sun compiler if they fix it) MAC GCC, Code Warrior
So I'd say by default this is the 'highest priority' set. That's a minimum of 4 machines and 9 compilers to cover our current set. We should probably count on 2 compiler versions for each vendor making 18 compilers. If we were going to get crazy, we might want 2 windows machines to support the long list of compilers (btw, we also seem to have lost our MingW tester -- did we ever have a cygwin tester?)
Seems like the real variation here is in OS's. Of course, there are hundreds of Linux distros so Linux isn't just Linux -- not sure which one to pick there. There's other platforms some people might like such as BSD (see http://lists.boost.org/MailArchives/boost/msg77524.php) Not to mention HP, AIX, etc. BTW, maybe someone should ask some of these companies to provide the tools and a platform to OSL -- it's all to thier benefit if Boost works with their platform. Certainly IBM has been a big open source supporter recently...
Obviously the C++ community is going to owe a big debt to OSL -- let me revise that -- an even bigger debt!
Jeff
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Solaris GCC, (could add Sun compiler if they fix it)
Apparently release 10 of Sunpro will fully support Boost, see: https://sourceforge.net/tracker/?func=detail&atid=107586&aid=1155439&group_id=7586. John.
"John Maddock" <john@johnmaddock.co.uk> wrote in message news:03b001c52e07$8ee04b80$d8200d52@fuji...
Solaris GCC, (could add Sun compiler if they fix it)
Apparently release 10 of Sunpro will fully support Boost, see: https://sourceforge.net/tracker/?func=detail&atid=107586&aid=1155439&group_id=7586.
John, Did the Sunpro 10 patch ever get applied? I'm not seeing it in CVS unless I'm looking at the wrong file. Thanks, --Beman
"John Maddock" <john@johnmaddock.co.uk> wrote in message news:016601c52ecd$d68b8b90$82560e52@fuji...
Did the Sunpro 10 patch ever get applied? I'm not seeing it in CVS unless I'm looking at the wrong file.
It did, but double checking sunpro_cc.hpp, I think I messed it up, the check <= 0x570 should be < 0x570, fixed now.
Thanks for checking, --Beman
On Sun, Mar 20, 2005 at 04:52:23PM -0700, Jeff Garland wrote:
Seems like the real variation here is in OS's. Of course, there are hundreds of Linux distros so Linux isn't just Linux -- not sure which one to pick
But there's only one glibc, which they all use. Different glibc versions shouldn't be _too_ different, for Boost's purposes. No more different than windows NT, 95, 98, 2k etc. for instance. So just pick one, it shouldn't matter which. A distro that doesn't ship with Boost might make sense, since presumably the distros that provide precompiled versions of Boost are already testing it. jon -- There are only 10 types of people in the world - Those who understand binary, and those who don't.
Jonathan Wakely wrote:
On Sun, Mar 20, 2005 at 04:52:23PM -0700, Jeff Garland wrote:
Seems like the real variation here is in OS's. Of course, there are hundreds of Linux distros so Linux isn't just Linux -- not sure which one to pick
But there's only one glibc, which they all use. Different glibc versions shouldn't be _too_ different, for Boost's purposes.
In my experience, they make a difference. And there's not only one glibc, since there are also many versions of it. Tests also depend on the kernel version, on the CPU(s), on the version of the system compiler, on the Python version et cetera. So just pick one, it
shouldn't matter which.
It does matter, unfortunately. A distro that doesn't ship with Boost might make
sense, since presumably the distros that provide precompiled versions of Boost are already testing it.
I doubt any distributor runs the regression tests. In fact, support for Boost packages on various Linux distributions isn't too great. There's no reason to expect distributors to run the tests. Also, it actually doesn't matter whether a distribution tested the Boost version it has a package for. It's an already released version which might even contain patches from the distributor. We're talking about testing from the CVS, though. Regards, m
On Mon, Mar 21, 2005 at 05:37:07PM +0100, Martin Wille wrote:
Jonathan Wakely wrote:
On Sun, Mar 20, 2005 at 04:52:23PM -0700, Jeff Garland wrote:
Seems like the real variation here is in OS's. Of course, there are hundreds of Linux distros so Linux isn't just Linux -- not sure which one to pick
But there's only one glibc, which they all use. Different glibc versions shouldn't be _too_ different, for Boost's purposes.
In my experience, they make a difference. And there's not only one glibc, since there are also many versions of it. Tests also depend on the kernel version, on the CPU(s), on the version of the system compiler, on the Python version et cetera.
Then I stand corrected. (The system compiler surely only matters if you're using the system compiler? If you do parallel installs of five versions of GCC and use them for the tests, does the system compiler matter?) I may well be wrong again, but IMHO aiming for wider coverage (by testing on a couple of versions of Linux and some version of FreeBSD, for example) would pick up more portability problems than testing on a number of subtley different flavours of Linux. I know I've found several problems when I run the tests on FreeBSD that have clearly never showed up on any of the versions of Linux that are tested by other people. Even if it does make quite a difference, it seems obvious to choose from the most popular distros, since that ensures compatibility with a higher number of users. Choosing to run tests on the more obscure distros might find quirks of those distros but it might be that no Boost user ever uses that distro.
A distro that doesn't ship with Boost might make
sense, since presumably the distros that provide precompiled versions of Boost are already testing it.
I doubt any distributor runs the regression tests.
Again, I'm a bit surprised.
In fact, support for Boost packages on various Linux distributions isn't too great. There's no reason to expect distributors to run the tests. Also, it actually doesn't matter whether a distribution tested the Boost version it has a package for. It's an already released version which might even contain patches from the distributor. We're talking about testing from the CVS, though.
If *I* was the maintainer of a distro's Boost package I'd test against CVS, to know what problems might be coming up soon, but I'm not, so my opinion counts for nowt! jon -- "Once, during Prohibition, I was forced to live for days on nothing but food and water." - W.C. Fields
[ cross-posted to the -testing list, where I think this belongs ] Just another data point: Apple ships several versions of GCC. For current shipping systems, they ship v3.3. For the next version of Mac OS (10.4, also known as Tiger), they will ship 3.4 and 4.0. Which one(s) should we be testing with? When should we change? People are testing releases of CodeWarrior for Mac OS X. Mostly v8 and v9. The latest version for each release is 8.3, and 9.4. Again, which version(s) should we test? I'll be setting up a machine this weekend to run regressions; I'm happy to run any/all of these, but I'm looking for guidance. -- -- Marshall Marshall Clow Idio Software <mailto:marshall@idio.com> "I need a girlfriend whose name doesn't end in .JPG"
"Marshall Clow" <marshall@idio.com> wrote in message news:p06210202be64c3d2ec0e@[192.168.16.235]...
[ cross-posted to the -testing list, where I think this belongs ] Just another data point:
Apple ships several versions of GCC. For current shipping systems, they ship v3.3. For the next version of Mac OS (10.4, also known as Tiger), they will ship 3.4 and 4.0.
Which one(s) should we be testing with? When should we change?
People are testing releases of CodeWarrior for Mac OS X. Mostly v8 and v9. The latest version for each release is 8.3, and 9.4.
Again, which version(s) should we test?
I'd say the highest priority is always the most recent shipping (that is, non-beta) version of each compiler. Older compilers, or betas of compilers which haven't been released yet, are interesting, but not at the expense of the latest shipping compiler tests. --Beman
Jonathan Wakely wrote:
In my experience, they make a difference. And there's not only one glibc, since there are also many versions of it. Tests also depend on the kernel version, on the CPU(s), on the version of the system compiler, on the Python version et cetera.
Then I stand corrected.
(The system compiler surely only matters if you're using the system compiler? If you do parallel installs of five versions of GCC and use them for the tests, does the system compiler matter?)
It matters for icc. It also matters if you use Python compiled with the system compiler and with C++ support enabled (like the distro's Python).
I may well be wrong again, but IMHO aiming for wider coverage (by testing on a couple of versions of Linux and some version of FreeBSD, for example) would pick up more portability problems than testing on a number of subtley different flavours of Linux.
Yes, I'm all for testing on a larger variety of systems. If I had a spare box then I'd set up BSD tests here myself.
I know I've found several problems when I run the tests on FreeBSD that have clearly never showed up on any of the versions of Linux that are tested by other people.
Yes, and it is a pity we don't have enough BSD testers.
Even if it does make quite a difference, it seems obvious to choose from the most popular distros, since that ensures compatibility with a higher number of users. Choosing to run tests on the more obscure distros might find quirks of those distros but it might be that no Boost user ever uses that distro.
Well, that depends. If Boost doesn't work on certain distributions then it could be the distro's fault. However, more likely, it's lack of compatibility between Boost and the software versions the distro happens to use. I don't think we can blame a distro for that. (this is different if a distro uses broken software like gcc "2.96")
In fact, support for Boost packages on various Linux distributions isn't too great. There's no reason to expect distributors to run the tests. Also, it actually doesn't matter whether a distribution tested the Boost version it has a package for. It's an already released version which might even contain patches from the distributor. We're talking about testing from the CVS, though.
If *I* was the maintainer of a distro's Boost package I'd test against CVS, to know what problems might be coming up soon, but I'm not, so my opinion counts for nowt!
Please, become a maintainer ;-) Seriously, the problem likely is that a distro's maintainer for Boost also has a bunch of other stuff to maintain. Chances are a maintainer doesn't have the time to properly test a package like Boost from the CVS regularly or even only at (Boost) release preparation times. Regards, m
On Sun, 20 Mar 2005 16:52:23 -0700, Jeff Garland <jeff@crystalclearsoftware.com> wrote:
Solaris GCC, (could add Sun compiler if they fix it)
Sun C++ 5.5 or later should be OK. Not sure what the marketing droid "Forte" version number is that corrsponds to this (Forte 8 maybe?), but thats what "CC -V" should report. I was running the tests on 5.3, which is hopeless. -- Caleb Epstein caleb dot epstein at gmail dot com
I have done some investigation and it may be possible for OSL to provide a testing farm for Boost developers. OSL would like to know what kind of hardware, OS, compilers are most needed. Please post your requests in this thread.
Good timing: I've just inquired of SAP what platforms they support (they're using Boost.Regex inside their NetWeaver product, and to provide the regex functionality inside their next release of the ABAP language) and they're supporting quite a few platforms that we currently don't test: - HP-UX (HP-UX 9000/800) - HP-UX Itanium (ia64) - Linux PPC - Linux OS/390 - Windows 64 Bit (Intel und AMD) - Sun Solaris (Sparc) - AIX - zOS - Alpha (OSF1) - AS/400 I'm don't know what compilers their using, but I'm assuming the platform vendor's native compiler where possible. I'll bcc this to the SAP developers in case they're willing to help run and publish Boost's tests on any of those platforms. John.
"David Abrahams" <dave@boost-consulting.com> wrote in message news:u4qf5vrrs.fsf@boost-consulting.com...
I have done some investigation and it may be possible for OSL to provide a testing farm for Boost developers. OSL would like to know what kind of hardware, OS, compilers are most needed. Please post your requests in this thread.
The test I'd like to see run regularly are for the OS's we don't already have good coverage on. For example, pick several from the list that John Maddock posted: - HP-UX (HP-UX 9000/800) - HP-UX Itanium (ia64) - Linux PPC - Linux OS/390 - Windows 64 Bit (Intel und AMD) - Sun Solaris (Sparc) - AIX - zOS - Alpha (OSF1) - AS/400 I'd also like to see a test or two of a compiler targeting embedded systems. Looking at a couple of vendor web pages, I see "complete STL support" listed, so presumably these compilers could handle Boost too. --Beman
On Mon, 21 Mar 2005 21:27:44 -0500, Beman Dawes wrote
I'd also like to see a test or two of a compiler targeting embedded systems. Looking at a couple of vendor web pages, I see "complete STL support" listed, so presumably these compilers could handle Boost too.
LOL -- now that's optimism! I'm guessing VC6 was advertised the same way -- and as we all know it wouldn't even get started with Boost if it weren't for a lot of hard effort on the developers part to port their code... Jeff
The test I'd like to see run regularly are for the OS's we don't already have good coverage on. For example, pick several from the list that John Maddock posted:
Can I also make a plea for testing 64-bit builds? In the pasts when we've tested 64-bit OS's we've tended to build with default compiler options (32-bit usually), rather than enabling 64-bit pointer models. I've had some obscure bugs go undetected because of that in the past. So maybe if someone wants to set up another linux machine for example, it should be on Intel/AMD64 + SUSE Linux 64, with gcc and Intel's 64 bit compiler? BTW with Solaris 10 now a free download, if we can persuade Sun to offer up a few free copies of their latest toolchain, we really ought to be able to improve testing on that platform. Here's wishing anyway... John.
"John Maddock" <john@johnmaddock.co.uk> wrote in message news:016501c52ecd$d61d8780$82560e52@fuji...
BTW with Solaris 10 now a free download, if we can persuade Sun to offer up a few free copies of their latest toolchain, we really ought to be able to improve testing on that platform.
Here's wishing anyway...
Maybe several of us can gang up on the Sun folks at the C++ committee meeting in a couple of weeks. They've been showing more interest in Boost lately, so who knows... --Beman
participants (16)
-
Aaron W. LaFramboise -
Aleksey Gurtovoy -
Beman Dawes -
Bronek Kozicki -
Caleb Epstein -
David Abrahams -
Jeff Garland -
John Maddock -
Jonathan Wakely -
Marshall Clow -
Martin Slater -
Martin Wille -
Rene Rivera -
Thorsten Ottosen -
Toon Knapen -
Vladimir Prus