what happened to the "tools/regression" directory - and the stuff in it?
The tools in the tools/regression directory are critical to the testing of the serialization library on one's local machine. What happened to these. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
On Wed, Mar 18, 2015 at 3:07 PM, Robert Ramey <ramey@rrsd.com> wrote:
The tools in the tools/regression directory are critical to the testing of the serialization library on one's local machine. What happened to these.
They went here.. <https://github.com/boostorg/regression>. What is critical and how? -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
Rene Rivera-2 wrote
On Wed, Mar 18, 2015 at 3:07 PM, Robert Ramey <
ramey@
> wrote:
The tools in the tools/regression directory are critical to the testing of the serialization library on one's local machine. What happened to these.
They went here.. <https://github.com/boostorg/regression>;. What is critical and how?
I used them to generate at massive table which detailed all results by build variant. So they were critical for me. Since no one else complains I presume that I'm the only one that use them. So my question is - how is one supposed to invoke this functionality in the current setup? Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
On Wed, Mar 18, 2015 at 3:24 PM, Robert Ramey <ramey@rrsd.com> wrote:
Rene Rivera-2 wrote
What is critical and how?
I used them to generate at massive table which detailed all results by build variant. So they were critical for me. Since no one else complains I presume that I'm the only one that use them.
Probably :-) So my question is - how is one supposed to invoke this functionality in the
current setup?
I don't know.. Since you have told me what the functionality is specifically. What did you used to build/run? And how? -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
Rene Rivera-2 wrote
So my question is - how is one supposed to invoke this functionality in the
current setup?
I don't know.. Since you have told me what the functionality is specifically. What did you used to build/run? And how?
I've used the procedure described here http://www.boost.org/doc/libs/1_57_0/tools/regression/doc/library_status.htm... Obviously something equivalent to this is necessary for every library author. What do other library authors use? What do you recommend for testing on one's own machine? Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
AMDG On 03/18/2015 02:44 PM, Robert Ramey wrote:
I've used the procedure described here http://www.boost.org/doc/libs/1_57_0/tools/regression/doc/library_status.htm...
Obviously something equivalent to this is necessary for every library author. What do other library authors use? What do you recommend for testing on one's own machine?
I just use b2 | less In Christ, Steven Watanabe
Steven Watanabe-4 wrote
I just use b2 | less
LOL - for 250 tests with multiple compilers/configurations? and re-run b2 every time you want to go back and see a previous result? Seriously - how do other people deal with this? Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
On Wednesday 18 March 2015 14:09:41 Robert Ramey wrote:
Steven Watanabe-4 wrote
I just use b2 | less
LOL - for 250 tests with multiple compilers/configurations? and re-run b2 every time you want to go back and see a previous result?
Seriously - how do other people deal with this?
I don't have 250 tests, and b2 >1.txt works just fine for me. I run tests on one compiler (typically, gcc) before pushing to develop and let the testers do the rest. If I suspect the change can affect compatibility with a specific compiler (typically, msvc) I may run the tests on that compiler as well. That is a rare case, though.
AMDG On 03/18/2015 03:09 PM, Robert Ramey wrote:
Steven Watanabe-4 wrote
I just use b2 | less
LOL - for 250 tests with multiple compilers/configurations? and re-run b2 every time you want to go back and see a previous result?
Seriously - how do other people deal with this?
I was completely serious. None of my libraries have more than 100 tests, and I don't commit changes at all until the tests are completely clean on all the toolsets I'm testing with. There's a lot of output, but I can ignore most of it and just search for "failed." When I'm debugging I generally only run one toolset at a time. When I run multiple toolsets, it's almost always a last check that everything's okay before I commit. In Christ, Steven Watanabe
On 03/18/2015 02:09 PM, Robert Ramey wrote:
Steven Watanabe-4 wrote
I just use b2 | less
LOL - for 250 tests with multiple compilers/configurations? and re-run b2 every time you want to go back and see a previous result?
Seriously - how do other people deal with this?
Robert Ramey
We have 230 top level tests in Spirit. I use: b2 > out.txt a simple: grep failed out.txt | wc -l tells me how many failed. I can then go and look at details in the output. michael -- Michael Caisse ciere consulting ciere.com
On 03/19/2015 12:30 PM, Michael Caisse wrote:
On 03/18/2015 02:09 PM, Robert Ramey wrote:
Steven Watanabe-4 wrote
I just use b2 | less
LOL - for 250 tests with multiple compilers/configurations? and re-run b2 every time you want to go back and see a previous result?
Seriously - how do other people deal with this?
Robert Ramey
We have 230 top level tests in Spirit. I use:
b2 > out.txt
a simple:
grep failed out.txt | wc -l
Well.... grep "^\*\*passed" out.txt | wc -l will get the passed ones. I have actually have some post processing that I do for the automated Bamboo builds to show failures differently. The above expression wont actually work for what you want. Adjust the grep expression to meet your needs. (o;
a simple:
grep failed out.txt | wc -l
Well....
grep "^\*\*passed" out.txt | wc -l
will get the passed ones. I have actually have some post processing that I do for the automated Bamboo builds to show failures differently. The above expression wont actually work for what you want. Adjust the grep expression to meet your needs. (o;
grepping for "failed" shows the "failed as expected" messages as well. I grep for \.\.failed which works for me. John.
On Wed, Mar 18, 2015 at 3:44 PM, Robert Ramey <ramey@rrsd.com> wrote:
Rene Rivera-2 wrote
So my question is - how is one supposed to invoke this functionality in the
current setup?
I don't know.. Since you have told me what the functionality is specifically. What did you used to build/run? And how?
I've used the procedure described here
http://www.boost.org/doc/libs/1_57_0/tools/regression/doc/library_status.htm...
OK. I'll see about updating that to where the code for that went to. And making sure it's working as best as I can. Won't be soon though. Also, note that the code for that tool is now at < https://github.com/boostorg/regression/tree/develop/library_status>. And IIRC it's updated to build if you clone/get the regression repo. But I never tried using the resulting executable. And process_jam_log is at < https://github.com/boostorg/regression/tree/develop/testing>. Obviously something equivalent to this is necessary for every library
author. What do other library authors use? What do you recommend for testing on one's own machine?
Like Steven I just use b2 directly and read the output. But then I have a very simple library to deal with. So I'm not a good example. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
Rene Rivera-2 wrote
Also, note that the code for that tool is now at < https://github.com/boostorg/regression/tree/develop/library_status>. And IIRC it's updated to build if you clone/get the regression repo. But I never tried using the resulting executable. And process_jam_log is at < https://github.com/boostorg/regression/tree/develop/testing>.
Obviously something equivalent to this is necessary for every library
author. What do other library authors use? What do you recommend for testing on one's own machine?
Like Steven I just use b2 directly and read the output. But then I have a very simple library to deal with. So I'm not a good example.
In my source tree there was a folder icon named regression and I just clicked it and the folder appeared in the right place in my modular-boost tree. Which I suppose is OK - except that there's another corner of git that I have to figure out how to use - this is a burden to me. When I first saw this I presumed that the boost test python scripts had been enhanced to provide this sort of facility in one's local environment. Now I'm poking around to see how all this works. There is a directory with 15K lines of *.py code it !!! Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
On Wed, Mar 18, 2015 at 4:52 PM, Robert Ramey <ramey@rrsd.com> wrote:
Rene Rivera-2 wrote
Also, note that the code for that tool is now at < https://github.com/boostorg/regression/tree/develop/library_status>. And IIRC it's updated to build if you clone/get the regression repo. But I never tried using the resulting executable. And process_jam_log is at < https://github.com/boostorg/regression/tree/develop/testing>.
Obviously something equivalent to this is necessary for every library
author. What do other library authors use? What do you recommend for testing on one's own machine?
Like Steven I just use b2 directly and read the output. But then I have a very simple library to deal with. So I'm not a good example.
In my source tree there was a folder icon named regression and I just clicked it and the folder appeared in the right place in my modular-boost tree. Which I suppose is OK - except that there's another corner of git that I have to figure out how to use - this is a burden to me.
When I first saw this I presumed that the boost test python scripts had been enhanced to provide this sort of facility in one's local environment. Now I'm poking around to see how all this works. There is a directory with 15K lines of *.py code it !!!
Note.. I've been cleaning up the regression tree to reduce that sort of thing. But of course things take a long time to accomplish as I can't interrupt testing and the release process. Anyway.. I do want to make your job as easy as possible in this regard so I do plan to make the library_status table generation easy to use. But.. I was planning on working on this a bit this weekend and it's not going to happen. My dev machine is having problems and looks like it's close to imminent death. So I'm doing backups, ordering a new machine, and then setting up that new machine from the backups. So It's going to be a much longer time before I get to this again :-( -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
Rene Rivera-2 wrote
So It's going to be a much longer time before I get to this again :-(
I've got my own copy and no one else uses it so it's really a non-problem for me. Don't spend any measurable time on this. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 3/20/2015 2:07 PM, Robert Ramey wrote:
Rene Rivera-2 wrote
So It's going to be a much longer time before I get to this again :-(
I've got my own copy and no one else uses it so it's really a non-problem for me. Don't spend any measurable time on this.
Could you share your copy via Git ?
I repackaged this is a git project in the form compatible with <boost directory>/tools/... https://github.com/robertramey/library_status So you can clone it into this directory, build it via boost build and/or CMake, copy the executable to a convenient place and start generating your local test table. If there's interest, we can add it back into the boost tools directory. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 3/26/2015 1:39 PM, Robert Ramey wrote:
I repackaged this is a git project in the form compatible with <boost directory>/tools/...
https://github.com/robertramey/library_status
So you can clone it into this directory, build it via boost build and/or CMake, copy the executable to a convenient place and start generating your local test table.
If there's interest, we can add it back into the boost tools directory.
Thank you ! I will try it out.
Robert Ramey
-- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
On Thu, Mar 26, 2015 at 1:39 PM, Robert Ramey <ramey@rrsd.com> wrote:
I repackaged this is a git project in the form compatible with <boost directory>/tools/...
https://github.com/robertramey/library_status
So you can clone it into this directory, build it via boost build and/or CMake, copy the executable to a convenient place and start generating your local test table.
If there's interest, we can add it back into the boost tools directory.
Robert, Why on earth you do that rather than just cloning git@github.com:boostorg/regression.git and using it directly? If there are problems with git@github.com:boostorg/regression.git, it is fine to fork it, fix the problems, then submit pull requests. But there is no reason it needs to be part of the Boost super-project. I guess we should have left a dummy boost-root/tools/regression directory in the super-project to point people to the new location, and provided explicit build instructions. I'll try to do that in the next few days, but it won't happen for 1.58.0. --Beman
Beman Dawes wrote
On Thu, Mar 26, 2015 at 1:39 PM, Robert Ramey <
ramey@
> wrote:
I repackaged this is a git project in the form compatible with <boost
directory>/tools/...
https://github.com/robertramey/library_status
So you can clone it into this directory, build it via boost build and/or CMake, copy the executable to a convenient place and start generating your local test table.
If there's interest, we can add it back into the boost tools directory.
Robert,
Why on earth you do that rather than just cloning
git@
:boostorg/regression.git and using it directly?
Rene had told me that the directory was obsolete and for that reason it was dropped. I found it and resurrected it on my system. I looked at and it contains a huge number of files whose purpose I couldn't understand. Also the directory structure was quite different that than of the other tools files which I wanted to replicate. It seemed to me that I would have to undo Rene's delete then do ... what exactly?. I foresaw a disproportional amount of git ju-jitso in order to resurrect (and merge in recent changes) something that had already been deprecated and no one seemed to have interest in anyway. Then I would have to move the files and directories around. I actually did consider it. But it all just looked to hairy for something no one had an interest in but me. So I just cut the gordian knot and did it. Actually, it's not even clear that it will be part of boost. No one seems to be interested in it and it's never been reviewed. It was dropped and no one complained except me. That's why I did what I did what I did. I still don't seen anything wrong with it.
If there are problems with
git@
:boostorg/regression.git, it is fine to fork it, fix the problems, then submit pull requests. But there is no reason it needs to be part of the Boost super-project. I guess we should have left a dummy boost-root/tools/regression directory in the super-project to point people to the new location, and provided explicit build instructions. I'll try to do that in the next few days, but it won't happen for 1.58.0.
Of course it's up to you, but it still doesn't seem worth your time to me. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 1/04/2015 8:20 AM, "Robert Ramey" <ramey@rrsd.com> wrote:
Beman Dawes wrote
On Thu, Mar 26, 2015 at 1:39 PM, Robert Ramey <
ramey@
> wrote:
I repackaged this is a git project in the form compatible with <boost
directory>/tools/...
https://github.com/robertramey/library_status
So you can clone it into this directory, build it via boost build
and/or
CMake, copy the executable to a convenient place and start generating your local test table.
If there's interest, we can add it back into the boost tools directory.
Robert,
Why on earth you do that rather than just cloning
git@
:boostorg/regression.git and using it directly?
Rene had told me that the directory was obsolete and for that reason it was dropped.
I found it and resurrected it on my system.
I looked at and it contains a huge number of files whose purpose I couldn't understand. Also the directory structure was quite different that than of the other tools files which I wanted to replicate. It seemed to me that I would have to undo Rene's delete then do ... what exactly?. I foresaw a disproportional amount of git ju-jitso in order to resurrect (and merge in recent changes) something that had already been deprecated and no one seemed to have interest in anyway. Then I would have to move the files and directories around. I actually did consider it. But it all just looked to hairy for something no one had an interest in but me. So I just cut the gordian knot and did it. Actually, it's not even clear that it will be part of boost. No one seems to be interested in it and it's never been reviewed. It was dropped and no one complained except me.
That's why I did what I did what I did. I still don't seen anything wrong with it.
If there are problems with
git@
:boostorg/regression.git, it is fine to fork it, fix the problems, then submit pull requests. But there is no reason it needs to be part of the Boost super-project. I guess we should have left a dummy boost-root/tools/regression directory in the super-project to point people to the new location, and provided explicit build instructions. I'll try to do that in the next few days, but it won't happen for 1.58.0.
Of course it's up to you, but it still doesn't seem worth your time to me.
Actually, I had just asked on the boost testing list if there was a tool I could run to get a nice HTML report from for regression testing that I can view locally. There's a large number of failures for me to investigate for the Haiku port, and waiting for the regression web pages to update is painfully slow ;-)
On Tue, Mar 31, 2015 at 2:08 PM, Robert Ramey <ramey@rrsd.com> wrote:
Beman Dawes wrote
On Thu, Mar 26, 2015 at 1:39 PM, Robert Ramey <
ramey@
> wrote:
I repackaged this is a git project in the form compatible with <boost
directory>/tools/...
https://github.com/robertramey/library_status
So you can clone it into this directory, build it via boost build and/or CMake, copy the executable to a convenient place and start generating your local test table.
If there's interest, we can add it back into the boost tools directory.
Robert,
Why on earth you do that rather than just cloning
git@
:boostorg/regression.git and using it directly?
Rene had told me that the directory was obsolete and for that reason it was dropped.
Not exactly what I said :-) I found it and resurrected it on my system.
I looked at and it contains a huge number of files whose purpose I couldn't understand. Also the directory structure was quite different that than of the other tools files which I wanted to replicate.
I've been restructuring (but very carefully as to not interrupt the release schedule) in order to separate out the "user" level tools vs. the regression testers and reports users. With the hope of making it easier to maintain and use. And that's still a work in progress. So most things are not obsolete in the regression.git repo. The one thing that is obsolete (and will removed soonish) is the XSLT code... It's all been replaced by a combination of Python and C++ by now.
It seemed to me that I would have to undo Rene's delete then do ... what exactly?. I foresaw a disproportional amount of git ju-jitso in order to resurrect (and merge in recent changes) something that had already been deprecated and no one seemed to have interest in anyway. Then I would have to move the files and directories around. I actually did consider it. But it all just looked to hairy for something no one had an interest in but me.
I did promise to clean up the library_status docs and make it "easy" to use.. Once I got my new machine (I have it finally and spent all weekend restoring from my almost dead old one). So I will be looking at what you did and incorporating some of it back into the Boost version :-)
So I just cut the gordian knot and did it. Actually, it's not even clear that it will be part of boost. No one seems to be interested in it and it's never been reviewed. It was dropped and no one complained except me.
That's why I did what I did what I did. I still don't seen anything wrong with it.
I don't see anything wrong with it either. It might sound like a waste of time.. But since it helps you out while the Boost version of the tool is back to usability I'd say it's worth a little waste.
If there are problems with
git@
:boostorg/regression.git, it is fine to fork it, fix the problems, then submit pull requests. But there is no reason it needs to be part of the Boost super-project. I guess we should have left a dummy boost-root/tools/regression directory in the super-project to point people to the new location, and provided explicit build instructions. I'll try to do that in the next few days, but it won't happen for 1.58.0.
Of course it's up to you, but it still doesn't seem worth your time to me.
Yea.. Doesn't need to be part of the super-project. I think some pointers to it in the library author directed documentation should be enough. And it is at least worth my time if someone else wants to spend time on it that's fine too. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
I did promise to clean up the library_status docs and make it "easy" to use.. Once I got my new machine (I have it finally and spent all weekend restoring from my almost dead old one). So I will be looking at what you did and incorporating some of it back into the Boost version :-) Note that the only thing I actually did was to make directory structure which looks like all the other subdirectories in the "tools" directory. I saw this as analogous to our libraries where they all have similar directory/file layout. There's nothing more to it than that. You're owner of the "regression" directory and it's contents so I defer to you any decisions regarding it. Its totally up to you whether or not library status is included and if so - where you want to included it. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 3/18/2015 5:18 PM, Rene Rivera wrote:
On Wed, Mar 18, 2015 at 3:44 PM, Robert Ramey <ramey@rrsd.com> wrote:
Rene Rivera-2 wrote
So my question is - how is one supposed to invoke this functionality in the
current setup?
I don't know.. Since you have told me what the functionality is specifically. What did you used to build/run? And how?
I've used the procedure described here
http://www.boost.org/doc/libs/1_57_0/tools/regression/doc/library_status.htm...
OK. I'll see about updating that to where the code for that went to. And making sure it's working as best as I can. Won't be soon though.
Thanks ! I agree with Robert on this even if I had not used local regression testing but just executed b2 manually many times with different compilers for my libraries. I test on quite a few compilers and different versions largely on Windows and less so on Linux. Having an automated procedure that can test all the compilers I want to use in their various configurations at once, and then show me the regression tests results locally, seems much better than what I do manually, even if though I have my own command files both on Windows and Linux to make things easier for myself.
Also, note that the code for that tool is now at < https://github.com/boostorg/regression/tree/develop/library_status>. And IIRC it's updated to build if you clone/get the regression repo. But I never tried using the resulting executable. And process_jam_log is at < https://github.com/boostorg/regression/tree/develop/testing>.
Obviously something equivalent to this is necessary for every library
author. What do other library authors use? What do you recommend for testing on one's own machine?
Like Steven I just use b2 directly and read the output. But then I have a very simple library to deal with. So I'm not a good example.
thanks for all the helpful grep scripts. are you guys arguing that this is more convenient than the following: http://www.boost.org/doc/libs/1_54_0/tools/regression/doc/library_status.htm... In any case, It's not a huge deal for me as I'm the only one using it I'll just move it to my own repository. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/what-happened-to-the-tools-regression-dir... Sent from the Boost - Dev mailing list archive at Nabble.com.
AMDG On 03/19/2015 06:57 PM, Robert Ramey wrote:
thanks for all the helpful grep scripts.
are you guys arguing that this is more convenient than the following:
http://www.boost.org/doc/libs/1_54_0/tools/regression/doc/library_status.htm...
I'm not claiming that my method is better, but it works well enough, and I don't think that using this tool would give any significant benefit for me. If it helps you, that's great.
In any case, It's not a huge deal for me as I'm the only one using it I'll just move it to my own repository.
In Christ, Steven Watanabe
participants (9)
-
Andrey Semashev -
Beman Dawes -
Edward Diener -
Jessica Hamilton -
John Maddock -
Michael Caisse -
Rene Rivera -
Robert Ramey -
Steven Watanabe