Refactoring tools/regression and tools/release into separate sub-modules

newer
NT² 3.0 and Boost.SIMD - Stable...

Beman Dawes

17 Jan 2014 17 Jan '14

4:25 p.m.

The tools used by regression testers and regression reporting, and the tools used by release managers are currently part of the boost-super project. As we gain experience with Git and submodules, it is becoming clear these need to be decomposed into two separate submodules. Among other advantages, that would speed work on these tools as we could better utilize volunteers without having to hand out excessive super-project write permissions. We would like one or two volunteers to do the refactoring. You don't need to be a library maintainer or release manager, although that's fine if you are. You do need git skills, as we do want to preserve history. Comments? --The release managers

Show replies by date

Ahmed Charles

11 Feb 11 Feb

8:22 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

...

Date: Fri, 17 Jan 2014 11:25:49 -0500 From: bdawes@acm.org To: boost@lists.boost.org Subject: [boost] Refactoring tools/regression and tools/release into separate sub-modules

The tools used by regression testers and regression reporting, and the tools used by release managers are currently part of the boost-super project. As we gain experience with Git and submodules, it is becoming clear these need to be decomposed into two separate submodules. Among other advantages, that would speed work on these tools as we could better utilize volunteers without having to hand out excessive super-project write permissions.

We would like one or two volunteers to do the refactoring. You don't need to be a library maintainer or release manager, although that's fine if you are. You do need git skills, as we do want to preserve history.

Comments?

I just worked on this, the results are: https://github.com/ahmedcharles/regression https://github.com/ahmedcharles/release And the additions to the superproject are: https://github.com/ahmedcharles/boost Let me know if you have any questions.

Daniel James

11:59 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

On 11 February 2014 08:22, Ahmed Charles wrote:

...

I just worked on this, the results are:

https://github.com/ahmedcharles/regression https://github.com/ahmedcharles/release

And the additions to the superproject are:

https://github.com/ahmedcharles/boost

Let me know if you have any questions.

Looks good, would it be possible to describe the method you used? As others might need to do something similar. I think the next steps are: 1. Transfer the modules into 'boostorg', I think you can do that through github, although you might have to be added to a team if you don't have any write permissions in the organisation. 2. Change the regression testers to use the regression module. We need to do that before updating the super project as they download directly from it. 3. Once they're successfully using it, merge the super project. Does that sound right? I don't think the release module even needs to be a submodule, as nothing depends on it. Ideally the regression tests also shouldn't be, but I suspect some of the infrastructure relies on it.

Beman Dawes

2:07 p.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

On Tue, Feb 11, 2014 at 3:59 AM, Daniel James wrote:

...

On 11 February 2014 08:22, Ahmed Charles wrote:

...
I just worked on this, the results are:

https://github.com/ahmedcharles/regression https://github.com/ahmedcharles/release

And the additions to the superproject are:

https://github.com/ahmedcharles/boost

Let me know if you have any questions.

Looks good, would it be possible to describe the method you used? As others might need to do something similar.

I think the next steps are:

1. Transfer the modules into 'boostorg', I think you can do that through github, although you might have to be added to a team if you don't have any write permissions in the organisation. 2. Change the regression testers to use the regression module. We need to do that before updating the super project as they download directly from it. 3. Once they're successfully using it, merge the super project.

Does that sound right?

I don't think the release module even needs to be a submodule, as nothing depends on it. Ideally the regression tests also shouldn't be, but I suspect some of the infrastructure relies on it.

This is very helpful! Thank you both for working on it! I'm at the C++ committee meeting and totally distracted, but I can't think of any reason not to make the change as soon as Daniel feels comfortable. Please do be sure the regression testers know what to do and watch the regression tests after the change to make sure all is well. --Beman

Ahmed Charles

9:17 p.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

...

Date: Tue, 11 Feb 2014 06:07:43 -0800 From: bdawes@acm.org To: boost@lists.boost.org Subject: Re: [boost] Refactoring tools/regression and tools/release into separate sub-modules

This is very helpful! Thank you both for working on it!

You're welcome. Glad to help.

...

I'm at the C++ committee meeting and totally distracted, but I can't think of any reason not to make the change as soon as Daniel feels comfortable. Please do be sure the regression testers know what to do and watch the regression tests after the change to make sure all is well.

Have fun at the meeting.

Ahmed Charles

9:15 p.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

...

Date: Tue, 11 Feb 2014 11:59:00 +0000 From: dnljms@gmail.com To: boost@lists.boost.org Subject: Re: [boost] Refactoring tools/regression and tools/release into separate sub-modules

On 11 February 2014 08:22, Ahmed Charles wrote:

...
I just worked on this, the results are:

https://github.com/ahmedcharles/regression https://github.com/ahmedcharles/release

And the additions to the superproject are:

https://github.com/ahmedcharles/boost

Let me know if you have any questions.

Looks good, would it be possible to describe the method you used? As others might need to do something similar.

I used filter-branch --subdirectory-filter, basically. The only interesting thing other than that is that I also added a .gitattributes file to each commit using a --tree-filter so that checkout's of the history work well with file endings (or at least as well as they do currently). Note that I didn't migrate tags over, mostly because I deleted the local copy before I realized that I hadn't pushed them. Anyways, if you want tags as well, I can just redo the commands tonight and upload new versions of the repositories.

...

I think the next steps are:

1. Transfer the modules into 'boostorg', I think you can do that through github, although you might have to be added to a team if you don't have any write permissions in the organisation.

I don't have perms on any team, so I can't do that.

...

2. Change the regression testers to use the regression module. We need to do that before updating the super project as they download directly from it.

Ah, who do I have to coordinate with to get this done?

...

3. Once they're successfully using it, merge the super project.

Does that sound right?

I don't think the release module even needs to be a submodule, as nothing depends on it. Ideally the regression tests also shouldn't be, but I suspect some of the infrastructure relies on it.

In terms of steps: 1. Make sure the content is right (mostly whether tags are desired or not). 2. Move the repositories into the boostorg organisation (I can do this if I get permissions and signoff on step 1). 3. Figure out who needs to change the regression testers for this to work. 4. Determine if the superproject content should simply be deleted or if it should contain a submodule reference. (Who do I talk to about regression or release and whether they should be a submodule or not?)

Daniel James

11 p.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

On 11 February 2014 21:15, Ahmed Charles wrote:

...

...
From: dnljms@gmail.com

Looks good, would it be possible to describe the method you used? As others might need to do something similar.

I used filter-branch --subdirectory-filter, basically. The only interesting thing other than that is that I also added a .gitattributes file to each commit using a --tree-filter so that checkout's of the history work well with file endings (or at least as well as they do currently).

Thanks for the info, adding gitattributes is a good idea.

...

Note that I didn't migrate tags over, mostly because I deleted the local copy before I realized that I hadn't pushed them. Anyways, if you want tags as well, I can just redo the commands tonight and upload new versions of the repositories.

I don't think it really matters for these modules, unless someone disagrees.

...

...
I think the next steps are:

1. Transfer the modules into 'boostorg', I think you can do that through github, although you might have to be added to a team if you don't have any write permissions in the organisation.

I don't have perms on any team, so I can't do that.

I added you to a new 'regression' team, so you should be able to now. I'll downgrade it to write access once you're done.

...

...
2. Change the regression testers to use the regression module. We need to do that before updating the super project as they download directly from it.

Ah, who do I have to coordinate with to get this done?

I think you just need to send an email to the testing list: http://lists.boost.org/mailman/listinfo.cgi/boost-testing

...

In terms of steps:

1. Make sure the content is right (mostly whether tags are desired or not). 2. Move the repositories into the boostorg organisation (I can do this if I get permissions and signoff on step 1). 3. Figure out who needs to change the regression testers for this to work. 4. Determine if the superproject content should simply be deleted or if it should contain a submodule reference. (Who do I talk to about regression or release and whether they should be a submodule or not?)

Beman and Marshall are the release people. If they have a problem with removing it from the tree they should speak up now. For the regression module, see what people on the testing list say.

Beman Dawes

12 Feb 12 Feb

4:33 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

On Tue, Feb 11, 2014 at 3:00 PM, Daniel James wrote:

...

On 11 February 2014 21:15, Ahmed Charles wrote:

...
In terms of steps:

1. Make sure the content is right (mostly whether tags are desired or not). 2. Move the repositories into the boostorg organisation (I can do this if I get permissions and signoff on step 1). 3. Figure out who needs to change the regression testers for this to work. 4. Determine if the superproject content should simply be deleted or if it should contain a submodule reference. (Who do I talk to about regression or release and whether they should be a submodule or not?)

Beman and Marshall are the release people. If they have a problem with removing it from the tree they should speak up now. For the regression module, see what people on the testing list say.

I'm very much in favor. Marshall is here at the meeting, so may be slow to reply. I'll bend his arm if necessary. --Beman

Rene Rivera

5:06 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

Note.. Whenever this move happens.. Please make sure the copied commits are up to date. As I'm making changes continually this past week. For example I made changes today. On Tue, Feb 11, 2014 at 10:33 PM, Beman Dawes wrote:

...

On Tue, Feb 11, 2014 at 3:00 PM, Daniel James wrote:

...
On 11 February 2014 21:15, Ahmed Charles wrote:

...
In terms of steps:

1. Make sure the content is right (mostly whether tags are desired or not). 2. Move the repositories into the boostorg organisation (I can do this if I get permissions and signoff on step 1). 3. Figure out who needs to change the regression testers for this to work. 4. Determine if the superproject content should simply be deleted or if it should contain a submodule reference. (Who do I talk to about regression or release and whether they should be a submodule or not?)

Beman and Marshall are the release people. If they have a problem with removing it from the tree they should speak up now. For the regression module, see what people on the testing list say.

I'm very much in favor. Marshall is here at the meeting, so may be slow to reply. I'll bend his arm if necessary.

--Beman

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

-- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

Ahmed Charles

6:27 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

...

Date: Tue, 11 Feb 2014 23:06:13 -0600 From: grafikrobot@gmail.com To: boost@lists.boost.org Subject: Re: [boost] Refactoring tools/regression and tools/release into separate sub-modules

Note.. Whenever this move happens.. Please make sure the copied commits are up to date. As I'm making changes continually this past week. For example I made changes today.

That will make things interesting. I don't actually know how regression is used, so I'm unfamiliar with the process to do the move. Other than mailing the test list, is there anything else I should know? Given that the new location will be https://github.com/boostorg/regression What needs to actually happen to do the migration?

Daniel James

11:47 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

On 12 February 2014 05:06, Rene Rivera wrote:

...

Note.. Whenever this move happens.. Please make sure the copied commits are up to date. As I'm making changes continually this past week. For example I made changes today.

Please can you arrange a schedule for the transition? So that Ahmed will only need to create the new repo once.

Rene Rivera

3:05 p.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

On Wed, Feb 12, 2014 at 5:47 AM, Daniel James wrote:

...

On 12 February 2014 05:06, Rene Rivera wrote:

...
Note.. Whenever this move happens.. Please make sure the copied commits are up to date. As I'm making changes continually this past week. For example I made changes today.

Please can you arrange a schedule for the transition? So that Ahmed will only need to create the new repo once.

Ahmed.. Whatever time is fine with me.. Just that I have enough of a warning to stop making changes to the old one so that they get copied over to the new repo. After it's copied over I'll have to start making changes at the new place. Then get all the testers to switch to the new regression script location. The testing switch itself will take 3-4 calendar days. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

Ahmed Charles

4:32 p.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

...

Date: Wed, 12 Feb 2014 09:05:48 -0600 From: grafikrobot@gmail.com To: boost@lists.boost.org Subject: Re: [boost] Refactoring tools/regression and tools/release into separate sub-modules

On Wed, Feb 12, 2014 at 5:47 AM, Daniel James wrote:

...
On 12 February 2014 05:06, Rene Rivera wrote:

...
Note.. Whenever this move happens.. Please make sure the copied commits are up to date. As I'm making changes continually this past week. For example I made changes today.

Please can you arrange a schedule for the transition? So that Ahmed will only need to create the new repo once.

Ahmed.. Whatever time is fine with me.. Just that I have enough of a warning to stop making changes to the old one so that they get copied over to the new repo. After it's copied over I'll have to start making changes at the new place. Then get all the testers to switch to the new regression script location. The testing switch itself will take 3-4 calendar days.

Switching tonight (like 8-12 PST US) is fine with me. I'll upload the new repo and send a mail saying that I'm done and you can start switching people to using it. Then we can decide what to do with the old content in the superproject and whether this should be a submodule or not.

Ahmed Charles

13 Feb 13 Feb

5:31 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

...

From: acharles@outlook.com To: boost@lists.boost.org Subject: RE: [boost] Refactoring tools/regression and tools/release into separate sub-modules Date: Wed, 12 Feb 2014 08:32:35 -0800

...
Ahmed.. Whatever time is fine with me.. Just that I have enough of a warning to stop making changes to the old one so that they get copied over to the new repo. After it's copied over I'll have to start making changes at the new place. Then get all the testers to switch to the new regression script location. The testing switch itself will take 3-4 calendar days.

Switching tonight (like 8-12 PST US) is fine with me. I'll upload the new repo and send a mail saying that I'm done and you can start switching people to using it. Then we can decide what to do with the old content in the superproject and whether this should be a submodule or not.

I just finished, so the new regression repo is at: https://github.com/boostorg/regression Let me know if you have any questions.

Rene Rivera

3:05 p.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

On Wed, Feb 12, 2014 at 11:31 PM, Ahmed Charles wrote:

...

...
From: acharles@outlook.com To: boost@lists.boost.org Subject: RE: [boost] Refactoring tools/regression and tools/release into separate sub-modules Date: Wed, 12 Feb 2014 08:32:35 -0800

...
Ahmed.. Whatever time is fine with me.. Just that I have enough of a warning to stop making changes to the old one so that they get copied over to the new repo. After it's copied over I'll have to start making changes at the new place. Then get all the testers to switch to the new regression script location. The testing switch itself will take 3-4 calendar days.

Switching tonight (like 8-12 PST US) is fine with me. I'll upload the new repo and send a mail saying that I'm done and you can start switching people to using it. Then we can decide what to do with the old content in the superproject and whether this should be a submodule or not.

I just finished, so the new regression repo is at: https://github.com/boostorg/regression

Let me know if you have any questions.

Many thanks for the work. I'll start making changes to adjust to the new location. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

Ahmed Charles

14 Feb 14 Feb

5:15 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

----------------------------------------

...

Date: Thu, 13 Feb 2014 09:05:52 -0600 From: grafikrobot@gmail.com To: boost@lists.boost.org Subject: Re: [boost] Refactoring tools/regression and tools/release into separate sub-modules

On Wed, Feb 12, 2014 at 11:31 PM, Ahmed Charles wrote:

...
I just finished, so the new regression repo is at: https://github.com/boostorg/regression

Let me know if you have any questions.

Many thanks for the work. I'll start making changes to adjust to the new location.

You're welcome. At what point should the code (for both regression and release) be removed from the superproject (or possibly have it changed to being a submodule)?

Rene Rivera

5:49 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

On Thu, Feb 13, 2014 at 11:15 PM, Ahmed Charles wrote:

...

----------------------------------------

...
Date: Thu, 13 Feb 2014 09:05:52 -0600 From: grafikrobot@gmail.com To: boost@lists.boost.org Subject: Re: [boost] Refactoring tools/regression and tools/release into separate sub-modules

On Wed, Feb 12, 2014 at 11:31 PM, Ahmed Charles
...
I just finished, so the new regression repo is at: https://github.com/boostorg/regression

Let me know if you have any questions.

Many thanks for the work. I'll start making changes to adjust to the new location.

You're welcome.

At what point should the code (for both regression and release) be removed from the superproject (or possibly have it changed to being a submodule)?

For "release" it can be removed at any time. For "regression" we need to wait until all the changes to work with the separate repo are working and propagated to all the testers. That is likely to take a week, at least. I'm hoping I can get the regression scripts working as an independent project, and hence not need to be added to the super-project at all. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

Beman Dawes

9:07 p.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

On Thu, Feb 13, 2014 at 7:05 AM, Rene Rivera wrote:

...

On Wed, Feb 12, 2014 at 11:31 PM, Ahmed Charles
...
wrote:

...
...
From: acharles@outlook.com To: boost@lists.boost.org Subject: RE: [boost] Refactoring tools/regression and tools/release into separate sub-modules Date: Wed, 12 Feb 2014 08:32:35 -0800

...
Ahmed.. Whatever time is fine with me.. Just that I have enough of a warning to stop making changes to the old one so that they get copied over to the new repo. After it's copied over I'll have to start making changes at the new place. Then get all the testers to switch to the new regression script location. The testing switch itself will take 3-4 calendar days.

Switching tonight (like 8-12 PST US) is fine with me. I'll upload the new repo and send a mail saying that I'm done and you can start switching people to using it. Then we can decide what to do with the old content in the superproject and whether this should be a submodule or not.

I just finished, so the new regression repo is at: https://github.com/boostorg/regression

Let me know if you have any questions.

Many thanks for the work. I'll start making changes to adjust to the new location.

+1 --Beman

Ahmed Charles

12 Feb 12 Feb

6:09 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

...

Date: Tue, 11 Feb 2014 20:33:38 -0800 From: bdawes@acm.org To: boost@lists.boost.org Subject: Re: [boost] Refactoring tools/regression and tools/release into separate sub-modules

I'm very much in favor. Marshall is here at the meeting, so may be slow to reply. I'll bend his arm if necessary.

Release is here: https://github.com/boostorg/regression The next step is deleting the files from the superproject: git rm -r tools/release (I don't think I have permission to submit that change). Let me know if you need anything else related to 'release'.

Ahmed Charles

6:17 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

...

Date: Tue, 11 Feb 2014 20:33:38 -0800 From: bdawes@acm.org To: boost@lists.boost.org Subject: Re: [boost] Refactoring tools/regression and tools/release into separate sub-modules

I'm very much in favor. Marshall is here at the meeting, so may be slow to reply. I'll bend his arm if necessary.

Release is here: https://github.com/boostorg/release (sorry, wrong url last time) The next step is deleting the files from the superproject: git rm -r tools/release (I don't think I have permission to submit that change). Let me know if you need anything else related to 'release'.

Robert Kawulak

13 Feb 13 Feb

1:44 a.m.

...

From: Ahmed Charles Release is here: https://github.com/boostorg/regression

BTW, as to tools/regression - there were many changes in trunk/develop that were never merged into release/master (process_jam_log.cpp comes to my mind, but there's probably more) - maybe it would be a good idea to finally synchronise the code in those branches while doing the move? Best regards, Robert

Ahmed Charles

2:54 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

...

From: robert.kawulak@gmail.com To: boost@lists.boost.org Date: Thu, 13 Feb 2014 02:44:59 +0100 Subject: Re: [boost] Refactoring tools/regression and tools/release into separate sub-modules

...
From: Ahmed Charles Release is here: https://github.com/boostorg/regression

BTW, as to tools/regression - there were many changes in trunk/develop that were never merged into release/master (process_jam_log.cpp comes to my mind, but there's probably more) - maybe it would be a good idea to finally synchronise the code in those branches while doing the move?

I'll look into that, but not as part of the move, since that would introduce more variables that could go wrong.

Rene Rivera

4:30 a.m.

New subject: Refactoring tools/regression and tools/release into separate sub-modules

On Wed, Feb 12, 2014 at 8:54 PM, Ahmed Charles wrote:

...

...
From: robert.kawulak@gmail.com To: boost@lists.boost.org Date: Thu, 13 Feb 2014 02:44:59 +0100 Subject: Re: [boost] Refactoring tools/regression and tools/release into separate sub-modules

...
From: Ahmed Charles Release is here: https://github.com/boostorg/regression

BTW, as to tools/regression - there were many changes in trunk/develop that were never merged into release/master (process_jam_log.cpp comes to my mind, but there's probably more) - maybe it would be a good idea to finally synchronise the code in those branches while doing the move?

I'll look into that, but not as part of the move, since that would introduce more variables that could go wrong.

Yes.. Doing a merge to master will be much easier and less disruptive after getting the transition working. And will only really make sense after since right now nothing should be using the master branch for actual regression testing. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

3922

Age (days ago)

3950

Last active (days ago)

List overview

Download

22 comments

5 participants

participants (5)

Ahmed Charles
Beman Dawes
Daniel James
Rene Rivera
Robert Kawulak