The current schedule allows ten days for each review. It has been brought to my attention that this is too aggressive. For the next couple of reviews, I will contact the library review managers to arrange new start dates to allow for longer review periods and to also ensure that they do not overlap. In my next status report, I will explain this change in the schedule and try to contact the remaining library authors for new start dates. My Concerns: 1) The review process is slow. 2) The current backlog of libraries to review is quite large. 3) Some libraries have been waiting to be reviewed for 6 or more months. 4) To few reviews of the boost libraries. I would like to see more people participate in the review process. 5) The policy of not allowing overalaping review dates means that I will not be able to plan ahead more than about 2 to 3 weeks. 6) Not having firm start dates could be a source of frustration for library authors if they have other commitments. 7) The reason for not having overalpping reviews is based on the preference for a flexible review schedule. This flexibility, however, has a large cost in terms of not being able to provide firm start dates for our library authors. Tom Brinkman Review Wizard David Abrahams wrote:

"Jeff Garland" <jeff@crystalclearsoftware.com> writes:

...

Hi all -

Our review wizard, Tom B. has suggested to me that we allow subsequent reviews to begin even if a previous review has been extended. This means 2 or more reviews will possibly be running in parallel. I don't believe we have a written policy, but I'm quite certain we have never run reviews in parallel. I also think it is a very bad idea.

I agree completely. But we haven't heard from Tom; maybe he had something particular in mind that makes it OK?

Thorsten Ottosen wrote:

I could accept review overlaps if it became the review managers job to put together a group of people guaranteed to make a review. The size of this group would depend on the library's size.

I like this idea. Its a good solution to the problem.

Jeff Garland wrote:

Well, I don't see the 'slowness' as a problem. I think we have a problem because we have a backlog of submissions. There were some significant periods where reviews didn't keep up with the pace of submissions and now we have a bunch of catch-up to do.

Agreed. The slowness is as a result of the backlog, not the reviews themselves. Keeping and maintaining an up-to-date schedule will help a great deal to avoid this in the future.

My only suggestion for shortening the formal review period would be to somehow encourage or develop reviews of libraries in advance -- thus reducing the amount of comment during the formal review. Anyone can go review boost::fsm, just to pick one, and go review the code and docs and post it to the list. The problem is, however, there is an additional dynamic during the formal review -- reviewers read other reviews and discuss them. Perhaps if there was a way of gathering review comments over a longer period (via wiki page or something) we could shorten this last phase.

Agreed. This would also be helpful.

...

I could accept review overlaps if it became the review managers job to put together a group of people guaranteed to make a review. The size of this group would depend on the library's size.

I think this is very difficult to do practically...

Yes, difficult. But it is worth further consideration. I would be in favor of this approach as a reasonable compromise.

"Jeff Garland" <jeff@crystalclearsoftware.com> wrote in message:

My only suggestion for shortening the formal review period would be to somehow encourage or develop reviews of libraries in advance -- thus reducing the amount of comment during the formal review. Anyone can go review boost::fsm, just to pick one, and go review the code and docs and post it to the list. The problem is, however, there is an additional dynamic during the formal review -- reviewers read other reviews and discuss them. Perhaps if there was a way of gathering review comments over a longer period (via wiki page or something) we could shorten this last phase.

This sounds like a good idea. But would it require that proposed libraries be frozen for a certain period before the formal review begins? I know I took advantage of the long wait (something like 8 months) between proposal and review of iostreams to make lots of improvements, some as recent as one month ago. Jonathan

From: "Jonathan Turkanis" <technews@kangaroologic.com>

"Jeff Garland" <jeff@crystalclearsoftware.com> wrote in message:

...

My only suggestion for shortening the formal review period would be to somehow encourage or develop reviews of libraries in advance -- thus reducing the amount of comment during the formal review. Anyone can go review boost::fsm, just to pick one, and go review the code and docs and post it to the list. The problem is, however, there is an additional dynamic during the formal review -- reviewers read other reviews and discuss them. Perhaps if there was a way of gathering review comments over a longer period (via wiki page or something) we could shorten this last phase.

This sounds like a good idea. But would it require that proposed libraries be

I think this might be workable.

frozen for a certain period before the formal review begins? I know I took advantage of the long wait (something like 8 months) between proposal and review of iostreams to make lots of improvements, some as recent as one month ago.

Perhaps a library must be frozen to be submitted for review, provided the review occurs within 30 days (or some such time) after being put on the review queue. If the author decides some dramatic changes are in order, the library would be withdrawn from the review queue, changes would be made to it, and it would be resumbitted when ready again. Otherwise, early reviewers would be looking at a moving target. Perhaps for a library to be put on the review queue, it must be carefully reviewed by a panel, of some minimum size, not including those closely following or involved in the development of the library. IOW, get some fresh, independent perspective on the library, which should flesh out many documentation issues and even some design issues, prior to the formal review. Such reviews could take place via e-mail or on the Wiki. Still, larger libraries take time to be reviewed, and invite lots of comments and discussion. If a person reviewed a library ahead of time, their review would appear early in the review cycle. However, others would still need time to perform an on-demand review, so you really can't compress the review period. This process would just help to ensure that libraries are more mature before being subjected to a formal review. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;

On Tue, 07 Sep 2004 14:48:52 -0700, tom brinkman wrote

My Concerns: ...snip... 4) To few reviews of the boost libraries. I would like to see more people participate in the review process.

This is one of the reasons for making sure there an adequete time and non-overlapping review period. If there are too many reviews on top of each other the reviewers are sapped and unable to participate fully. As for getting more people to help review, I just don't know. People volunteer what time they can in the areas they can. The only thing I can think is if we advertise to a wider audience -- like posting the review schedule up on comp.lang.c++.

5) The policy of not allowing overalaping review dates means that I will not be able to plan ahead more than about 2 to 3 weeks.

I don't think this is the case. If we reschedule the current docet with adequate time and buffer between we won't need to extend and change the schedule as we go. When reviews are rescheduled you might want to ask the review manager to let you know if he thinks the library is large and might need more than the standard 10 day period. Then you can tack on a few extra insurance days for the large libs.

6) Not having firm start dates could be a source of frustration for library authors if they have other commitments.

I see no reason why you can't plan out at least 2-3 months or 4-6 reviews ahead. That should be plenty of notice.

7) The reason for not having overalpping reviews is based on the preference for a flexible review schedule. This flexibility, however, has a large cost in terms of not being able to provide firm start dates for our library authors.

No, it is quality of reviews that is my only concern. As for flexibility, even with additional time and buffer there may come a point where we have to bite the bullet and reschedule, but I think it will be fairly rare. Still, it will be worth it if we need the extra time to get the review right. And, of course, once a review is scheduled an author or review manager might have to drop out because of an emergency that is higher priority -- so the review schedule is necessarily a plan that will not be followed exactly. Jeff

tom brinkman <reportbase@yahoo.com> writes:

With 10-15 days allocated per review and a 5-10 day buffer, we will be able to review at-most 1 to 2 reviews per month or 12 to 24 libraries per year.

I'm not sure we can handle faster growth than that. Should we be able to? I wonder if enough libraries are going through the preliminary submission/refinement phase outlined at http://www.boost.org/more/submission_process.htm#Preliminary -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

tom brinkman <reportbase@yahoo.com> writes:

David Abrahams wrote:

...

tom brinkman <reportbase@yahoo.com> writes:

...

With 10-15 days allocated per review and a 5-10 day buffer, we will be able to review at-most 1 to 2 reviews per month or 12 to 24 libraries per year.

I'm not sure we can handle faster growth than that. Should we be able to?

If we added 12 significant libraries in a given year, I'm sure most would consider that it had been a successful year.

...

I wonder if enough libraries are going through the preliminary submission/refinement phase outlined at http://www.boost.org/more/submission_process.htm#Preliminary

Many are not.

That's a problem. I remember the last time I tried to submit a library for formal review without doing that. Beman said, "aren't you forgetting something?" Checking that out should be a responsibility of the Review Wizard. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

"Paul A Bristow" <pbristow@hetp.u-net.com> writes:

| -----Original Message----- | From: boost-bounces@lists.boost.org | [mailto:boost-bounces@lists.boost.org] On Behalf Of David Abrahams | Sent: 09 September 2004 03:27 | To: boost@lists.boost.org | Cc: reportbase@yahoo.com | Subject: [boost] Re: [admin] Overlapping reviews -- should | this be allowed? | | I wonder if enough libraries are going through the preliminary | submission/refinement phase outlined at | http://www.boost.org/more/submission_process.htm#Preliminary

I strongly agree with this - IMO far too many libraries are only getting serious attention at the review stage, when useful input is really much too late.

And in many cases there is far too little user 'real-life' experience - the source of useful examples and confirmation that theory matches practice.

I think we need to somehow get much closer to final agreement before the Formal review stage, when, in a way, acceptance should be almost a formality.

Is a two stage review/acceptance process a way to improve?

1 Float the idea. 2 Get some support. 3 Get some feedback. 4 'in principle' review and if OK then 5 Boost library 'candidate' status, and move to a separate 'nearly ready' files area. 5 Refine with feedback. 6 Formal review, and if OK then else revert to 'nearly ready'. 7 Formal acceptance. 8 Add to next Boost release.

My fear is that this process would be too slow and heavyweight. IMO it should be up to the Review Wizard to only schedule reviews for libraries that have gone through the preliminaries. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Thomas Wenisch wrote:

On Wed, 8 Sep 2004, tom brinkman wrote:

...

With 10-15 days allocated per review and a 5-10 day buffer, we will be able to review at-most 1 to 2 reviews per month or 12 to 24 libraries per year.

A goal of 24 review per year strikes me as far too ambitious. I think 12 libraries is a more realistic goal (and allows for some dead time, such as right before a release, where no reviews are in progress).

I second that. 12 reviews per year is quite enough. Best, John -- John Torjo -- john@torjo.com Contributing editor, C/C++ Users Journal -- "Win32 GUI Generics" -- generics & GUI do mix, after all -- http://www.torjo.com/win32gui/ -- v1.3beta released - check out splitter/simple_viewer, a File Explorer/Viewer all in about 200 lines of code!

"Thorsten Ottosen" <nesotto@cs.auc.dk> writes:

"John Torjo" <john.lists@torjo.com> wrote in message news:4140023B.2070608@torjo.com... | Thomas Wenisch wrote: | > On Wed, 8 Sep 2004, tom brinkman wrote: | > | > | >>With 10-15 days allocated per review and a 5-10 day buffer, we will be able | >>to review at-most 1 to 2 reviews per month or 12 to 24 libraries per year. | > | > | > A goal of 24 review per year strikes me as far too ambitious. I think 12 | > libraries is a more realistic goal (and allows for some dead time, such as | > right before a release, where no reviews are in progress). | > | | I second that. 12 reviews per year is quite enough.

I would rather see a number that reflects how big the libraries are. Some are small, some are huge.

In the past we have always chosen the length of review period based on the scope of the submission. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com