Andrey Semashev wrote:
Hi Gennadiy,
While I mostly agree with you pointing out the drawbacks of the current system, I don't quite agree with your proposal.
On 02/28/2010 04:24 PM, Gennadiy Rozental wrote:
That's said, here's how better procedure might look like IMO. This will require some initial investment in writing scripts for process automation, but in a long run we should be very well compensated.
1. Any library author interrested in submission of new library should come to the "Candidate" page and register. Once registered candidate gets: a) svn repository for the library b) standardized page on boost website (something like boost.org/candidate/<candidate name> c) announcement post is sent automatically (with abstract and link to above page) to the mailing list.
Good. Having a central place for potential Boost libraries to evolve may simplify development. Although I'm not sure there are resources to maintain this kind of hosting.
We already hosting most of the library wait in svn. Candidate page should be comparatively tiny and only contains abstracts and links.
2. The candidate page should contain abstract and links to the sources and docs. Also it should include some kind of "voting" mechanism, where people would express the interest. Preferable with authentication, which would link to the mailing list members. To qualify for the review candidate should exceed some predefined threshold of minimum number of "supporters". These people are expected to post a review later on for the library to have a chance of being accepted.
Voting is good. I appreciated the feature on SourceForge. Although I don't think that the right to vote should be tied with posting a review later. I consider voting as afeedback mechanism, nothing more.
I am not gonna push this point. IMO it's fair to expect people who seconded the submission to provide a review within rather extended review period. On the other hand we can't really put a "requirement" clause to the support vote.
Regarding the candidate page, do you mean that the library docs should be hosted somewhere outside the Boost web site? If so, I don't like that idea. IMO, if we pursue the idea of a central hosting for the candidate libraries (with SVN, web access, etc.), it should include online documentation hosting, too.
No. I think the link should be somewhere inside the svn, or be the part of candidate page. If former case we need a script+link which extracts docs from svn, in later case - overhead is bigger.
3. Once candidate have proper number of supporters and passed all other formal requirements (docs, tests, directory structure) - all validated against repository, candidate author can schedule a review from reviews schedulers (whatever the proper name). Once review manager is assigned candidate page is transformed into "candidate review" page.
It's not clear how it's transformed and in what way. Regarding the review scheduling, it's pretty much like it happens nowdays.
Not really. The transition occurs almost automatically and asynchronously. There is no any queue. If one library has bigger support and interest in a community, it will go through the phases much faster.
4. Review process. The candidate review can start at any time by the review manager (no queue) and should take at least 2-4 month. There can be any number of reviewed being run concurrently. The "candidate review" page should include abstract, review package, and some kind of review submission mechanism (maybe boolean yes/no + an actual review). The review should be per person and each reviewer should have an ability to modify the review. Review discussion mechanism can be web based on rely on mailing list or some mixture of these.
I disagree, in several points.
* 2-4 months is a very long period.
Somehow you are fine with C++ standard changes being reviewed for years, but find 2-4 month for non-trivial C++ library too long?
You can't expect review manager and the library authors focused on the review that long.
Actually the point was to decrease the pressure. Longer time period means that both reviewers and author can take their time doing their job. I expect short short period of times with high activity with some gaps in between, where sides consider the matter.
Also, for simple tools, such as Boost.Move that is in the queue now, there's nothing to review during all that time.
For smaller/simple components we might have a policy of fast track review (no more than a week. These we can have a queue for, but there should be rather strict requirements to qualify: * 1 header only (or totally no more than N lines). Header only libraries qualify * All tests should work on all required platforms * Docs should be in boostbook format * To be accepted the library should receive 90%(?) approval within review period with at least 10(?) reviewers * Review period is never extended
On the other hand, I agree that a few weeks may not be enough for some larger scaled libraries. Which leads me to conclusion that the review duration should be individual, decided by the author, review manager and review wizards, taking into account other reviews.
Ultimately yes, but only if library qualify for fast track review. Otherwise I do not see basis for review wizard to warrant it.
* Concurrent reviews is wrong. We don't have enough reviewers and wizards to make sequential reviews. Allowing parallel reviews won't make it better. The review quality will also drop.
Even in rare situation where the same person is interested in several concurrent reviews, long review period should give one a chance to participate in both.
* Review mechanism should be convenient for both the reviewers and the author/review manager. It should allow an easy conversation between the reviewers and the author. Mailing list is good enough, I think.
IMO review mechanism can be much better. Ideally coming to the one should have a change to get a summary of current discussions, open issues and author comments to them. Not sure if we can automate the procedure or it should be a review manager job. Maybe we can combine active discussion going on mailing list with summaries appearing on candidate review page.
5. Review manager have a right to stop a review at any time and make a decision if there is an overwhelming evidence that the library is going to be accepted/rejected.
Ok.
6. If there is not enough reviews with first 2-4 month, the library is rejected due to lack of support.
Hmm, arguable, at least. If it made it to the review, there surely is an interest to the library.
There maybe number of reasons. It possible that the final product did not meet the expectations or original supported disappeared and no one else come through. In any case there is no reason to expect the situation will change. IMO in this case library can do back to the candidate phase and wait for new set of supporters.
7. If there is no review manager found within a year, the library is rejected due to lack of support.
I think, there are several useful libraries in the queue that fit that criteria. My Boost.Log was surely longer than a year without a review manager, and I can't say there's no interest.
I do not believe the Boost.Log waiting for review solely due to the lack of the review manager. I managed last one. And I believe there were couple candidates for the job for new take.
For both 6 and 7, bouncing candidates away won't help the situation.
And the most important objection from my side is that your proposal doesn't change anything to solve the root problem - there are not enough people (or free time of theirs) to manage and write full reviews. I actually makes it a bit worse.
IMO : 1. It should allow more people to write full review given more time and less pressure to do it within short period of time 2. It should allow more libraries to go through the review process ultimately, due to concurrent nature of the process. 3. It should make more people willing to manage the review, given that procedure does not require paying significant attention to the library within the review period. Gennadiy