{{{ #!html

The Boost Development Process - A proposal

Abstract

With this document I would like to present my contribution to the currently ongoing discussion on how to improve the Boost development process. I tried to focus on changes that may be applied now, in the 1.35 release time frame, rather than envision a more revolutionary approach that would take a long time to be put in practice. For this reason I tried to avoid placing requirements on the availability of tools that would take a significant amount of time to be developed. Most of the concrete changes I suggest revolve around the future structure of the Boost Subversion repository and the ways to use it, as I expect that this will already require a significant amount of work and impact the every day activity of each developer.
The key elements of my proposal are a three part Subversion repository structure and a staged development process, inspired by the one used for GCC.
I also concentrated on the changes with respect to the current process and did not attempt to provide a complete description.

The Subversion repository

The Subversion repository should be divided in the following areas:
development
 +spirit
 | +trunk
 | | +boost
 | | | +spirit
 | | +libs
 | |   +spirit
 | +tags
 | +branches
 +serialization
Developers will be free to choose whether to base their development environment on the Integration branch, the Release branch or some previous revision of either, combined with a development branch of their library, and possibly also of some other libraries theirs depends on.
Tests will be continuously run on the integration branch to which developers will merge their work when they see fit, provided they follow the restrictions placed by the current development stage.

The release cycle

Each release cycle should take three months and be divided in three one month stages:

Supported platforms and testing

At the start of a release cycle the set of currently supported platforms should be evaluated. The minimum non negotiable criterium for the inclusion of a platform is the availability of a volunteer to run the regression tests for the duration of the release cycle. Ideally one or more platform mantainers should be identified who are willing to help developers with problems specific to their platform. This is especially important for less common platforms or compilers. Other criteria for the selection of supported platforms are beyond the scope of this document.
The chosen platforms are assigned to a core set of testing volunteers, who commit to running tests regularly for the duration of the release cycle. Each single platform/compiler/variant combination should only be tested by a single volunteer. During stages 1 and 2 test results for additional combinations are accepted and included in the result reports, even if they are sporadical. Tests that are older than a given amount of time are excluded from the reports. During stage 3 only test results for the supported platforms are included in the reports.
Tests are run continuously on the integration branch, except that the Release Manager may ask that a test run be performed on the release branch instead, to ensure that merging didn't cause any problem.

Tools present and future

Apart from issues connected with the move to Subversion, the present proposal is meant to be applicable without introducing new tools or modifying the existing ones. On the other hand there are a few improvements that would be desirable.
First and foremost the reliability of incremental tests should be verified and fixed.
Developers should be encouraged to run tests locally in their development environment on as many compilers as possible. This is already possible with the present tool combination, but in perspective it would be a very good thing if the very same tools used for official regression tests could be made to work locally.
Another area where an increase in ease of use would be desirable is the specification of expected errors. At the very least splitting the current single file configuration among the libraries should be considered.

Evolution

Many interesting ideas have emerged from the ongoing discussion that, while highly desirable, cannot be put in practice in a short time, mainly due to the requirements they place on the development infrastructure. While the main focus of this proposal is what can be put in practice for the next release cycle, I believe that some of these idea should be considered as an evolution path.

Dependency management and partitioning

In the not so long run the necessity of partitioning Boost in more manageable packages will have to be addressed. In this respect work on a plan to takle issues such as inter-library dependency management and package level versioning should start as soon as possible.
On the other hand even a sensible decision on the course to take is going to take months and we cannot wait that long to address the problems of the current development process. In order to limit the risk of causing delays a complete strategy will have to be devised and prototyped and all the required tools shall have to be developed and tested before moving in such a direction.
Personally I'm not convinced that partitioning should take place at the single library level, except for some of the larger ones. I also fear that per library, on demand testing is going to prove too complicate to be viable, however I admit that such topics should be given more consideration. I'm also convinced that proponents of switching to a single library oriented process haven't addressed integration issues in a convincing way.   

Financial support

One topic that wasn't addressed so far is how to ensure that sufficient time is devoted, not only to library development but above all to management. In my opinion the only definitive answer is professional involvement, i.e. Boost needs a few individuals to devote paid time to it. I realize that this is a thorny issue, because it contrasts with Boost's current status of non-entity, but I'm convinced that part of the problem has to do with the fact that voluntary work alone cannot guarantee the degree of continuous commitment that a project this size requires, the heroic efforts put in by some of the most experienced people around here notwithstanding.
At the very least companies that use Boost for commercial development should consider hiring experts if they require a superior support level or some different form of packaging.

Open issues

This proposal fails to address several important topics:

Acknowledgements

The ideas in this document draws on Beman Dawes's and Gennadiy Rozental's proposals, and on the comments of many participants in the Boost Development Environment proposal discussion thread. }}}