With this document I would like to present my contribution to the
currently ongoing discussion on how to improve the Boost development
process. I tried to focus on changes that may be applied now,
in the 1.35 release time frame, rather than envision a more
revolutionary approach that would take a
long time to be put in practice. For this reason I tried to avoid
placing requirements on the availability of tools that would take a
significant amount of
time to be developed. Most of the concrete changes I
suggest revolve around the future structure of the Boost Subversion
repository and the ways to use it, as I expect that this will already
require a significant amount of work and impact the every day activity
of each developer.
The key elements of my proposal are a three part Subversion
repository structure and a staged development process, inspired by the
one used for GCC.
I also concentrated on the changes
with respect to the current process and did not attempt to provide a
complete description.
The Subversion repository
The Subversion repository should be divided in the following areas:
development
- This shall contain one directory for each library, under which the
customary Subversion directories shall be found: trunk, tags and branches.
Subdirectories will be under full control of their respective library
owners who shall be free to adopt the tagging/branching policy of their
preference.
integration
- This
shall be structured as the current Boost development tree in CVS and
will play the role currently played by both HEAD and the
Release
Candidate branches.
release
- This shall
also be structured as the current Boost development tree and will
mostly serve as a work area for the release manager and as an official
source for the most current stable version.
Developers will be free to choose whether to base their development
environment on the Integration branch, the Release branch or some
previous revision of either, combined with a development branch of
their library, and possibly also of some other libraries theirs depends
on.
Tests will be continuously run on the integration branch to which
developers will merge their work when they see fit, provided they
follow the restrictions placed by the current development stage.
The release cycle
Each release cycle should take three months and be divided in three one
month stages:
During Stage 1 new library additions and major changes to
existing ones shall be allowed. It is understood that during this stage
the integration branch may remain "broken" for short periods of time,
where by broken it is meant that checkins related to one library may
cause tests in other libraries or the overall build to fail. Developers
are not expected to give precedence to handling failed tests in their
own libraries during stage 1.
In Stage 2 only less extensive changes to existing
libraries
shall be allowed. Developers whose checkins to integration cause
breakage are expected to give precedence to fixing such problems. The
Release Manager may ask developers to back out their changes if s/he
feels that either they are too extensive or fixing problems takes too
long. Developers are encouraged to deal with failing tests in their
libraries, but "all green" test results are not an objective for this
stage.
Stage 3 is devoted to bug fixing and to preparing for
release.
All activity on the integration branch shall be devoted to ensure that
tests either pass or are marked expected on all the primary platforms.
When "all green" test results are achieved, the Release
Manager
will merge the integration branch to the release one. Towards
the end of this stage the Release Manager will evaluate, together with
the developers, whether the latest revision on the Release branch is a
reasonable candidate for official release or additional time should be
devoted to try and reach an additional, more complete "all green"
revision. When the agreed upon revision becomes available the Release
Manager proceeds to
package the release. In case of delays an attempt shall be made to
preserve the subsequent release cycle by compressing its stage 1 and
possibly stage 2 phases; should a release be completed earlier than
scheduled the time saved shall be devoted to the subsequent stage 1 and
2 phases.
Supported platforms and testing
At the start of a release cycle the set of currently supported
platforms should be evaluated. The minimum non negotiable criterium for
the inclusion of a platform is the availability of a volunteer to run
the regression tests for the duration of the release cycle.
Ideally one or more platform mantainers should be identified who are
willing to help developers with problems specific to their platform.
This is especially important for less common platforms or compilers.
Other criteria for the selection of supported platforms are beyond the
scope of this document.
The chosen platforms are assigned to a core set of testing volunteers,
who commit to running tests regularly for the duration of the release
cycle. Each single platform/compiler/variant combination should only be
tested by a single volunteer. During stages 1 and 2 test results for
additional combinations are accepted and included in the result
reports, even if they are sporadical. Tests that are older than a given
amount of time are excluded from the reports. During stage 3 only test
results for the supported platforms are included in the reports.
Tests are run continuously on the integration branch, except that the
Release Manager may ask that a test run be performed on the release
branch instead, to ensure that merging didn't cause any problem.
Tools present and future
Apart from issues connected with the move to Subversion, the present
proposal is meant to be applicable without introducing new tools or
modifying the existing ones. On the other hand there are a few
improvements that would be desirable.
First and foremost the reliability of incremental tests should be
verified and fixed.
Developers should be encouraged to run tests locally in their
development environment on as many compilers as possible. This is
already possible with the present tool combination, but in perspective
it would be a very good thing if the very same tools used for official
regression tests could be made to work locally.
Another area where an increase in ease of use would be desirable is the
specification of expected errors. At the very least splitting the
current single file configuration among the libraries should be
considered.
Evolution
Many interesting ideas have emerged from the ongoing discussion that,
while highly desirable, cannot be put in practice in a short time,
mainly due to the requirements they place on the development
infrastructure. While the main focus of this proposal is what can be
put in practice for the next release cycle, I believe that some of
these idea should be considered as an evolution path.
Dependency management and partitioning
In the not so long run the necessity of partitioning Boost in more
manageable packages will have to be addressed. In this respect work on
a plan to takle issues such as inter-library dependency management and
package level versioning should start as soon as possible.
On the other hand even a sensible decision on the course to take is
going to take months and we cannot wait that long to address the
problems of the current development process. In order to limit the risk
of causing delays a complete strategy will have to be devised and
prototyped and all the required tools shall have to be developed and
tested before moving in such a direction.
Personally I'm not convinced that partitioning should take place at the
single library level, except for some of the larger ones. I also fear
that per library, on demand testing is going to prove too complicate to
be viable, however I admit that such topics should be given more
consideration. I'm also convinced that proponents of switching to a
single library oriented process haven't addressed integration issues in
a convincing way.
Financial support
One topic that wasn't addressed so far is how to ensure that sufficient
time is devoted, not only to library development but above all to
management. In my opinion the only definitive answer is professional
involvement, i.e. Boost needs a few individuals to devote paid time to
it. I realize that this is a thorny issue, because it contrasts with
Boost's current status of non-entity, but I'm convinced that part of
the problem has to do with the fact that voluntary work alone cannot
guarantee the degree of continuous commitment that a project this size
requires, the heroic efforts put in by some of the most experienced
people around here notwithstanding.
At the very least companies that use Boost for commercial development
should consider hiring experts if they require a superior support level
or some different form of packaging.
Open issues
This proposal fails to address several important topics:
Testing of release variants. It is certainly desirable that
at least for the most popular platforms regression tests are run for
both the debug and the release variant. The main obstacle is resource
availability, even though the rationalization of the current resources
may produce some benefit in this respect.
It has been correctly pointed out that release quality
criteria should include more than just regression test
results.
Providing tool support for the enforcement of stage policy
should be considered.
Some of the problems faced during the development of 1.34.0
have to do with communication among developers. These should also be
addressed. The languishing of bug reports is in my opinion related to
these problems.
This proposal should address how bugfix
releases should be handled. Hoping that they won't be necessary is not enough :-)
I believe it's way to early to address changes in our
numbering scheme.
I have no idea about what can be found at http://beta.boost.org:8081/ ; I'm currently behind a firewall that blocks all http traffic that doesn't use port 80.
Acknowledgements
The ideas in this document draws on Beman Dawes's and Gennadiy Rozental's proposals, and on the
comments of many participants in the Boost
Development Environment proposal discussion thread.
}}}