Development and Release Practices
Introduction
Boost developers are currently reengineering development tools, workflow, and practices aimed at more timely Boost releases with less heroics required to get releases ready to ship.
This Wiki page focuses on reorganizing our development practices and repository structure, and assumes only existing tools and scripts. Other parallel efforts focus on tools and other aspects of our release process.
Objectives
- Professional, reliable Boost releases suitable for users ranging from simple projects by individuals on up to very large multi-developer projects in large organizations.
- Timely Boost releases on a predictable schedule, with no possibility that problems with a few libraries can delay a release of Boost as a whole.
- The upcoming release available from the Subversion repository at all times, release-ready and including both fixes and new libraries added subsequent to the last official release. Users of this version should not have to wait for a formal release, and should not run the risk of instability associated with purely developmental branches.
- Release preparation does not require super-human effort by the release manager, developers, or testers.
- A stable development environment is maintained at all times, with the development of any one library decoupled from development of other libraries. In particular, there is no "wild west" syndrome where the overall development environment is unstable because the current development state of one or more individual libraries is unstable.
- Boost subset releases are both possible and reasonably easy. For example, Spirit should be able to do a release of it and its dependencies at any time the Spirit team wishes to do so. Note: This implies many of the other objectives, and may also require changes in where Boost header files live within the directory structure. Forwarding headers or links can be used so that such a change is transparent to users.
- release-ready criteria include not only regression test failures, but also inspection failures, tool-chain errors, configuration errors, missing files, and any other detectable errors that reduce release quality or impact development environment stability.
- Error reporting, including pestering, does not produce false positives, such as when failure for a library is due to instability in its dependencies.
- Boost can to scale up to an ever-larger number of libraries, and do so without the incremental weight of each new library having a negative impact on development of other libraries or on release management.
Definitions
Release-ready
A library or branch that passes all appropriate tests and inspections, and meets all other release criteria, such as documentation being up-to-date. Note that the requirements for being release-ready are transitive; if a library has dependent libraries, it isn't considered release-ready if it breaks any of those dependent libraries.
Stable
A library or branch that passes regression tests, but does not necessarily meet other release criteria.
Component
A library or tool. The terms "library" and "component" may be used interchangeably in this document.
Prerequisite component
A library or tool other libraries or tools are dependent upon.
Strategies
- The starting point for each new release is the prior release.
- A release-ready repository tree is maintained at all times.
- Reduce the time lag between when a library developer needs fresh test results and when those results actually become available.
- Use existing tools wherever possible, particularly the tools Boost is already using like bjam and Subversion. Avoid introducing additional complex tool chains.
- Encourage tool maintainers to increase tool robustness.
Repository Organization
Overview:
//svn.boost.org/svn/boost/
trunk
tags/
...
releases/
...
1_34_1
1_35_0
branches/
release
libs/
...
python
serialization
...
Trunk
The main development and test location.
Every effort must be made to keep the main trunk stable. Changes should tested locally by developers before they are committed to the main trunk.
Breaking changes to prerequisite components must be coordinated with developers of dependent libraries, to ensure that changes to all affected libraries are committed to the trunk at the same time. Such multi-library changes may first require special testing against a branch before being merged into the main trunk.
The URL for the main trunk is [source:/trunk http://svn.boost.org/svn/boost/trunk]
Release tags
These are copies of the the release branch made by the release manager, identifying the basis for generating tarballs and such. These copies are locked so that they aren't inadvertently modified.
The URLs for release candidates are in the form http://svn.boost.org/svn/boost/tags/releases/RC_1_35_0_n
The URLs for final releases are in the form http://svn.boost.org/svn/boost/tags/releases/1_35_0
Release branch
The release-ready branch of Boost that will become the next release. Library changes from the trunk are merged into this branch when an update for the library is release-ready.
The release branch is where release testing occurs. The release branch is under the control of the release manager. The release manager sets the policies for when and by whom updated libraries can be merged into the release branch, and may set locks and permissions on the release branch to enforce those policies. The policies will vary at different points of time during the release cycle. The URL for the release branch is http://svn.boost.org/svn/boost/branches/release
Development branches
Location for speculative/experimental/future work on an individual library or set of libraries.
The URL for development branches is in the form http://svn.boost.org/svn/boost/branches/libs/library-name
Development Cycle
The policy of keeping the trunk stable is intended to ensure that the trunk is a stable environment for most development.
Developers can ensure a extra stable development environment by checking out [source:/release/trunk http://svn.boost.org/svn/boost/branch/release] as their working copy, and then switching only the library they are working with to [source:/trunk http://svn.boost.org/svn/boost/trunk] or a library specific development branch. But the goal is that the trunk is stable enough that this procedure isn't usually necessary.
Merging changes from development trunk to release branch
When the next version of a library has been tested and is stable on the main trunk, and is fully release-ready, developers are free to merge their library into the release branch as soon the release manger announce the branch is open for merging. Merging can continue until the release manager announces the release branch is closed later in the release cycle.
Detailed procedure for merging:
- Verify that the library meets all release criteria. Important: Libraries must not be merged into the release branch until they are passing all trunk tests (or the failures marked-up) and are otherwise fully release-ready.
- Merge
- Command line:
- Start with a working copy of https://svn.boost.org/svn/boost/branches/release.
- Go to the release branch directory where the merge will take place.
cd <rel>/<directory_path>
- Merge the directory or a specific file from trunk
svn merge <trunk>/<directory_path>@HEAD svn merge <trunk>/<directory_path>/<file_name>@HEAD
- TortoiseSVN:
- Start with a working copy of https://svn.boost.org/svn/boost/branches/release.
- Select the directory and/or files to be merged.
- Right-click selection for context menu, and "TortoiseSVN" > "Merge..."
- For recent versions of TortoiseSVN (e.g., 1.7.6+):
- Select "Merge a range of revisions"; click "Next >".
- Populate "URL to merge from": https://svn.boost.org/svn/boost/trunk/...
- Leave "Revision range to merge" empty to merge all revisions (typical).
- Click "Next >".
- Click "Test merge" to be sure the setup is correct.
- Click "Merge". (Note that this modifies your working copy, not the repository.)
- For older versions of TortoiseSVN:
- From: https://svn.boost.org/svn/boost/branches/release/...
- To: Uncheck the Use "From:" URL checkbox.
- To: https://svn.boost.org/svn/boost/trunk/...
- To: Head revision
- Click "Dry run" button to be sure setup is correct.
- Click "Merge". (Note that this modifies your working copy, not the repository.)
- Command line:
- Inspect resulting working copy for correctness.
- Run the test with as many compilers as you can.
- Commit working copy changes.
- Watch test results closely; Revert if merge causes any problems whatsoever.
Special case: Prerequisite components
Prerequisite component changes that run any risk of breaking components that depend on them should always be first applied and tested on a branch, and only merged into the main trunk when stable.
The rationale is that changes to the main trunk that break prerequisite components are a serious problem because these changes destabilize the test environment for all components that depend on them. This causes developers of other components to expend needless effort tracking down test failures that aren't their fault.
In an ideal world changes to prerequisite component branches should be tested on various platforms prior to merging into the main trunk. In the absence of test-on-demand facilities, we will have to rely on rapid test cycling and a willingness to revert changes when changes to a prerequisite component inadvertently breaks tests of other components.
Special case: Breaking changes to prerequisite components
This case will require special coordination among affected developers. Details yet to be worked out. It may simply come down to good coordination via the developers mailing list.
Testing
Both development trunk and release branch testing go on continuously, a independent of when releases actually occur. Testers never have to change the repository URLs they test against.
Trunk testing
The main trunk is where most testing resources should be concentrated, since these tests are what tell developers that their library works on all tested platforms and tell the release management team that it is OK to merge a library into the release branch.
Release branch testing
In theory, it should not be necessary to test the release branch at all, since everything merged into it should already be known to be release-ready.
In practice, the release criteria platform/compiler combinations at the least must be tested on the release branch. A single reliable test runner for each release criteria platform/compiler combination may be sufficient, but that's up to the release manager to decide.
Role of the Release Manager
The release manager has the responsibility of generating a timely high-quality release.
The release manager has the authority to make all critical decisions about releases, including setting release criteria, release cycle timing, revisions of problem libraries to prior versions, and all other release related decisions.
Release Schedule
- Quarterly release schedule, with the target release date the 15th of the first month of each quarter.
- If a release is late, that does not slip the date of the next quarter's release.
- If something isn't release-ready for the current release, it is simply held until the next release. The current release is not delayed.
- Being ready means being ready by the cutoff date set by the release manager. That will be relative early in the release cycle.
Rationale
Regular releases that occur on a predictable schedule have many benefits:
- Developers know far in advance when library updates must be ready, and that's both an aid to planning and a motivation to get stuff done.
- Users know far in advance when library updates will be ready, and that helps their planning and reassures them of continuing support.
- Regularity creates a sense of quality and trust in Boost for both developers and users.
Quarterly releases seem about right:
- It's often enough that the process will become smooth.
- It's often enough that developers won't feel pressure to release immature updates.
- It's often enough that users will feel bug fixes and new libraries are being released on a timely basis.
- It isn't so often that users will view Boost as unstable, particularly given the predictable schedule.
- It isn't so often that developers and release managers will become fatigued.
- A quarterly (or similar) schedule has been very successful for both open source and commercial projects.
Acknowledgments
This Wiki page is being maintained by Beman Dawes.
Although the details differ, the release model described here is similar to one proposed by Robert Ramey where he suggested "Each library could be developed at its own pace and schedule."
Refinements of a May, 2006, draft proposal by Beman Dawes were based on comments from Michael Fawcett, Simon Carter, Arkadiy Vertleyb, Pedro Lamarão, Robert Ramey, Anthony Williams, and Thorsten Ottosen.
Discussion during BoostCon 2007 between Beman Dawes, Thomas Witt, Troy Straszheim, Dave Abrahams, Eric Niebler, and Doug Gregor shaped this proposal.
Comments and corrections to this page were contributed by Stefan Seefeld, Peter Dimov, Rene Rivera, Robert Ramey, Michael Caisse, Nicola Musatti, Dave Abrahams, John Maddock, Tony aka Gottlob Frege...
Distributed under the Boost Software License, Version 1.0. (See http://www.boost.org/LICENSE_1_0.txt)
