Child pages
  • Release Practice Guidelines -- Past Versions
Skip to end of metadata
Go to start of metadata

Old Proposal

The information on this page represents early versions and background information gathtered during the development of our current Release Practice Guidelines, and is archived here for refernence purposes.

Release Practices

Sakai's release practices need to evolve in order to meet our community's goals for 2008:

  • Create the highest quality core software.
  • Seek to engage new members of the community.

How it should evolve was the subject of much on-line discussion leading up to the Newport Beach 9th Sakai Conference, further discussion in conference sessions and BOFs, and resulted in three main options being presented to the community for additional discussion through 7 January 2008. The culmination of these discussions are reflected in the proposed release practice guidelines below. Feedback on this proposal is welcome through the end of January 2008, at which time we plan to begin migrating to this practice as we continue forward, unless serious, significant objections are raised.

Comment Period Ends 31 January 2008

To comment on this proposal, please do so by the end of January, and either comment on this page or respond to the "Sakai Release Practices Proposal" thread on the sakai-dev, production, or management email lists.

Proposed Release Practice Guidelines

The community discussion identified two different, but not opposing opinions, and the proposal below strives to satisfy both groups' needs:

  • high-quality, easy to deploy and maintain software with long-term support, and
  • frequent, easy access to new functionality

Proposal:

  • Major Releases – Targeted at organizations for which a stable, long-term deployment is desired, and for which access to maintenance releases as a resource for bug fixes is critical.
    • Predominately driven by the completion of significant improvements and enhancements in Sakai; also includes all the limited improvements in functionality introduced in minor releases since the last major release.
    • Expected that such releases will occur approximately every twelve months.
    • Entire release tested through the coordinated activities of the QA Working Group and early adopters, culminating in an initial release after approximately four to six months of QA, beta, and release candidate tags. (For example, Sakai 2.5 was branched from trunk in October 2007, a beta2 tag is currently available, and the initial release is expected in February 2008.)
    • Bug fixes merged to the release's maintenance branch as appropriate.
    • Periodic tagging of maintenance releases from the maintenance branch, with QA on the maintenance release focused on the bug fixes incorporated.
    • The maintenance branch will be managed and maintenance releases provided for two years after the initial release (or longer, if there is sufficient community interest and resources).
  • Minor Releases – Targeted at organizations who desire access to new functionality, and who have a higher tolerance for and ability to mitigate risk and short-term change.
    • Predominately driven by the readiness of a limited set of new features and functionality.
    • Expected that many such releases will occur between major releases, with a new minor release occurring roughly every one-to-three months.
    • Newly introduced functionality tested through the coordinated activities of the QA Working Group and early adopters, culminating in a release after approximately one month of QA, beta, and release candidate tags.
    • Maintenance branches (and maintenance releases) would be dependent on community volunteers.
      • Organizations that are responsible for the new functionality, and those that intend to use such releases in production, are likely to volunteer such resources.
      • If a particular minor release contains a key set of features, and comes at a convenient time of year, such that a large number of organizations deploy it, then it is very probable that those organizations will also be interested in contributing volunteers to maintain it over a long period of time.
  • General Notes:
    • Trunk will no longer be treated as wild. Bug fixes and minor Tasks can continue to be done to trunk, however, major new functionality and significant enhancements should be developed in branches and merged back to trunk as appropriate.
    • There should be sufficient advance notice of new functionality so that it can be appropriately placed on a roadmap for a future major or minor release of Sakai. The sooner the better, however, features should be put forth for consideration no later than the Feature Freeze for a release.
      • A major release of Sakai would have a Feature Freeze two months before the code is frozen, branched from trunk, and QA begins.
      • A minor release of Sakai would have a Feature Freeze two weeks before the code is frozen, branched from trunk, and QA begins.
      • If new functionality fails to meet a release deadline, then an evaluation will be made to balance the benefits of sliding the deadline versus delaying the timely availability of other new functionality in the release. (Rescheduling new functionality to a later release will generally not significantly delay its availability, as the time period between minor releases can be short.)
    • If sufficient QA resources become available over time, then the scope of QA on maintenance releases and minor releases could certainly be expanded beyond focusing on the changes.
    • What does this mean for 2.6? 2.7? 2.8? ... 3.0?
      • We would immediately solicit suggestions for a limited set of new or enhanced functionality that folks are already working on or would like to get started on, and which could be completed in time for a 2.6 release in April/May 2008, a 2.7 release one to three months after, and so on; however, those time-frames are merely suggestions that need to be adapted to the functionalities' availability.
      • The next major release (3.0?) would likely be in early 2009 at the soonest, but again timing really depends on what folks are prepared to work on between now and then, and how significant of a change that work would imply for Sakai as a whole. If the changes are more of an incremental nature, rather than a re-architecting or re-factoring of major portions of Sakai, then it is likely that a 2.x version (perhaps 2.11?) will be the next major release. Either way it would include all the incremental functionality changes and bug fixes between 2.5 and then, subjected to full QA, and followed up with support for two-years.
    • What is a clear naming convention for easily distinguishing major/minor releases?
      • Do we increment major version number, the first number in a version's name (i.e., the "2" in 2.5.0), for every major release, regardless of how significant the changes are? Or, do we potentially have multiple major releases in the "2" series, such as 2.5.0, 2.11.0, etc., and only increment the major version number when there is sufficiently significant change?
      • Do we name each release separately from its version number, using themes to distinguish between major releases, regardless of how significant the changes are from the last major release? Each major release could introduce a theme and each minor release would be a variation on that theme; however, the change in theme would not be dependent on their being significant changes to warrant incrementing the major version number. For example, 2.5 would be Lambchop, 2.6 Porkloin, 2.7 Beefrib, 2.8 Turkeyleg, 2.9 Ostrichwing, ..., 2.11 would be Blackbird, 2.12 Sparrow, 2.13 Pigeon, 2.14 Heron, 2.15 Parrot, ..., 3.0 would be Oak, 3.1 Pine, 3.2 Maple, 3.3 Larch, and so on...
      • What are some good examples of how other software does this?
    • What is the impact on Feature Branches?
      • The need for Feature Branches is likely to be supplanted by the minor version releases.
      • The minor versions offer a better path for easy adoption and maintenance, as compared to individual projects' Feature Branches.

An example of what the release time-line might look like (and it might help to think of 2.11 as being 3.0 in the image below, or 2.5 as Lambchop and 2.11 as Parrot, as we hadn't really thought too much about naming conventions at the point this was drawn):

For reference, here are the three general options that emerged from discussions at the end of the Newport Beach conference:

Option #1 - Focus is on adding functionality and less so on quality.

  • Basically continuing with our current practices, as exemplified by the 2.5 release process.
  • Only a maintenance branch for releases (e.g., 2.5.x, 2.6.x), no maintenance releases.
  • A 2.6 with a only a small set of feature enhancements and QA'ed across the board, with an initial beta release in June 2008 and a final release in September 2008; a similarly-scoped 2.7 with a beta in November 2008 and final release in February 2009; and so on...
  • This option really doesn't make things any easier than now for deployers.

Option #2 - Focus is on quality and less so on functionality.

  • 2.5 maintenance releases through August 2009, scheduled based on availability of QA resources.
  • A 2.6 in March/April 2009, with beta and release candidates prior to that for use by early adopters in their beginning-of-the-year semesters.
  • Means deployers have to wait a long time for new functionality and there will likely be a lot of it at once, or be comfortable using Feature Branches in the interim.

Option #3 - A compromise focusing on both functionality and quality.

  • Only a maintenance branch for 2.5, no maintenance releases.
  • Minor releases (2.6, 2.7, etc.) multiple times a year that include only a small, limited, incremental set of feature enhancements; QA would be focused mostly on areas of change, not everything would be tested.
  • A major release (3.0) in the future, when there a some big, broad-scope changes that need to be brought into the release, with sufficient lead-time to organize the QA effort required to fully test the distribution.
  • This is sort of a "maintenance release" path for deployers, but brings functionality with it, which can complicate keeping help and training materials up-to-date.

Comments from past versions of this page

Posted by Jason Shao at Jan 25, 2008 23:10

I wonder if major/minor releases is a poor choice in terminology given that many software versioning schemes adopt a major.minor.patch.hotfix style numbering scheme - I suspect the way they're being used in this context is more like the way Ubuntu has adopted their LTS plan, or the way Debian treats Stable vs. Unstable branches. The intention is every year or so there will be a well-tested, everyone base their long term plans off this release.

I'm a little concerned that the distinction between major/minor releases seems to be driven by features rather than change/risk. Addition of a new provisional tool, or optional feature for instance, could represent a huge feature advancement for instance, but if it was fairly orthogonal to the rest of Sakai up to that point potentially relatively low risk. At the same time, would an aggressive internal refactoring that changed implementation details, but (hopefully) not behavior be considered a feature?

I'm also somewhat concerned about branch proliferation under this scheme... we're potentially looking at 3 active long-term/major branches? and probably 3-4 current minor releases? That's a lot of places for a patch to land, and be verified in.

Final thoughts:

  • Many instititutions appear to be looking to upgrade less frequently than every year - does it make sense to have the long-term/major release cycle be every 2 years?
  • I wonder if the intention between the minor/experimental releases is really a reflection of a desire for regularization of something like a milestone/beta release cycle where early adopters can get access to changes which are on their way to stabilization and testing for a major/long-term release. This would have a nice dynamic in allowing us to do major/long-term release QA as we progress through a series of sequential milestone/minor releases.
Posted by Michael Korcuska at Jan 26, 2008 11:47

To be clear the terms "major" and "minor" are not suggested as final terminology but as descriptive labels to get the idea across, for which I think they work. If we agree with the general shape of this proposal then we should start to look for better terminology.

The point about releases being driven by features vs. risk is a good one. We actually discussed that somewhere along the way (not sure where it went in the documentation), saying that the minor releases would be scoped, at least in part, by what could be QA'd, which I take as a measure of change/risk. I think, though, that it's a bit hard to come up with too strict criteria in advance of getting a little experience with a scheme like this.

Yes, we have a few major branches active at the same time. I don't see that as being any different from how we operate today, but maybe I'm missing your point. We do not expect that the minor releases will be maintained. If you want a maintained branch, stay with the major release. You might get lucky with a minor release (if others are running it in production, for example) but you shouldn't count on it. We won't do maintenance releases of the minor release either.

Final thoughts:

  1. I think we'd target every year for now. There still a lot of functionality coming pretty frequently.
  2. Yes, this is part of the intention. Do you see the scheme as articulated (potentially, at least) delivering on that?
Posted by Jason Shao at Jan 29, 2008 19:58

I very much agree with the general shape of the proposal. In particular, I think some of the jewels are:

  • patch releases (though likely everyone knows how I feel about these (smile) )
  • minimum supported windows
  • attempts to segment based on stability/maintenance while consolidating feature testing & bleeding edgers

JA-SIG went through this kind of release process wranging a couple of years ago, and Bill Thompson brought a lot of clarity to that discussion and produced a document and convention which has served us reasonable well http://www.ja-sig.org/wiki/display/UPC/Release+Strategy (it says uPortal, but CAS also follows a similar if not identical release strategy). We also have some explanation of how we view various roles at http://www.ja-sig.org/wiki/display/UPR/Development+Process though the entire process outlined in that document is only somewhat in effect.

The gist of my comment on major branches is related to the burden of maintaining and testing multiple parallel lines of development. Currently we have branch managers who basically merge in most any patch that goes in cleanly. If we had multiple concurrent managed releases, then we'd likely need a number of Release Engineers who would take a firmer lead in coordinating commits, tagging, and testing, rather than risking over-burdening Anthony, Megan and the usual suspects. It may be hard to find 3 such individuals with the time to spare.

I think one important consideration in this discussion is the distinction being drawn between supported and unsupported releases.

  • I would posit that the concept of an "unsupported official release" is somewhat confusing and unintuitive - there's a great tendency to download the "latest version" and blow past the explanatory text.
  • At the same time I think there are a number of schools still on the bleeding edge who would benefit from having common, frequent tags for evaluation and possibly even potential deployment.

I would suggest a slight repositioning:

  • "major" releases become "official" or "GA" releases - something like 2.5.0 and can expect to have patch and security releases - e.g. 2.5.1, 2.5.2. These are prominently displayed on Sakai release pages, etc.
  • "minor" releases become "developmental" releases - equivalent to beta tags. These get tagged something like 2.6.0m1 or by codeword "amazing alphabet drop 1". They are not "click here to download" links off the main page, though there's a way to get them (Eclipse does something similar)
  • No labels

1 Comment

  1. I don't know the current status of this proposal, or the appropriateness of commenting on it, but has there been any thought of allowing tools to have their own independent release cycles? They would probably still participate in the "global" sakai releases, but the times in between they could cut intermediate feature/bugfix releases. This may also help with the backlog of QA at full sakai release times.