Release Practices

Sakai's release practices have evolved to meet our community's goals for 2008 and beyond:

• Create the highest quality core software.
• Seek to engage new members of the community.

We plan to meet these by satisfying two key release requirements:

• high-quality, easy to deploy and maintain software with long-term support, and
• frequent, easy access to new functionality

The evolution of our release practices was the subject of much on-line discussion leading up to the Newport Beach 9th Sakai Conference, further discussion in conference sessions and BOFs, and post-conference proposals and community feedback. The discussion was also influenced by the community's recent experience with the Sakai 2.5 release process, as well as areas of past consensuses, including maintenance branches, feature branches, experimental branches, and release or freeze timelines. Two additional topics that were considered (minor releases and kernel releases) are not yet incorporated in the guidelines, as community consensus is still being built in these areas.

The most recent evolution of these guidelines, including thoughts on minor releases, can be viewed here. The Original Sakai Release Practices document, is now deprecated, but also provides useful reference materials for those interested in this topic.

Release Practice Guidelines

The following guidelines outline community expectations and practices for the various Sakai release and branch types. They represent the culmination of extensive community discussion and deliberation, as noted above. It is likely, however, that future needs will require the evolution of these guidelines, so we will periodically review them in conjunction with each major release, or more often, if circumstances dictate. (Also, see "Jira – Sakai's Issue Management and Tracking System" for best practices for using Jira in Sakai development.)

• |#Major Releases
• |#Maintenance Branches
• |#Maintenance Releases
• |#Jira Branches
• |#Feature Branches
• |#Trunk

Major Releases

Major Releases are targeted at organizations for which a stable, long-term deployment is desired, and for which access to Maintenance Releases as a resource for bug fixes is critical.

• Predominately driven by the completion of significant improvements and enhancements in Sakai, developed in |#Jira Branches, since the previous major release.
• Also incorporates bug fixes and minor enhancements developed in trunk, and functionality developed in |#Feature Branches since the last major release.
• If such functionality fails to meet a release deadline, then an evaluation will be made to balance the benefits of adjusting the deadline versus delaying the timely availability of other new functionality in the release.
• Major releases are expected to occur approximately every twelve months, early in the calendar year, and be supported by an active Maintenance Branch with periodic |#Maintenance Releases for two years (or longer, if there are sufficient community resources and interest).
• A Feature Freeze approximately six months before the official release, and roughly two months before Code Freeze and QA begins, will help solidify expectations of what will be in a release. (See the Sakai Roadmap for a current overview of tentatively planned Sakai functionality.)
• QA across the whole application carried out through the coordinated activities of the QA Working Group and early adopters during approximately four to six months of QA, beta, and release candidate tags.

Release Timeline

The Release Timeline (or Freeze Timeline) provides a series of freeze dates to help coordinate activities across the projects and to encourage communication to the community about what is intended to be in a release. The release team reserves the right to adjust deadlines, in order to produce a quality release. (See Sakai 2.6 Release Schedule for an example of this timeline in practice.)

Managing the Release

A major release is based on trunk, and is tagged from a branch of trunk. That branch is cut from trunk at Branch Freeze, roughly ten weeks prior to release, and becomes the maintenance branch for the release. Once the release is branched from trunk, fixes suitable for inclusion in the release should be handled as follows (after the release see #Managing the Maintenance Branch:

If you are managing your own project's release branch:

1. Comment on the Jira providing recommendations for how QA can test your changes.
2. Merge your change to the release branch.
3. Add the appropriate release branch version (e.g., 2.5.x) to the Fix Version list for the Jira.

If you are not managing your own project's release branch:

1. Comment on the Jira providing recommendations for how QA can test your changes.
2. Add the appropriate release branch version to the Target Version list for the Jira.
3. Assign the Jira to the appropriate release's Branch Manager (e.g., "2.5.x Branch Manager").
4. The Branch Manager will add the appropriate release branch version (e.g., 2.5.x) to the Fix Version list for the Jira after they have successfully merged it.

Maintenance Branches

Maintenance Branches are targeted at organizations interested and able to locally incorporate bug fixes as they become available.

• Bug fixes-only, no new functionality will be introduced in the maintenance branch.
• Bugs will be fixed in trunk first, reviewed and tested, and then merged to maintenance branches.

Managing the Maintenance Branch

The following guidelines address when it is appropriate for a Project Team to incorporate fixes in the maintenance branch, either as merges from trunk or to address issues specific to a maintenance branch. They are designed in part to help the community gauge for themselves what level of trust they can place in an individual fix and in the maintenance branch as a whole. These guidelines apply regardless of whether the Project Team is managing its own maintenance branch, or relying on the help of the Branch Manager.

• The maintenance branch contains only bug fixes, no new features. A bug is something that the code intended to do, but is doing incorrectly, incompletely, or is not properly integrated with Sakai. There is code in place to provide a feature, but that feature is broken, missing important aspects, or not well integrated with Sakai's framework or other applications. To be considered a bug, the code must have already attempted to provide some feature, but fell short of actually doing so. All else are features, and should not be merged into the maintenance branch.
• The Project Team responsible for the component(s) needs to have the fix verified against trunk. (If the fix is specific to the maintenance branch, then it needs to be verified against a build based on the maintenance branch.) In general, this process should involve:
  • Test conditions for the fix need to be provided by the Project Team, or the issue's Reporter, in the Jira associated with the fix.
  • The person doing the verification needs to be someone other than those involved in developing the fix. The issue's Reporter is often a good candidate for helping to verify a fix, or you can seek the help of the Sakai QA Working Group in coordinating testing.
  • The person verifying the fix needs to record pertinent information on the verification with comments in Jira:
    • When the verification took place (i.e., date and time).
    • Where the verification was done (e.g., on nightly, a local test instance).
    • On what version of Sakai the verification was performed (e.g., nightly revision 32956).
• Fixes that require changes to the Sakai db schema need to include a fix-specific conversion script and should have notice posted to sakai-dev. The db schema for a Sakai version is locked-in at release time. Any fix for the maintenance branch that subsequently requires a schema change needs to be coordinated with the branch manager to ensure it properly integrates with other schema changes in the branch, and can be utilized independently by an implementor interested in only deploying that fix (and any dependent fixes). The conversion script needs to be labeled in part as "last-release-version" to "maintenance-branch-version", (e.g., 2_4_0-2_4_x). The name should also include a unique version identifier (i.e., 001, 002, 003, ...), provided by the branch manager, to help distinguish and track each db schema change made in the maintenance branch. Also, notification of the community is a critical step, as some deployments may be building from the branch without much oversight and need to be pro-actively alerted to the need to run the conversion script.

In order to encourage general compliance with the above guidelines, we ask everyone to follow this general workflow for Maintenance Branch merges:

• Pre-Requisites: Before an issue's fix can be merged to a Maintenance Branch, it needs to meet the the above criteria; however, you can flag an issue for consideration at any point. (Occassionally there are issues that only affect a Maintenance Branch, and those you can treat as you would a regular Bug and work directly on in the branch.)

1. Anyone
   1. To indicate an issue should be considered for merging, set the Maintenance Branch's status field (e.g., 2.4.x Status, 2.5.x Status) to "Merge".
2. Branch Manager
   1. Verify that the issues is appropriate and ready to merge to the Maintenance Branch.
   2. Merge the issue. (If the merge isn't clean, contact the appropriate Project Team for assistance.)
   3. Set the Maintenance Branch's status field to "Resolved" and set the Fix Version to the appropriate maintenance branch version (e.g., 2.4.x, 2.5.x).
3. Release Team
   1. For issues being included in a maintenance release tag, the maintenance branch Fix Version (e.g., 2.4.x, 2.5.x) needs to be changed to the tag's version number (e.g., 2.5.2-rc01, 2.5.2-rc02, 2.5.2).
4. QA
   1. Test the fix in the Maintenance Branch.
   2. If successful, then set the Maintenance Branch's status field to "Closed". Otherwise, re-open the issue and assign it back to the appropriate Project Team member (or, if that is not clear, then assign it to Former user (Deleted)), along with an explanation of the problem you've encountered.

Maintenance Releases

Maintenance Releases are targeted at organizations for which closely following advancements in the maintenance branch is not feasible or desired.

• Periodically maintenance releases will be tagged from the maintenance branch, probably on the order of every one-to-two months; security-related issues may at times shorten that time frame. Such releases will include fixes for only a selected subset of components, so that the release can be rapidly QA'ed, and maintenance releases can be done frequently. Fixes in the maintenance branch that are not included a particular release will be picked-up in a subsequent release that focuses on a different set of components.
• Limited testing of the release through the coordinated activities of the QA Working Group will be carried out. It will focus on regression testing the areas impacted by bug fixes incorporated in the release.
• The maintenance branch will be managed and maintenance releases provided for two years after the initial x.x.0 release (or longer, if there is sufficient community interest and resources).

Managing the Maintenance Release

Maintenance Release Management is tightly coupled to the Maintenance Branch. (Once we have created our first Maintenance Release, likely Sakai 2.5.1, there may be more details to share on this point.)

Jira Branches

Jira Branches (sometimes referred to as Experimental Branches or SAK Branches) should be used when there is interest in modifying Sakai in a manner that requires extensive work, a need to not adversely impact Sakai developers in general, for which there may or may not be general agreement, and/or folks are just interested in experimenting with something but unsure yet of its applicability. Such work will be supported by the creation of Jira Branches, which can later be merged, if desirable, back to trunk, as trunk is only for bug fixing and minor improvements or enhancements.

Managing Jira Branches

Here are some general guidelines to follow for managing work in Jira Branches:

• A Jira (of issue type Branch) should be created that proposes what will be done in the branch and explains why it's a good idea. It should also include an estimate of when the work is expected to be completed and ready for merging back to the trunk.
• There should be a general announcement of the proposal on Sakai-dev, which should contain a pointer to the Jira, where discussion can take place; discussion can take place on sakai-dev as well, and the proposer should summarize such discussion in the Jira or provide a pointer to the thread.
• The Project Leads for the areas of Sakai that the branch will impact, and any interested parties in the Sakai community, should review the proposal.
• A request for a branch in SVN to support the work should be sent to svn@collab.sakaiproject.org. The impacted project lead(s) should be cc'ed on the request, so they are aware of the new branch.
• As work proceeds on the branch, if you would like to use additional Jira Issues to help manage your work, the one caveat is to use Sub-Tasks of the Branch Jira, rather independent issues, in order to aid in distinguishing work going on in the branch from that in trunk. You should also use the generic "branch" version for Fix Versions; this should be updated to Nightly/Trunk-SVN when your work is merged back in.
• When work on the branch is completed, the issue is Resolved, and ready for review, another announcement should be sent to sakai-dev to alert folks to review the work and express opinions on its return to trunk. At least three working days should allowed for comment and voting.
• The Project Leads should also review the work and there may be some back-and-forth required here to get the branch into shape for agreement on merging it. If work is rejected, a reason must be provided.
• If everything checks out okay, then the Project Leads and the branch developers will work together to merge the changes into trunk, and send out a final notice when the merge is completed and the issue is Closed.
  • The Branch issues should be converted to a Task when the decision to merge it back to trunk is made, and the Fix Version should be set appropriately.
  • When the merge is completed, the issue should be Closed. Any subsequent issues that arise, should be dealt with via new Jira Bugs, Tasks, Feature Requests, etc., rather than reopening the original issue.
  • Note that code needs to properly licensed before it can be merged to main Sakai trunk, so it behooves those developing branches to ensure they have completed the necessary employer and employee contributor licensing agreements (CLAs) early in this process. For assistance with this, please contact the Sakai Project Coordinator (Former user (Deleted)).

Feature Branches

Feature Branches provide a way for folks to access functionality developed for a future release of Sakai in a current release.

Projects can provide early access to features and functionality destined for a future release in a form compatible with existing releases through Feature Branches. These branches contain work that is not appropriate for a release's maintenance branch, as it goes beyond bug fixes. They provides folks with a way to utilize forthcoming functionality in their existing deployment, without having to wait for the next release.

These branches are offered as-is. There is no official QA of this work. It should be noted, however, that the need for such branches is often driven by local needs of Project Team members to provide such functionality locally sooner, rather than later, in production. Hence, you can expect that at least one institution is testing and running the code in production.

Managing Feature Branches

Project Teams are strongly encouraged to merge appropriate bug fixes from maintenance branches and/or trunk into the Feature Branch as often as possible. Project Teams are unlikely to support a Feature Branch in the long run, however, as their priorities are likely to shift as the next release becomes available.

• Naming Scheme:
  • Name the feature branch after the release with which the work is compatible. For example, work that is intended to be used in a Sakai 2.4 deployment should be done in a post-2.4 branch.
  • Name tags on the feature branch iteratively after the branch. For example, tags along a project's post-2.4 branch would be named: post-2.4-a, post-2.4-b, post-2.4-c, etc.
  • In the SVN world, the naming scheme translates into names like post_2-4, post_2-4-a, etc.; this is modeled on the "sakai_<x>" branches and tags in SVN.

Trunk

|#Major Releases come from trunk, however, direct work in trunk should be restricted to bug fixing and small tasks. More significant tasks or experimental work should first be tackled in |#Jira branches.

Also, work should only be checked into trunk that is appropriately covered by Contributor Licensing Agreements (CLAs). This includes patches contributed by others (perhaps attached to a Jira or an email), which you might merge yourself after reviewing them. In practice, no one will be given committ access to trunk until they have completed the appropriate individual and orgainzational CLAs. (For questions regarding licensing in Sakai please contact the Sakai Project Coordinator (Former user (Deleted)) or the Licensing Working Group.)