Skip to end of metadata
Go to start of metadata
Note: The release process changed on release 2.9.0-b03. Please reference these instructions for the old guide: Sakai CLE release guide (pre 2.9)

The current "full" release process (2.9.0-b03+) basically involves these steps:

This guide is based on the release order defined at Indie release order - 2.9. Please refer to that and update that when any new tools are added to indie.

Pre-Release step: Obtaining Credentials

In order to perform a release, you must have credentials on the build server (http://builds.sakaiproject.org:8080/) as well as access to commit to every branch/tag in source.sakaiproject.org.  To just release an individual tool will should only require access to Jenkins. This guide will only provide brief details about this process, if you are seriously interested in getting more involved in the release process, have a CLA on file and are an active member of the Sakai CLE community we'd welcome the participation in the release process.

  • You need to have an account on Jenkins. If you click login->login with open id, a temporary account will be created. This needs to be given permission to the system otherwise you'll just get an error.
  • All artifacts are uploaded and released from Sonatype. There is a special account for Jenkins to actually upload the releases. All other users need to follow the procedures for registering an account.
  • If you add a task in https://jira.sakaiproject.org/secure/Dashboard.jspa under "Infrastructure" the other appropriate access will be granted.
After all of this, all you will need is your OpenId token from Jenkins (on your profile page) to schedule remote tasks and release indies.

Release Kernel and Master corresponding to this release "manually"

After all issues are merged to the appropriate branches, the first step in creating a release is releasing Kernel and Master.

Icon
First create a new SAK in the component "Release Management" with the subject "Prepare 2.9.0-b07 release" (or whichever version you're releasing). This is the SAK you'll track all of the changes under. 

All of these 2.9.x projects (the current release) are in Jenkins under the CLE 2x indies tab in the top. Most indies are released semi-automatically (later steps) but these 2 have to be released manually. Kernel and master both have to be released manually under a special process because properties need to be updated in the XML so the process is a little different. Ideally this could be simplified by

  • Only having the properties in one pom (moving all of the kernel dependencies and properties into master)
  • Have some of the properties protected with profiles
  • Using the versions:update-properties goal

But this is how I've been doing it for this release. 

Releasing the Kernel

TODO: Review this kernel master process for making a release branch. This was useful during the Release Candidate Phase but when the real release happens, if you don't remember to go back and change the regular pom's then the build will fail as all of the SNAPSHOT's actually do get changed, so probably this should be altered to just include editing these two 'in-place' and notifying the community that the build will be broken for a few hours. Ideally kernel will just be a regular indie with master on top, making it so only master needs to have special consideration.

Pom changes

The kernel is released from the Jenkins page. All of the projects should be properly minimally configured with the Jenkins M2 Release Plugin. This is similar to doing the maven release:prepare release:preform from your local machine. The server does the branch, tags the new release and releases the artifacts to sonatype (since sonatype is the parent for the master and the kernel). Kernel and master are a little different, because as mentioned some manual work is involved in this process.

What I've been doing is releasing from a separate branch

kernel-1.3.x-release and master-2.9-x-release

Check out the kernel release somewhere

svn co https://source.sakaiproject.org/svn/kernel/branches/kernel-1.3.x-release 

Then modify the pom.xml file, changing only the property to whatever the next version will be. The release plugin will change the rest.

For example for the 2.9.0-b07 release (and the 1.3.0-b07 kernel) the properties block 

becomes

Icon

 VERY IMPORTANT - Update the <scm> block in this pom adding -release, otherwise the release plugin will grab the wrong directory. Currently I don't know of any way to override this.

svn commit this file 

Jenkins Changes

To start this process, login into Jenkins at builds.sakaiproject.org. Find the project called "kernel 1.3.x - release" and click on the link "Perform Maven Release". This is already setup to point to this directory and be a release build.

 

On the page that follows, you need to fill in the release version which you can usually figure out by looking at the tags (https://source.sakaiproject.org/svn/kernel/tags/) for instance "1.3.0-b04". Otherwise ask the release team. This property was defined in the previous step. The development version (which is during most releases is still the current release-SNAPSHOT, for instance "1.3-SNAPSHOT".) You also need to supply your SVN credentials (currently). Hitting release should release the kernel to the repository as well as create the tag.

Note! If any part of the release process fails, you might have to do cleanup (svn rm) manually, deleting broken tags or fixing branches. There is no mvn release:rollback option currently in the maven release plugin. Hopefully the kernel will be released after 15 minutes or so (longer depending on the existing jobs) and appear as a module you can release on sonatype (https://oss.sonatype.org). Before releasing kernel, you should also just release master.

Releasing the Master

After the kernel is released, master can be released. Kernel needs to be released first because master has a dependencyManagement import dependency on the kernel.  SAK-18921 - Import kernel dependencies rather than defining them ( Closed)

The properties in master need to be updated similar to kernel, so checkout master

svn co https://source.sakaiproject.org/svn/master/branches/sakai-2.9.x-release 

 

Icon

 VERY IMPORTANT - Update the <scm> block in this pom, otherwise the release plugin will grab the wrong directory. Currently I don't know of any way to override this.

  • Only update the properties block, making sure to update only the SNAPSHOT properties.
  • After this edit is done, commit this back and go to the Jenkins server. A special job is created presently called "master 2.9.x release". 
  • Now you can click the button on the side "Perform maven release" similar to kernel. Enter the two versions (in this case 2.9.0-b04, and 2.9-SNAPSHOT) as well as your SCM login/password again. Then click "Schedule Maven Release Build"
Icon
Both Kernel and Master should now be in the Repository. Double check that these tags are correct before going any further or you will waste a lot of time! (smile)

NOTE! At this point before you go any further, you need to login to http://oss.sonatype.org and verify that the kernel and master are in the staging repository. It should look like this:


There might be other people that have released for OAE but just look for what you released. If they are 100% correct you need to close and release both kernel and master. Click on the close button, type in a description then the release button.

Release Indie Tools in Order

To perform the release, some ruby scripts are needed. You might be best installing RVM (or maybe jRuby) rather than using the ruby in your distribution. This was tested on ruby 1.8.7.

Check out this directory with svn: https://source.sakaiproject.org/contrib/cle-release/scripts/

The script needs two modules nokogiri and httpparty. cd to the directory "scripts/jenkinsrelease". You can install these two gems with

 

In order to get this program to run, you need to create a special configuration file. By default this file is located in ${HOME}/.m2/release.yaml

This is a yaml file with 4 values:

  • jenkinsusername: (the name you use to login to Jenkins)
  • jenkinspassword: Your API key for Jenkins. Click on your name in the upper right, click configure and click "Show API token". This is the password you want here
  • scmusername: The cle-release-team user
  • scmpassword: The cle-release-team password provided!

 

The file in this directory "tools.yaml" defines the jobs and the release order. This order is on the top of this page.

It has a format like

Icon
  • One other thing to check before running this is on the builds server the jenkins user.
    • in /var/lib/jenkins/.subversion/auth/svn.simple this file sometimes seems to remove the password. I'm not sure what's doing this, but it needs the password to be able to run the post fix shell script. If this persists this would have to be fixed some other way.
    • Generally I'll try to commit something as a test from this server before starting the release, or notice it when entitybroker fails right at the end.

 

You need to edit this file and update all of the release versions to match what you edited in master in the previous step. Ideally this process will also be able to release master. 

Then you can release all of the indies with this script. The order is already good to go.

You run this with

Doing this will release entitybroker the item in the o01 group. You can specify multiple jobs to deploy, or if you're feeling lucky you can just do it all! The -m parameter is required and it specifies the master version.

The clebuild script will examine the yaml file, and sort all of the jobs in alphanumeric order. All of the jobs you specify on the command line will be released.

Note! Because of the bug in MRELEASE-449, this script has to checkout the project, run a mvn versions:update-parent on it, and commit it prior to performing the release.

Frequently check the Jenkins page to make sure nothing failed (nothing has a red dot). If something failed, then future groups will not run because each depends on artifacts from a previous group existing, so you'll have a bunch of failures which you probably will need to cleanup. So it's safer to do fewer and keep checking on it.

Icon

Some things that could go wrong with to above steps.

  • The SVN key for the release team might not be in the Jenkins home directory. It needs this currently to run t he post release script. If you have failures in the jobs right at the end this is usually the cause. Check to make sure that this is correct
  • If this is your first time running ever (or in awhile), you might just want to run ./clebuild.rb -o o01 first to see what happens, that will limit the amount of possible problems later on
  • <Insert more problems as they come up, it's not 100% fool proof!>

Jenkins takes about 2-3 hours to process and build all of the indies. If everything completes succesfully, login to http://oss.sonatype.org, change the filter to org.sakaiproject and there is usually 2 projects.

I usually do an extensive verification to make sure everything is there and check it off on this page. Indie release order - 2.9. If you release something and it's wrong, that artifact cannot be used and you have to re-release it and master with a new version.

After everything looks good, click on the checkbox next to the bundle and click Close (Wait a few minutes) then Click Release (Waiting again). Sometimes the page times out, but if you refresh, eventually it works.

Now you need to wait a few more hours for the indies to get to maven central.

Release Non-Indie Tools as a package

<TODO - THIS SHOULD BE MUCH BETTER, THE PROCESS FOR UPDATING VERSIONS IS BAD AND THIS SECTION NEEDS EDITING>

 

Icon

Before you can do the release, all of the indies need to be in the repository. You can check the dates on http://repo.maven.apache.org/maven2/org/sakaiproject/ to see if they get thee. It usually takes about 2 hours after releasing from Sonatype.

This process uses svnmucc for releases which isn't included in CentOS. I had to compile SVN from source and install it. (sad) http://superlinuxadmin.wordpress.com/2012/02/11/upgrade-subversion-1-7-on-centos/

To do the core release:

Run the corerelease.sh script in the cle-release/jenkinsrelease path. This would ideally do everything. I think at this moment it just copies all of the tags for the core. It should be able to do the rest once I get around to it. :/

You run it with this command filling in the appropriate values: ./corerelease.sh https://source.sakaiproject.org/svn/sakai/branches/sakai-2.9.x 2.9.0-b06 2.9-SNAPSHOT SAK-22348. 

That command runs svnmucc and makes all of the branches

Note: The master is being copied to the "master-2.9.0-b06" tag, rather than the "sakai" tag. All of these may need to be changed around eventually for the 2.10 release too to match the project name rather than the sakai name like the indies. Note the name for the master is wrong because it is an indie.

Note: You really need to update the externals: svn propset svn:externals . -F .externals .

After you've updated the pom (before committing) you can run "svn up" which will get all of the new externals from their tags, however the versions are wrong.

I probably used pomversionupdater.bash 2.9-SNAPSHOT 2.9.0-b06 /tmp/sakai-2.9.0-b06 . . . updateparents task didn't seem to work.

Commit all of this now.

(Run this command in the root of the directory to generate the command to commit it)

Icon

On some OSes the above command gives me problems, remove the svn ci from the beginning and try to run svn ci manually. Remember to also commit the current directory to commit the externals.

 

Now run mvn clean install -Ppack

(Deploy all to shared release)

./ssh/config

Hostname source.sakaiproject.org
User shared_release

Add release java docs (TODO: Clarify this a little more)

This step should be done on the sakai-2.9.x-all branch to get all of the docs (or whatever version)

  • Run svn up to update the files for the new release (or checkout a new full branch for the -all

  • Run the goals
    • mvn clean install javadoc:javadoc 
  • This will generate javadocs in the target/site/apidocs directory, tar up this directory and copy it to the shared release /var/www/html/release/$THISDIR/artifacts as above. 
  • Also uncompress a copy to the apidocs directory.

Update Jira

Visit the SAK project and release update the release notes. Move all open Jiras to next (tentative) version.

Update release documentation

Ideally this work should already be completed. But in case it's not update the following pages:

1. Release page: http://source.sakaiproject.org/release/ 

  • to update page on server, need to ssh to server (managed by Indiana) and find the index.html page in /var/www/html/release/[release id]. For example -  /var/www/html/release/2.9.1
  • May need to get the http://source.sakaiproject.org/release/ link to redirect to the latest release directory (also Indiana)

2. Update this page - https://sakaiproject.org/cle-release  for the latest version number, if relevant, and make sure the document links on the right are up to date. For the page update, Drupal access is needed. For the document links, may have to check with the web site maintainers hired by the Foundation.

3. Confluence release documentation: https://confluence.sakaiproject.org/display/DOC/

4. Implement the Release Communications Plan.

Common Problems and Solutions

<TODO>

Extra information about CLEBUILD SCRIPTS

<TODO>

How the Jenkins Jobs are setup

  •   prerelease script
  •   postrelease script

 

Creating a "-all" tag

What is a "-all" tag? It includes the source code for all the Indies. Without the "-all" tag Indies are included as binaries. Using a -all is only necessary if you need the source code of the Indie tools.

Overview: You will copy the "-all" for the previous release, for example for CLE 2.9.3 we copied the 2.9.2-all. Then update the .externals file and the pom files to reflect the update to the new version and the versions of all the Indies. 

Steps:

  1. Create a Jira in https://jira.sakaiproject.org/browse in the SAK project and of type Task. Title it something like "Create the 2.9.3 Tag".
  2. Copy the -all tag from the previous release. Include the Jira number from step 1 in the comments. Example: svn cp https://source.sakaiproject.org/svn//sakai/tags/sakai-2.9.2-all/ https://source.sakaiproject.org/svn//sakai/tags/sakai-2.9.3-all/ -m "SAK-23885 copied from 2.9.2-all tag, ready to mod" 
  3. Check out the new copy. Example: svn co --ignore-externals https://source.sakaiproject.org/svn//sakai/tags/sakai-2.9.3-all/
  4. Open .externals with your favorite editor. Update all the versions to the latest. A global search and replace of "2.9.2" with "2.9.3" (as examples, old version vs new version) will fix most, but not all versions. You can look for the latest tag, to be sure you are using the correct version numbers, in https://source.sakaiproject.org/svn/(project/tags/) , the project is the Indy project. For example, at the time of these instructions of CLE 2.9.3, the version for content-review is 2.9.5.  https://source.sakaiproject.org/svn/content-review/tags/  . Another, perhaps canonical, place to look for the correct version numbers is in the Master POM.  For CLE 2.9.3 the example is : https://source.sakaiproject.org/svn/master/tags/master-2.9.3/pom.xml
  5. Fix your POM files with the correct versions : 
    pom.xml
    deploy/pom.xml
    deploy/common/pom.xml
    deploy/shared/pom.xml
  6. type exactly, including the trailing period : svn propset svn:externals -F .externals .   
  7. as an example, replace with Jira id from step #1: svn commit -m "SAK-23885 - updated pom files"
  8. svn up
Verify:
mvn clean install sakai:deploy 
(verify build) 

(start tomcat and verify tools work)

  • No labels