Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

...

  • You need to have an account on Jenkins. If you click login->login with open id, a temporary account will be created. This needs to be given permission to the system otherwise you'll just get an error.
  • All artifacts are uploaded and released from Sonatype. There is a special account for Jenkins to actually upload the releases. All other users need to follow the procedures for registering an account.
  • If you add a task in https://jira.sakaiproject.org/secure/Dashboard.jspa under "Infrastructure" the other appropriate access will be granted.
After all of this, all you will need is your OpenId token from Jenkins (on your profile page) to schedule remote tasks and release indies.

Release Kernel and Master corresponding to this release "manually"

After all issues are merged to the appropriate branches, the first step in creating a release is releasing Kernel and Master.

Note
First create a new SAK in the component "Release Management" with the subject "Prepare 2.9.0-b07 release" (or whichever version you're releasing). This is the SAK you'll track all of the changes under. 

All of these 2.9.x projects (the current release) are in Jenkins under the CLE 2x indies tab in the top. Most indies are released semi-automatically (later steps) but these 2 have to be released manually. Kernel has and master both have to be released manually because it is first. Master has to be released manually because the XML needs to be edited prior to releasing to update propertiesunder a special process because properties need to be updated in the XML so the process is a little different. Ideally this could be simplified by

  • Only having the properties in one pom (moving all of the kernel dependencies and properties into master)
  • Have some of the properties protected with profiles
  • Using the versions:update-properties goal

But this is how I've been doing it for this release

Releasing the Kernel

TODO: Review this kernel master process for making a release branch. This was useful during the Release Candidate Phase but when the real release happens, if you don't remember to go back and change the regular pom's then the build will fail as all of the SNAPSHOT's actually do get changed, so probably this should be altered to just include editing these two 'in-place' and notifying the community that the build will be broken for a few hours. Ideally kernel will just be a regular indie with master on top, making it so only master needs to have special consideration.

Pom changes

The kernel is released from the Jenkins page. All of the projects should be properly minimally configured with the Jenkins M2 Release Plugin. This is similar to doing the maven release:prepare release:preform from your local machine. The server does the branch, tags the new release and releases the artifacts to sonatype (since sonatype is the parent for the master and the kernel). Kernel and master are a little different, because as mentioned some manual work is involved in this process.

What I've been doing is releasing from a separate branch

kernel-1.3.x-release and master-2.9-x-release

Code Block
svn rm https://source.sakaiproject.org/svn/kernel/branches/kernel-1.3.x-release
svn cp https://source.sakaiproject.org/svn/kernel/branches/kernel-1.3.x https://source.sakaiproject.org/svn/kernel/branches/kernel-1.3.x-release

svn rm  https://source.sakaiproject.org/svn/master/branches/sakai-2.9.x-release
svn cp https://source.sakaiproject.org/svn/master/branches/sakai-2.9.x/ https://source.sakaiproject.org/svn/master/branches/sakai-2.9.x-release

Check out the kernel release somewhere

svn co https://source.sakaiproject.org/svn/kernel/branches/kernel-1.3.x-release 

Then modify the pom.xml file, changing only the property to whatever the next version will be. The release plugin will change the rest.

For example for the 2.9.0-b07 release (and the 1.3.0-b07 kernel) the properties block 

Code Block
<properties>
  <sakai.kernel.version>1.3-SNAPSHOT</sakai.kernel.version>
  <sakai.jdk.version>1.6</sakai.jdk.version>
  <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

becomes

Code Block
<properties>
  <sakai.kernel.version>1.3.0-b07</sakai.kernel.version>
  <sakai.jdk.version>1.6</sakai.jdk.version>
  <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
Warning

 VERY IMPORTANT - Update the <scm> block in this pom adding -release, otherwise the release plugin will grab the wrong directory. Currently I don't know of any way to override this.

Code Block
    <scm>
        <connection>
            scm:svn:https://source.sakaiproject.org/svn/kernel/branches/kernel-1.3.x-release
        </connection>
        <developerConnection>
            scm:svn:https://source.sakaiproject.org/svn/kernel/branches/kernel-1.3.x-release
        </developerConnection>
        <url>https://source.sakaiproject.org/svn/kernel/branches/kernel-1.3.x-release</url>
    </scm>

svn commit this file 

Jenkins Changes

To start this process, after logging into Jenkins, login into Jenkins at builds.sakaiproject.org. Find the project called "kernel 1.3.x - release" and click on the link "Perform Maven Release"

Image Removed

. This is already setup to point to this directory and be a release build.

Image Added

 

On the page that follows, you need to fill in the release version which you can usually figure out by looking at the tags (https://source.sakaiproject.org/svn/kernel/tags/) for instance "1.3.0-b04". Otherwise ask the release team. This property was defined in the previous step. The development version (which is during most releases is still the current release-SNAPSHOT, for instance "1.3-SNAPSHOT".) You also need to supply your SVN credentials (currently). Hitting release should release the kernel to the repository as well as create the tag.

Note! If any part of the release process fails, you might have to do cleanup (svn rm) manually, deleting broken tags or fixing branches. There is no mvn release:rollback option currently in the maven release plugin. Hopefully the kernel will be released after 15 minutes or so (longer depending on the existing jobs) and appear as a module you can release on sonatype (https://oss.sonatype.org). Before releasing kernel, you should also just release master.

Releasing the Master

After the kernel is released, master can be released. Kernel needs to be released first because master has a dependencyManagement import dependency on the kernel. ( 

JIRA
serverSakai
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-
17162)

<TODO>

Release Indie Tools in Order

<TODO>

...

18921

The properties in master need to be updated similar to kernel, so checkout master

svn co https://source.sakaiproject.org/svn/master/branches/sakai-2.9.x-release 
Code Block
<sakai.version>{this version}</sakai.version>
<sakai.kernel.version>{version of the kernel you defined earlier}</sakai.kernel.version>

 

Warning

 VERY IMPORTANT - Update the <scm> block in this pom, otherwise the release plugin will grab the wrong directory. Currently I don't know of any way to override this.

  • Only update the properties block, making sure to update only the SNAPSHOT properties.
  • After this edit is done, commit this back and go to the Jenkins server. A special job is created presently called "master 2.9.x release". 
  • Now you can click the button on the side "Perform maven release" similar to kernel. Enter the two versions (in this case 2.9.0-b04, and 2.9-SNAPSHOT) as well as your SCM login/password again. Then click "Schedule Maven Release Build"
Warning
Both Kernel and Master should now be in the Repository. Double check that these tags are correct before going any further or you will waste a lot of time! (smile)

NOTE! At this point before you go any further, you need to login to http://oss.sonatype.org and verify that the kernel and master are in the staging repository. It should look like this:

Image Added

There might be other people that have released for OAE but just look for what you released. If they are 100% correct you need to close and release both kernel and master. Click on the close button, type in a description then the release button.

Release Indie Tools in Order

To perform the release, some ruby scripts are needed. You might be best installing RVM (or maybe jRuby) rather than using the ruby in your distribution. This was tested on ruby 1.8.7.

Check out this directory with svn: https://source.sakaiproject.org/contrib/cle-release/scripts/

The script needs two modules nokogiri and httpparty. cd to the directory "scripts/jenkinsrelease". You can install these two gems with

 

Code Block
gem install bundler
bundler install
 
(Note, sometimes you need some libs installed in-order to get these libraries installed. Perhaps it's possible without the libs, but these were avaiable both in Ubuntu and Centos). 
See http://nokogiri.org/tutorials/installing_nokogiri.html
 
For info about that!

In order to get this program to run, you need to create a special configuration file. By default this file is located in ${HOME}/.m2/release.yaml

This is a yaml file with 4 values:

  • jenkinsusername: (the name you use to login to Jenkins)
  • jenkinspassword: Your API key for Jenkins. Click on your name in the upper right, click configure and click "Show API token". This is the password you want here
  • scmusername: The cle-release-team user
  • scmpassword: The cle-release-team password provided!
Code Block
jenkinsusername: jonespm
jenkinspassword: <jenkins apipassword>

scmusername: cle-release-team@collab.sakaiproject.org
scmpassword: <scm password>

 

The file in this directory "tools.yaml" defines the jobs and the release order. This order is on the top of this page.

It has a format like

Code Block
tools:
o01:
- svnpath: https://source.sakaiproject.org/svn/entitybroker
  propname: sakai.entitybroker.version
  releaseversion: 1.5.0-b04
  devversion: 1.5-SNAPSHOT
  jobname: entitybroker%201.5.x
Warning
  • One other thing to check before running this is on the builds server the jenkins user.
    • in /var/lib/jenkins/.subversion/auth/svn.simple this file sometimes seems to remove the password. I'm not sure what's doing this, but it needs the password to be able to run the post fix shell script. If this persists this would have to be fixed some other way.
    • Generally I'll try to commit something as a test from this server before starting the release, or notice it when entitybroker fails right at the end.

 

You need to edit this file and update all of the release versions to match what you edited in master in the previous step. Ideally this process will also be able to release master. 

Then you can release all of the indies with this script. The order is already good to go.

You run this with

Code Block
./clebuild.rb -o o01 -m 2.9.3

Doing this will release entitybroker the item in the o01 group. You can specify multiple jobs to deploy, or if you're feeling lucky you can just do it all! The -m parameter is required and it specifies the master version.

Code Block
./clebuild.rb -o o01,o02,o03,o04,o05,o06,o07,o08,o09,o10 -m 2.9.3

The clebuild script will examine the yaml file, and sort all of the jobs in alphanumeric order. All of the jobs you specify on the command line will be released.

Note! Because of the bug in MRELEASE-449, this script has to checkout the project, run a mvn versions:update-parent on it, and commit it prior to performing the release.

Frequently check the Jenkins page to make sure nothing failed (nothing has a red dot). If something failed, then future groups will not run because each depends on artifacts from a previous group existing, so you'll have a bunch of failures which you probably will need to cleanup. So it's safer to do fewer and keep checking on it.

Info

Some things that could go wrong with to above steps.

  • The SVN key for the release team might not be in the Jenkins home directory. It needs this currently to run t he post release script. If you have failures in the jobs right at the end this is usually the cause. Check to make sure that this is correct
  • If this is your first time running ever (or in awhile), you might just want to run ./clebuild.rb -o o01 first to see what happens, that will limit the amount of possible problems later on
  • <Insert more problems as they come up, it's not 100% fool proof!>

Jenkins takes about 2-3 hours to process and build all of the indies. If everything completes succesfully, login to http://oss.sonatype.org, change the filter to org.sakaiproject and there is usually 2 projects.

I usually do an extensive verification to make sure everything is there and check it off on this page. Indie release order - 2.9. If you release something and it's wrong, that artifact cannot be used and you have to re-release it and master with a new version.

After everything looks good, click on the checkbox next to the bundle and click Close (Wait a few minutes) then Click Release (Waiting again). Sometimes the page times out, but if you refresh, eventually it works.

Now you need to wait a few more hours for the indies to get to maven central.

Release Non-Indie Tools as a package

<TODO>

...

<TODO - THIS SHOULD BE MUCH BETTER, THE PROCESS FOR UPDATING VERSIONS IS BAD AND THIS SECTION NEEDS EDITING>

 

Note

Before you can do the release, all of the indies need to be in the repository. You can check the dates on http://repo.maven.apache.org/maven2/org/sakaiproject/ to see if they get thee. It usually takes about 2 hours after releasing from Sonatype.

This process uses svnmucc for releases which isn't included in CentOS. I had to compile SVN from source and install it. (sad) http://superlinuxadmin.wordpress.com/2012/02/11/upgrade-subversion-1-7-on-centos/

To do the core release:

Run the corerelease.sh script in the cle-release/jenkinsrelease path. This would ideally do everything. I think at this moment it just copies all of the tags for the core. It should be able to do the rest once I get around to it. :/

You run it with this command filling in the appropriate values: ./corerelease.sh https://source.sakaiproject.org/svn/sakai/branches/sakai-2.9.x 2.9.0-b06 2.9-SNAPSHOT SAK-22348. 

That command runs svnmucc and makes all of the branches

Note: The master is being copied to the "master-2.9.0-b06" tag, rather than the "sakai" tag. All of these may need to be changed around eventually for the 2.10 release too to match the project name rather than the sakai name like the indies. Note the name for the master is wrong because it is an indie.

Note: You really need to update the externals: svn propset svn:externals . -F .externals .

After you've updated the pom (before committing) you can run "svn up" which will get all of the new externals from their tags, however the versions are wrong.

I probably used pomversionupdater.bash 2.9-SNAPSHOT 2.9.0-b06 /tmp/sakai-2.9.0-b06 . . . updateparents task didn't seem to work.

Commit all of this now.

(Run this command in the root of the directory to generate the command to commit it)

Code Block
clear ;echo -n "svn ci " ; find ./*/ -maxdepth 0 -type d -print | grep -v ".svn" | xargs -I {} echo -ne  {} "" ; echo -n " pom.xml -m 'Committing sakai core release poms'"

Note

On some OSes the above command gives me problems, remove the svn ci from the beginning and try to run svn ci manually. Remember to also commit the current directory to commit the externals.

 

Now run mvn clean install -Ppack

(Deploy all to shared release)

./ssh/config

Hostname source.sakaiproject.org
User shared_release

Code Block
PWD=`pwd`;THISDIR=`basename $PWD | awk -Fsakai- {'print $2'}`;ssh shared_release mkdir -p /var/www/html/release/$THISDIR/artifacts
PWD=`pwd`;THISDIR=`basename $PWD | awk -Fsakai- {'print $2'}`; scp `find pack -name "*sakai*"` shared_release:/var/www/html/release/$THISDIR/artifacts

Add release java docs (TODO: Clarify this a little more)

This step should be done on the sakai-2.9.x-all branch to get all of the docs (or whatever version)

  • Run svn up to update the files for the new release (or checkout a new full branch for the -all

  • Run the goals
    • mvn clean install javadoc:javadoc 
  • This will generate javadocs in the target/site/apidocs directory, tar up this directory and copy it to the shared release /var/www/html/release/$THISDIR/artifacts as above. 
  • Also uncompress a copy to the apidocs directory.

Update Jira

Visit the SAK project and release update the release notes. Move all open Jiras to next (tentative) version.

Update release documentation

Ideally this work should already be completed. But in case it's not update the following pages:

1. Release page: http://source.sakaiproject.org/release/ 

  • to update page on server, need to ssh to server (managed by Indiana) and find the index.html page in /var/www/html/release/[release id]. For example -  /var/www/html/release/2.9.1
  • May need to get the http://source.sakaiproject.org/release/ link to redirect to the latest release directory (also Indiana)

2. Update this page - https://sakaiproject.org/cle-release  for the latest version number, if relevant, and make sure the document links on the right are up to date. For the page update, Drupal access is needed. For the document links, may have to check with the web site maintainers hired by the Foundation.

3. Confluence release documentation: https://confluence.sakaiproject.org/display/DOC/

4. Implement the Release Communications Plan.

Common Problems and Solutions

<TODO>

Extra information about CLEBUILD SCRIPTS

<TODO>

How the Jenkins Jobs are setup

  •   prerelease script
  •   postrelease script

 

Include Page
Creating an All tag
Creating an All tag