Child pages
  • University of Murcia's upgrading experience from 2.9 to 10

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Section
Column
width65%

Upgrading Experience, from 2.9 to 10


Here we are again !! After sharing our experience upgrading from 2.7 to 2.9 now we are ready to share our experience upgrading to the brand new Sakai 10 at the University of Murcia - SPAIN (Universidad de Murcia http://www.um.es - https://aulavirtual.um.es). In the first experience we share a lot of info about the local customizations we have, and also spread them out when their were useful for others. I'm sure there are other customizations ready to be shared and also new ones.

We also want to share our route map so that others facing the same need for upgrade can use them, reduce their time and learn from it. At the same time, we would love to hear from you suggestions about tasks that might also be included in our upgrading process, or your advice for facing those tasks.

This page will be made by the technical staff and it is mainly focused on their concerns around the upgrade process. We also intend to mention all the other staff related to this process and the tasks concerned to them.

 


Upgrading Plan

After Sakai released its 10 version, we set a clear goal for our TEAM, move our deployed version (2.9.3) to the latest one. However, the first step is to write a simple plan, unfortunately for us and everyone in the Sakai community, Sakai upgrades do not arrives by clicking a 'magic button'. The main task concerns technical aspects, we have developed a lot of custom tools and have also changed some pieces of the original Sakai code. But, there are other tasks involved, so we are going to mention all of them at this master plan.

Evaluating Sakai 10

What do we really know about the new version? This is our first goal. When we started to migrate from 2.7 to 2.9 we didn't know too much about the 2.9 version, but know the situation is pretty different. The migration from 2.7 to 2.9 let us get much more involved in the community and do a lot of work into trunk that now is part of Sakai 10. So we know what's new in Sakai 10 because we developed some of the new features (of course not all). Also we were collaborating in different test fests during the Sakai 10 quality assurance process, and this let us know more about Sakai 10.

Look & Feel

Sakai 10 is not a challenge in terms of look&feel is pretty similar to 2.9, but we need to add some changes to get our users feel that something has change. Probably the main change of the look&feel will be on the gateway page, but not in the internal skin, we have to wait for Sakai 11 and the Morpheus project to see a new responsive design for Sakai.

New Features

There are a lot of new features in Sakai 10, some of them really interesting and also necessary. If you are interested in wht's new on Sakai 10, please look at the release notes.

Major Code Changes

Sakai 10 comes with some important upgrades, such as Hibernate 3.6.10.Final and Spring 3.2.3.RELEASE. Because of these libraries are deployed in shared folder, we also need to upgrade our tools in order to use these new versions. In the other hand another significant change is that the Sakai 10 release process only api artifacts were deployed into public sonatype repository.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keyKNL-515
Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keyKNL-517

Release Process

Developing Process. (Jul-Sep, 2014)

The development start July 1th, 2014. I don't know how much time it's going to take us release our customized version, but it's time to see if all the work done with the previous migration has a significative impact in this process.

Testing Process. (Jul-Sep, 2014)

During test process we found some issues and we decided to use a label in JIRA to keep track of them. "UMUtoSAK10" is the label and you could check the list of jiras at the right side of this page.

Load testing

I'm going to do some load testing before September and then a more specific tests.

Information Process. (Sep, 2014)

At the begining of the previous migration we wanted to make a promotional process about the steps of migration thinking in marketing perspective, but later on we realized that users don't care about migration, they only care about it works or lose data. So, for this migration we won't do anything but notifying users briefly about new features and when the change will be made, and spend time in quality assurance rather than in marketing things.

The Change. (Sep, 2014)

The best date for us to change from one version to another is right now, before the new term start in September. Actually Jul, 31th is the best date to make the migration, but that date seems to be impossible. Let's see what happend !!

Finally the migration will be Oct, 25th, is far for our initial point first because of the release dates of Sakai 10, but also due to the amount of bugs that we had to solve. We are going to be out with Sakai 10.1 (with a lot of patches added), and we are planning a minor upgrade to 10.3 at the beginning of 2015.

Info
Migration
in progress
done !!

 


Final Conclusions

Since we migrated to Sakai 2.9 we were closed to the Sakai community and its development, but it is hard to know every change and any of them could affect your migration process. The best part of this process is that now almost everything that does not work for us is a problem that sonner or later is solved in Sakai, but the bad thing is that there is a evident lack of QA in Sakai. We found some bugs that are pretty obvious, and made us think that nothing was tested after the feature was commited to trunk.

You could check all jiras opened by us during this process (more than 80), and this will continue during next weeks because with Sakai 10 in production we will discover more problems.

Finally we have to use the classic Roster tool because Roster 2 still have problems for us after 3 versions of Sakai 10, this is a good summary of the migration process, and a fantastic evidence of the lack of QA.

 

After a few days running the new version of Sakai there are not so much issues around, I think the change was pretty successful but there was a none reported issue with Sakai 10.1 that caused us a lot of troubles during our first hours with the new version. We've been working hard to adopt Sakai 10.1 to our needs so when 10.2 was released we've taken a fast look to the issues added (most of them were added to our 10.1 version too) but miss the most important one because it was fixed but not reported anywhere. Take a look at this jira.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-27571

Did you see something strange? This JIRA is resolved and fixed versions are 10.1 and 10.2, why?

The thing is that this JIRA was solved for 10.1 version, but it has a bug that was fixed for 10.2, but this bug it wasn't reported anywhere. The only clue that you have is the 10.2 in the fix version and of course the subversion tab that shows a commit into 10.x at the beginning of October (after 10.1 was released).

For us this fix was critical, our production server went down so we expected that a fix so critical was clearly mentioned in the release notes but in this case it wasn't even in this filter with "Blocker" issues fixed.

Jira
serverSakai
columnskey,summary,type,priority,status,resolution
maximumIssues5
jqlQuerycategory = Sakai and status in (resolved, verified, closed) and fixversion = "10.2" and priority = Blocker
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c

 

Column
width35%
Panel
bgColorwhite
borderStylenone

Check out our previous experience upgrading to Sakai 2.9 !!

Panel
borderColordarkblue
bgColorwhite
titleColorwhite
titleBGColordarkblue
titleTable of Contents
borderStylesolid

Table of Contents
maxLevel3
outlinetrue
exclude.*Logbook.*

Panel
bgColorwhite
titleBGColor#f1f1f1
titleRelated Jiras
borderStyledashed

Jira
serverSakai
columnskey,summary,status,resolution
maximumIssues10
jqlQuerylabels = UMUtoSAK10
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c

Panel
borderColorgreen
bgColorwhite
titleColorwhite
titleBGColorgreen
titleUpgrade's Logbook
borderStylesolid

Table of Contents
maxLevel3
include.*Logbook.*
stylesquare


Migration in progressDONE !!

 

 

Section
Column
width100%

Upgrade's Logbook (Jul)

Logbook (01/Jul/14). Missing artifacts.

I've noticed that after the release process some needed artifacts were missing, so you have to compile sakai's source code before trying to compile some contrib tool.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-26598

Logbook (04/Jul/14). Gradebook2 singletons.

We use GB2, the first step is upgrade from 1.8.1 to 1.9.1, and then to get this version up and running in Sakai 10 define some singletons.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keyGRBK-1418

I also put here information that was share in mail lists and could be important later during the migration process.

Roster to Roster2

Code Block
languagesql
titleUpdate Old Sites Roster Tool
UPDATE SAKAI_SITE_TOOL SET REGISTRATION = 'sakai.site.roster2' WHERE REGISTRATION = 'sakai.site.roster';

Sakai 10 Know Issues

Logbook (08/Jul/14). Remaining Jiras.

The migration process it's going pretty well, we realized that a lot of our patches are now in Sakai 10 so no more source code modifications are needed. But we located 3 patches that aren't fixed in Sakai, so far.
Jira
serverSakai
columnskey,summary,type,created,status,resolution
maximumIssues20
jqlQueryissuekey in (SAK-22763,SAK-25928,SAK-23795)
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c

Another great thing is that you can now configure CAS with a file inside sakai.home. Cool !!

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-23187

This one is from our last migration process and still unresolved, we need to push on it.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-23795

Logbook (10/Jul/14). Looking at the database conversion scripts.

Checking at the conversion scripts we detect some issues, and we also discover new permissions added in Sakai 10 (calendar.options,signup.attend.all and signup.view.all). We use signup with Sakai 2.9, so all database objects for signup are already created. I found some sintax problems in oracle script:

ALTER TABLE chat2_channel ADD COLUMN START_DATE TIMESTAMP NULL DEFAULT NULL;

In Oracle should be:

ALTER TABLE chat2_channel ADD START_DATE TIMESTAMP DEFAULT NULL;

Looking at some new options that come in Sakai 10. We came accross a possible regression in the actual trunk (Sakai 11). It is actually working fine in 10 but not in trunk. We thought that it was worth to report it so we created a JIRA. It is related to updating the user preferences.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-26629

This is one related with the new roster tool, nobody noticed that the overview is not sorting properly.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-26630

Logbook (11/Jul/14). Starting sakai.

We noticed that starting sakai 10 with our database (after conversion process), one ddl error appears about lessons. But how it's possible we set auto.ddl to false.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keyLSNBLDR-398

I18n issue, why we have to mantain spanish translations in two files?

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-26632

Logbook (17/Jul/14). Roster2 issues.

The new roster tool in Sakai 10 is great, but has some issues that makes it unusable. For example, if your membership is too large you have to hit tool twice to see the list. Another issue is that you need to have roster.viewallmembers or roster.viewgroup permission to see memberships (and by default student has none of them). Also in trunk the first issue is not reproducible, but there are a couple of i18n bugs that makes this version unusable. Another demanded feature is pagination to support sites with very large number of members.

Jira
serverSakai
columnskey,summary,created,priority,status,resolution
maximumIssues20
jqlQuerykey in (SAK-26265,RSTR-67,SAK-26260,SAK-26290,SAK-27501,SAK-27578,SAK-17493,SAK-27853)
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c

Days later this email thread at dev list bring to life the complete list of roster2 tool issues so far.

http://sakai-project-mail-list-archives.1343168.n2.nabble.com/Building-Sakai-Roster-Tool-Issue-tt7598260.html#none

Logbook (21/Jul/14). Bad performance with refreshAuthzGroup.

Since the begining of Sakai we experimented this issue:

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keyKNL-1183

But we solved using a nightly job that call refreshAuthzGroup to populate sites with correct members (enrollment doesn't change too frequently). Now in Sakai 10 the issue is solved but calling refreshAuthGroup method that, because of our implementation, has a significant cost, specially if we call it in sites with a lot of members. The dropdown menu now call this method, and this could cause that the menu is not displayed as fast as they could be.

Logbook (22/Jul/14). Starting to QA more seriously

We created our own google doc in order to register every issue. Doing a quick check around the UI, we found that the links to messages and forums in MyWorkspace are not working. It's also happening in Trunk but only for the Spanish (and maybe French) language. We created:

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-27562

We have also created the Label 'UMUtoSAK10' to be able to keep a better track of every issue that we find. JIRA Labels offer an easy way to keep your jiras well organized.

Logbook (25/Jul/14). Resources and "copy from other sites"

After the JIRA below you need permission content.revise.any to be able to copy resources from other sites.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-23367

Another minor issue the options button in notification center is still visible after click on it.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-27588

Logbook (29/Jul/14). Resources in Administration Workspace doesn't work if you have a lot of deleted resources.

After a little research I noticed that resources in Administration Workspace causes a serious performance problem, could it be a problem for other sites with a huge list of deleted resources?

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-27732

Maybe this quartz job is the solution.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-24426

Logbook (10/Sep/14). Oracle column now is managed as clob.

In Sakai 10 the BCC column in private messages is managed as clob but it was created as varchar2. This causes a error and you have to change column from varchar2 to clob with a script like this:

alter table mfr_message_t add (RECIPIENTS_AS_TEXT_BCC_B CLOB);
update mfr_message_t set RECIPIENTS_AS_TEXT_BCC_B=RECIPIENTS_AS_TEXT_BCC, RECIPIENTS_AS_TEXT_BCC=null;
alter table mfr_message_t drop column RECIPIENTS_AS_TEXT_BCC;
alter table mfr_message_t rename column RECIPIENTS_AS_TEXT_BCC_B to RECIPIENTS_AS_TEXT_BCC;

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-27890

Logbook (17/Sep/14). Samigo sequence renamed.

In Sakai 10 the sequence SAM_PUBLISHEDATTACHMENT_ID_S was renamed to SAM_PUBATTACHMENT_ID_S, but nothing was added in migration scripts, so I guess it wasn't on purpose.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAM-2408

Logbook (24/Sep/14). Resources different problems.

Digging in resources features I've seen different issues. In general the new trash feature has several problems due to its desing, it uses the same id to store resources in trash and that causes different problems.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-27925

In the other hand other problems were detected related with resources tool.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues20
jqlQuerykey in (SAK-27746,SAK-26000)
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c

Logbook (26/Sep/14). Number format error.

This one was closed as "cannot reproduce", but it is failling in 10. I've reopened the JIRA that was created during our migration from 2.7 to 2.9. I have to make a new patch because the existing one duplicates code.

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c
keySAK-23784

Logbook (23/Oct/14). Forums rank feature.

It seems like finally we are going to finish our migration process this weekend, but this wasn't a bed of roses. We found a lot of problems some of them really evident and it is hard to understand how some features were added to Sakai without better QA, for example the new rank feature in forums that is still having problems and there is no way to disable:

Jira
serverSakai
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues20
jqlQueryfilter=14670
serverId66e9c5b5-d22c-30a2-a9d4-703f4354570c

Almost each new feature included in Sakai 10 has bugs. Bugs are the natural way to success, but I think there were too much in this version.