University of Murcia's upgrading experience from 2.9 to 10
Upgrading Experience, from 2.9 to 10
Here we are again !! After sharing our experience upgrading from 2.7 to 2.9 now we are ready to share our experience upgrading to the brand new Sakai 10 at the University of Murcia - SPAIN (Universidad de Murcia http://www.um.es - https://aulavirtual.um.es). In the first experience we shared a lot of info about the local customizations we have, and made them available to others when they are useful enought. We are sure that there are customizations ready to be shared and also new ones that might come.
We also want to share our route map so that others facing the same needs for upgrade can use them, reduce their time and learn from it. This is the power of the community!! At the same time, we would love to hear from you suggestions about tasks that might also be included in our upgrading process, or your advice for facing those tasks and making them easier for us.
This page will be made by the technical staff and it is mainly focused on their concerns around the upgrading process. We also intend to mention all the other staff related to this process and the tasks concerned to them.
Upgrading Plan
After Sakai released its 10 version, we set a clear goal for our TEAM, move our deployed version (2.9.3) to the latest one. However, the first step is to write a simple plan, unfortunately for us and everyone in the Sakai community, Sakai upgrades do not arrives by clicking a 'magic button'. The main task concerns technical aspects, we have developed a lot of custom tools and have also changed some pieces of the original Sakai code. But, there are other tasks involved, so we are going to mention all of them at this master plan.
Evaluating Sakai 10
What do we really know about the new version? This is our first goal. When we started to migrate from 2.7 to 2.9 we didn't know too much about the 2.9 version, but know the situation is pretty different. The migration from 2.7 to 2.9 let us get much more involved in the community and do a lot of work into trunk that now is part of Sakai 10. So we know what's new in Sakai 10 because we developed some of the new features (of course not all). Also we were collaborating in different test fests during the Sakai 10 quality assurance process, and this let us know more about Sakai 10.
Look & Feel
Sakai 10 is not a challenge in terms of look&feel is pretty similar to 2.9, but we need to add some changes to get our users feel that something has change. Probably the main change of the look&feel will be on the gateway page, but not in the internal skin, we have to wait for Sakai 11 and the Morpheus project to see a new responsive design for Sakai.
New Features
There are a lot of new features in Sakai 10, some of them really interesting and also necessary. If you are interested in wht's new on Sakai 10, please look at the release notes.
Major Code Changes
Sakai 10 comes with some important upgrades, such as Hibernate 3.6.10.Final and Spring 3.2.3.RELEASE. Because of these libraries are deployed in shared folder, we also need to upgrade our tools in order to use these new versions. In the other hand another significant change is that the Sakai 10 release process only api artifacts were deployed into public sonatype repository.
Developing Process. (Jul-Sep, 2014)
The development start July 1th, 2014. I don't know how much time it's going to take us release our customized version, but it's time to see if all the work done with the previous migration has a significative impact in this process.
Testing Process. (Jul-Sep, 2014)
During test process we found some issues and we decided to use a label in JIRA to keep track of them. "UMUtoSAK10" is the label and you could check the list of jiras at the right side of this page.
Load testing
I'm going to do some load testing before September and then a more specific tests.
Information Process. (Sep, 2014)
At the begining of the previous migration we wanted to make a promotional process about the steps of migration thinking in marketing perspective, but later on we realized that users don't care about migration, they only care about it works or lose data. So, for this migration we won't do anything but notifying users briefly about new features and when the change will be made, and spend time in quality assurance rather than in marketing things.
The Change. (Sep, 2014)
The best date for us to change from one version to another is right now, before the new term start in September. Actually Jul, 31th is the best date to make the migration, but that date seems to be impossible. Let's see what happend !!
Finally the migration will be Oct, 25th, is far for our initial point first because of the release dates of Sakai 10, but also due to the amount of bugs that we had to solve. We are going to be out with Sakai 10.1 (with a lot of patches added), and we are planning a minor upgrade to 10.3 at the beginning of 2015.
Migration done !!
Final Conclusions
Since we migrated to Sakai 2.9 we got closer to the Sakai community and its SW development cycle, but it is hard to know every change happening and any of them, even a one line change, could affect your migration process. The best part of this process is that now, almost everything that does not work for us, is also a problem for many in the community and sonner or later it will be solved in Sakai, but, the bad thing is that there is an evident lack of QA in Sakai at this moment. We found some bugs that are pretty obvious, and those type of issues made us think that nothing was really well tested after the feature was introduced into trunk.
You can check all jiras opened by us during this process (more than 80), and this will continue during next weeks because with Sakai 10 in production we will discover more problems. As mentioned above, we have used a label in JIRA called 'UMUtoSAK10' to keep track of all our issues and we believe that it could a 'best practice' to share with the community since it could help someone that is upgrading to 10 at the same time or later than us. JIRA allows a user to filter by label and in an easy way you can see all issues related to someone's migration. We paid special attention to all JIRAs and messages comming from anyone from UCT (University of Cape Town) because they had upgraded a couple of months before us and therefore, they were usually describing issues or findings that could affect our new Sakai 10 instance.
One important decision made is that we have had to go into production with the classic Roster tool (Roster1) because Roster 2 still have problems for us after 3 versions of Sakai 10 (10.0, 10.1, 10.2), this is a good summary of the migration process, and a fantastic evidence of the lack of QA. We are still trying to provide patches for Roster 2 so that we can try to move to it as soon as possible.
After a few days running the new version of Sakai there are not so much issues around, we think that the change was pretty successful but there was a none reported issue with Sakai 10.1 that caused us a lot of troubles during our first hours with the new version. We've been working hard to adapt Sakai 10.1 to our needs so when 10.2 was released we had just taken a quick look to the issues added to 10.2 (and most of them were added to our 10.1 version too) but missed the most important one, mainly because it was fixed but not reported anywhere. We are talking about this jira, the issue fixed in that JIRA killed our production server twice in two days.
Did you see something strange? This JIRA is resolved, and it has two fixed versions in the same mayor release: 10.1 and 10.2, how could that happen? The thing is that this JIRA was solved for 10.1 version, but it has a bug that was fixed for 10.2, but this new bug wasn't reported anywhere else. The only clue that you have is the 10.2 in the fix version and of course the subversion tab that shows a commit into 10.x at the beginning of October (after 10.1 was released). For us, this fix was critical, our production server went down so we expected that a fix so critical was clearly mentioned in the release notes but in this case it wasn't even in this filter with "Blocker" issues fixed. A lesson to learn for the whole Sakai Community.
We whould like to specially thanks the folks attending the Sakai Core Team calls for the support they continiously gave us allowing us to contribute many fixes and letting us move forward in this community. Let's keep on improving Sakai!
Check out our previous experience upgrading to Sakai 2.9 !!
Migration DONE !!
Upgrade's Logbook (Jul)
Logbook (01/Jul/14). Missing artifacts.
I've noticed that after the release process some needed artifacts were missing, so you have to compile sakai's source code before trying to compile some contrib tool.
Logbook (04/Jul/14). Gradebook2 singletons.
We use GB2, the first step is upgrade from 1.8.1 to 1.9.1, and then to get this version up and running in Sakai 10 define some singletons.
I also put here information that was share in mail lists and could be important later during the migration process.
UPDATE SAKAI_SITE_TOOL SET REGISTRATION = 'sakai.site.roster2' WHERE REGISTRATION = 'sakai.site.roster';
Logbook (08/Jul/14). Remaining Jiras.
Another great thing is that you can now configure CAS with a file inside sakai.home. Cool !!
This one is from our last migration process and still unresolved, we need to push on it.
Logbook (10/Jul/14). Looking at the database conversion scripts.
Checking at the conversion scripts we detect some issues, and we also discover new permissions added in Sakai 10 (calendar.options,signup.attend.all and signup.view.all). We use signup with Sakai 2.9, so all database objects for signup are already created. I found some sintax problems in oracle script:
ALTER TABLE chat2_channel ADD COLUMN START_DATE TIMESTAMP NULL DEFAULT NULL;
In Oracle should be:
ALTER TABLE chat2_channel ADD START_DATE TIMESTAMP DEFAULT NULL;
Looking at some new options that come in Sakai 10. We came accross a possible regression in the actual trunk (Sakai 11). It is actually working fine in 10 but not in trunk. We thought that it was worth to report it so we created a JIRA. It is related to updating the user preferences.
This is one related with the new roster tool, nobody noticed that the overview is not sorting properly.
Logbook (11/Jul/14). Starting sakai.
We noticed that starting sakai 10 with our database (after conversion process), one ddl error appears about lessons. But how it's possible we set auto.ddl to false.
I18n issue, why we have to mantain spanish translations in two files?
Logbook (17/Jul/14). Roster2 issues.
The new roster tool in Sakai 10 is great, but has some issues that makes it unusable. For example, if your membership is too large you have to hit tool twice to see the list. Another issue is that you need to have roster.viewallmembers or roster.viewgroup permission to see memberships (and by default student has none of them). Also in trunk the first issue is not reproducible, but there are a couple of i18n bugs that makes this version unusable. Another demanded feature is pagination to support sites with very large number of members.
Days later this email thread at dev list bring to life the complete list of roster2 tool issues so far.
Logbook (21/Jul/14). Bad performance with refreshAuthzGroup.
Since the begining of Sakai we experimented this issue:
But we solved using a nightly job that call refreshAuthzGroup to populate sites with correct members (enrollment doesn't change too frequently). Now in Sakai 10 the issue is solved but calling refreshAuthGroup method that, because of our implementation, has a significant cost, specially if we call it in sites with a lot of members. The dropdown menu now call this method, and this could cause that the menu is not displayed as fast as they could be.
Logbook (22/Jul/14). Starting to QA more seriously
We created our own google doc in order to register every issue. Doing a quick check around the UI, we found that the links to messages and forums in MyWorkspace are not working. It's also happening in Trunk but only for the Spanish (and maybe French) language. We created:
We have also created the Label 'UMUtoSAK10' to be able to keep a better track of every issue that we find. JIRA Labels offer an easy way to keep your jiras well organized.
Logbook (25/Jul/14). Resources and "copy from other sites"
After the JIRA below you need permission content.revise.any to be able to copy resources from other sites.
Another minor issue the options button in notification center is still visible after click on it.
Logbook (29/Jul/14). Resources in Administration Workspace doesn't work if you have a lot of deleted resources.
After a little research I noticed that resources in Administration Workspace causes a serious performance problem, could it be a problem for other sites with a huge list of deleted resources?
Maybe this quartz job is the solution.
Logbook (10/Sep/14). Oracle column now is managed as clob.
In Sakai 10 the BCC column in private messages is managed as clob but it was created as varchar2. This causes a error and you have to change column from varchar2 to clob with a script like this:
alter table mfr_message_t add (RECIPIENTS_AS_TEXT_BCC_B CLOB);
update mfr_message_t set RECIPIENTS_AS_TEXT_BCC_B=RECIPIENTS_AS_TEXT_BCC, RECIPIENTS_AS_TEXT_BCC=null;
alter table mfr_message_t drop column RECIPIENTS_AS_TEXT_BCC;
alter table mfr_message_t rename column RECIPIENTS_AS_TEXT_BCC_B to RECIPIENTS_AS_TEXT_BCC;
Logbook (17/Sep/14). Samigo sequence renamed.
In Sakai 10 the sequence SAM_PUBLISHEDATTACHMENT_ID_S was renamed to SAM_PUBATTACHMENT_ID_S, but nothing was added in migration scripts, so I guess it wasn't on purpose.
Logbook (24/Sep/14). Resources different problems.
Digging in resources features I've seen different issues. In general the new trash feature has several problems due to its desing, it uses the same id to store resources in trash and that causes different problems.
In the other hand other problems were detected related with resources tool.
Logbook (26/Sep/14). Number format error.
This one was closed as "cannot reproduce", but it is failling in 10. I've reopened the JIRA that was created during our migration from 2.7 to 2.9. I have to make a new patch because the existing one duplicates code.
Logbook (23/Oct/14). Forums rank feature.
It seems like finally we are going to finish our migration process this weekend, but this wasn't a bed of roses. We found a lot of problems some of them really evident and it is hard to understand how some features were added to Sakai without better QA, for example the new rank feature in forums that is still having problems and there is no way to disable:
Almost each new feature included in Sakai 10 has bugs. Bugs are the natural way to success, but I think there were too much in this version.