How to Build and Run a Terracotta Enabled Sakai

Owned by Former user (Deleted)
Last updated: Dec 19, 2008 by Former user (Deleted)Version comment
4 min read

Introduction

This page will help someone build and run a Terracotta enabled Sakai instance.

Building A Terracotta Enabled Sakai

Source Code to Build From

When this testing was first conducted (early September 2008) the changes to enable Terracotta clustering in Sakai, were not checked into any mainline Sakai branch or trunk. Instead they exist only in institutional branches. The following institutional distributions may be used to experiment with Terracotta clustering in Sakai.

Session Clustering

This distribution represents the original development environment for the SAK-13324 work. This contains all the framework changes to enable allowing tools to be cluster enabled. It supports (as of October 2008) the Sakai Resources Tool and login failover.

Wiley Session Clustering

This distribution represents the same code as the previous Session Clustering distribution with the addition of a branch of the site-manage module.  This distribution does not contain a the cluster enabled Resources tool. This branch, introduces customizations to the Worksite Setup tool. This specific version of Worksite Setup was customized to enable clustering for the tool.

Build and Deploy the Terracotta Configuration

The terracotta-config module does not get deployed automatically. You must manually run a maven build with a property set in order to have the Terracotta configuration deployed. You can do this by running the following maven command from within the terracotta-config directory.

$ mvn -Dterracotta.enable=true clean install

This command will create a tcotta directory in your sakai.home directory (sakai.home is typically set to the sakai directory name under your tomcat distribution). In the tcotta directory you will find a tc-config.xml file at the top level. This is the file a Terracotta server should be pointed at. Additionally you will find several TIM (Terracotta Integration Module) jars in a sub-directory structure. It is important that this directory structure be retained if the contents of the tcotta directory are moved. Additionally, if you move the contents of this directory, you will need to edit the tc-config.xml file and update the directory information found in that file.

Configuration

The following sections describe how to configure an environment to work with Sakai and Terracotta Clustering (as developed under SAK-13324).

Sakai Configuration

You may configure the following property in your sakai.properties file.  This property controls which tools will attempt to be clustered.  Adding a tool to this list that is not cluster enabled will cause a problem.  Removing a cluster enabled tool from this list will simply have the effect of that tool not exhibiting failover behavior.

clusterableTools@org.sakaiproject.tool.api.SessionManager=sakai.login,sakai.membership,sakai.resources,
sakai.dropbox,sakai.filepicker,sakai.resource.type.helper

The above should all be on one line, but has been wrapped for legibility.

Tomcat Configuration

JAVA_OPTS


There are several options that must be passed to Tomcat to enable Terracotta clustering.

  • -Dsakai.cluster.terracotta=true
    Indicates that Terracotta clustering should be enabled. Initially this will cause the Sakai Component ClassLoaders to register themselves with Terracotta. After Sakai is running, it will cause the SessionManager to place any NON-clustered tool data in a special non-clustered data structure, while all cluster-enabled tools will be stored in the normal data structure, that is then clustered by Terracotta.
  • -Dtc.install-root=/path/to/terracotta/home
    This option must be set to the home directory of the Terracotta distribution. It will find different libraries and modules it needs to run in this directory.
  • -Dtc.config=hostname:port
    This option must be set to the hostname (or IP address) of the machine running the Terracotta server. You must also specify the port where Terracotta is running (the default port for Terracotta is port 9510).
  • -Xbootclasspath/p:$TC_HOME/lib/dso-boot/dso-boot-hotspot_linux_150_14.jar
    This option must be set to the custom boot jar in the Terracotta distribution. This boot jar can be created by running the following Terracotta command in the same environment where you start Tomcat (e.g., same JAVA_HOME environment variable). 
    $TC_HOME/bin/make-boot-jar.sh -f /path/to/tc-config.xml
    Once the boot jar has been created, then the previous -Xbootclasspath/p: JAVA_OPT should be set to that file.

Terracotta Configuration

Most of the Terracotta configuration is done with the tc-config.xml file and TIMs provided in the Sakai. These are built and rolled out to the tcotta directory in sakai.home as shown above.

Install Terracotta

The first version of Terracotta that was used successfully was version 2.6.4. Other versions may work, but 2.6.4 is known to work with the Sakai code tested at the beginning of September 2008. Terracotta 2.6.4 may be obtained from the www.terracotta.org website under downloads. Once the terracotta-generic-2.6.4.tar.gz package is obtained, simply unarchive it where you would like it to reside on your system. You must create the boot jar file as described in the previous section (see the make-boot-jar.sh command).

Install collection-tims

An additional Terracotta library is needed to successfully run the Sakai Terracotta code. This library is call the Terracotta Integration Module for auto-locked Collections or tim-collections for short. Tim-collections can be downloaded from the Terracotta Forge web site. Version 2.2.2 was known to work with the Sakai code tested at the beginning of September 2008. Once downloaded all of the jar files contained in the download should be copied to the modules subdirectory in the Terracotta installation directory.

$ ls $TC_HOME/modules
org                                   tim-synchronizedmap-2.2.2.jar
readme.html                           tim-synchronizedset-2.2.2.jar
tim-collections-2.2.2.jar             tim-synchronizedsortedmap-2.2.2.jar
tim-hashtable-2.2.2.jar               tim-synchronizedsortedset-2.2.2.jar
tim-synchronizedcollection-2.2.2.jar  tim-vector-2.2.2.jar

Starting Terracotta

To start terracotta, simply run the start-tc-server.sh command in the bin directory of the Terracotta installation. You must pass it a -f /path/to/tc-config.xml command line argument pointing to the Terracotta configuration file previously built and deployed (see the Build and Deploy the Terracotta Configuration section above). You may optionally set a TC_HOME environment variable to the root of the Terracotta installation directory before starting Terracotta.

Order of Application Startup

Although many variations potentially exist with regards to the order the applications may be started, the testing described on this page was done with the following order. It is vital, however, that with order is used, the Terracotta server be started before any Sakai Tomcat instance.

  1. Start Terracotta (Terracotta start-tc-server.sh command)
  2. Start the Sakai Tomcat instance that will participate in the Cluster (Tomcat startup.sh command)