Development Environment Setup Walkthrough

Information

This is a walkthrough for setting up a Sakai development environment on a laptop or desktop. These instructions will also work for setting up a development server though I suggest slightly beefier settings for memory. This is currently tailored for Sakai 2.9+. Once you have a Sakai development environment setup you should be able to program your own Sakai tools or modify existing tools.

Notes for developers using older versions of Sakai (2.8.x-)

Developers who are using Sakai 2.8.x through 2.5.x should use the Development Environment Setup Walkthrough (2.5.x-2.8.x)
Developers who are using Sakai 2.4.x or lower should use the Development Environment Setup Walkthrough (2.2.x-2.4.x)

Notes for windows users

change the "/" to "\" for all directory paths except those specified in the maven build.properties file
add the drive letter (C:) to your paths
use %VARIABLE% instead of $VARIABLE with environment variables
set your environment variables in Control Panel -> System -> Advanced -> Environment Variables

All of sakai and related programs should be installed in an opt directory which you will need full write access to, put this directory anywhere you want, this document will assume you have placed it in your root directory

Install/Verify you have Java 1.7
1. To check: Run java -version on the command line
2. If not correct, download the J2SE SDK (make sure you get the JDK and not JRE) from:
  - http://www.oracle.com/technetwork/java/javase/downloads/index.html
  - Mac OS X users (10.6+) will have Java 1.7 by default
3. Install it (the SDK) to /opt/java
  - Note: Install the JRE to a different directory (probably the default, especially under windows) or you will have problems
4. Set environment variable: JAVA_HOME=/opt/java
  - Mac users: JAVA_HOME=/Library/Java/Home
  - Windows users default: JAVA_HOME=C:\j2sdkXXX (where "XXX" is the version - for example "j2sdk1.7.0_11")
5. Add $JAVA_HOME/bin to PATH
6. Set JAVA_OPTS for tomcat in the tomcat/bin/setenv file (see Install Tomcat 7 / 8)
```
export JAVA_OPTS="-server -Xmx1028m -XX:MaxPermSize=320m -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Dsun.lang.ClassLoader.allowArraySyntax=true"
```
  - These settings are for developers
  - More info on JVM server settings: Sakai Admin Guide - JVM Tuning
Install/Verify you have MySQL 5.1 or later
- MySQL - http://www.mysql.com
  - You should download the MySQL Community Server
  - NOTE: MySQL 5.5.x works but there are minor issues (mostly related to creating some profile 2 tables). If you already have 5.5 then you probably do not need to downgrade
1. To check: Run mysql --help on the command line
2. If not, download MySQL from:
  - http://dev.mysql.com/downloads/mysql/5.1.html
    - Or if you want 5.5 or the current release: http://dev.mysql.com/downloads/mysql/
3. Custom install to /opt/mysql (options vary slightly based on operating system)
  - Linux users should install MySQL using a package or binaries if possible
  1. Choose standard configuration
  2. Install as a windows service (Windows only)
  3. Launch automatically (recommended)
    - If you choose not to do this, make sure that you startup MySQL each time before you try to run Sakai
  4. Include the bin directory in Path
  5. Don't use an anonymous account
  6. Set your root password to "mysqlpwd1"
4. Update the mysql config file
  - For linux/OSX this is going to be: /etc/my.cnf
  - Windows: this file will be called my.ini and located in the dir you installed mysql into
  - MySQL provides a number of preconfigured option files that can be used as a starting point for configuration. Look for my-small.cnf, my-medium.cnf, my-large.cnf and my-huge.cnf.
  1. Add the following to the [mysqld] section
    default-storage-engine = InnoDB innodb_file_per_table character-set-server=utf8 collation-server=utf8_general_ci lower_case_table_names = 1
  2. OPTIONAL for 5.5: Add this to enable logging
    log_output=FILE log=/tmp/mysql-query.log slow-query-log=1 long_query_time=1 slow_query_log_file=/tmp/mysql-slow-query.log expire_logs_days=5
Setup Sakai DB (schema) and user
1. Create the sakai database and sakai user (password=ironchef)
2. Run the following from a command line prompt:
```
mysql -u root -p
```
  - Note: You can also do the following if you prefer to be prompted for the password: mysql -uroot -p
3. Then run these commands from the mysql prompt (one command per line):
```
create database sakai default character set utf8;
grant all privileges on sakai.* to 'sakai'@'localhost' identified by 'ironchef';
grant all privileges on sakai.* to 'sakai'@'127.0.0.1' identified by 'ironchef';
flush privileges;
quit
```
4. [Optional] Download and install mysql query browser
  1. http://dev.mysql.com/downloads/query-browser/1.1.html
  2. You do not have to do anything with query browser now
Download and setup Maven 2.2.x+ stable package
- Maven - http://maven.apache.org/
1. Download Maven 2.2.X (minimum of 2.2.1) - http://maven.apache.org/download.html
  - NOTE: some support exists for Maven 3 but there are a few things that do not work as of 6 Sep 2011
2. Extract to /opt (should create apache-maven-2.X.X folder)
3. Set environment variable: MAVEN2_HOME=/opt/apache-maven-2.X.X
4. Add $MAVEN2_HOME/bin to PATH\
5. Set the MAVEN_OPTS environment variable (in linux/OSX this is typically done in ~/.bash_profile)
  export MAVEN_OPTS='-Xms128m -Xmx796m -XX:PermSize=64m -XX:MaxPermSize=172m'
  - Adding -XX:+AggressiveOpts may speed up your builds depending on your OS, JVM, and system hardware
Install/Verify you have Subversion
- Subversion - http://subversion.tigris.org/
1. To check: Run svn --version on the command line
  - http://subversion.apache.org/packages.html
  - Get the subversion binaries and not the source, if possible
  - If there are no binaries for your platform, get the source and use the configuration options --with-ssl and --with-libs
2. Extract to /opt (should create subversion-1.2.3)
  - Windows users will want to rename the extracted directory
  - Unix users will probably want to use a package for their flavor
3. Set environment variable: SUBVERSION_HOME=/opt/subversion-1.2.3
4. Add $SUBVERSION_HOME/bin to PATH
Download and setup tomcat 7.0.21+ (stable only)
Download and setup MySQL Connector/J stable
- Mysql Connector J - http://www.mysql.com/downloads/connector/j/
1. Download connector for your version of MySQL
  1. If running mySQL 5.0-5.1
    Download version 5.0.8+ from http://dev.mysql.com/downloads/connector/j/5.0.html
  2. If running MySQL 5.1+
    Download version 5.1+ from http://www.mysql.com/downloads/connector/j/
  - NOTE: the version numbers do not directly align despite the way they appear to here
2. Extract the file to /opt
3. Copy mysql-connector-java-<version>-bin.jar to $CATALINA_HOME/common/lib
4. Delete the extracted folder
Use Subversion to download sakai code (Sakai 10 and earlier) or use Git/Github to download sakai code (Sakai 11 and later)
1. Change to the /opt directory
2. Checkout source from trunk (absolute latest stuff) or a release branch or tag:
  1. Trunk: svn checkout https://source.sakaiproject.org/svn/sakai/trunk/ sakai-trunk
  2. 2.9.x Branch: svn checkout https://source.sakaiproject.org/svn/sakai/branches/sakai-2.9.x/
    Alternative Sakai 2.9.x repository
    
    Should you run into errors related to missing dependencies when building source from the above repository with Maven (Step 11 below), attempt to checkout from https://source.sakaiproject.org/svn/sakai/branches/sakai-2.9.x-all instead
  3. If you want a version other than these, browse the tags and branches at https://source.sakaiproject.org/svn/sakai/
3. Checkout will take about 5-10 minutes
4. The directory created by subversion will be referred to as <sakai-src> directory (this could be whatever you want; above examples use 'sakai-trunk' and 'sakai-2.9.x')

Setup sakai.properties file

Create sakaidirectory in $CATALINA_HOME
- $CATALINA_HOME should be /opt/tomcat if you have been following these instructions
Copy the default config from default.sakai.properties to $CATALINA_HOME/sakai/sakai.properties
- The default configuration template is at https://source.sakaiproject.org/svn/config/trunk/configuration/bundles/src/bundle/org/sakaiproject/config/bundle/default.sakai.properties.

Edit the sakai.properties file for a MySQL dev environment. Starting at the section marked # DATABASE:

Set BaseDataSource username
1. Set username@javax.sql.BaseDataSource=sakai
Set BaseDataSource password
1. Set password@javax.sql.BaseDataSource=ironchef

Find the section: # HSQLDB settings and verify following lines are commented out

# HSQLDB settings (DEFAULT)                                                                                                                               
#vendor@org.sakaiproject.db.api.SqlService=hsqldb
#driverClassName@javax.sql.BaseDataSource=org.hsqldb.jdbcDriver
#hibernate.dialect=org.hibernate.dialect.HSQLDialect
#validationQuery@javax.sql.BaseDataSource=select 1 from INFORMATION_SCHEMA.SYSTEM_USERS
# Two hsqldb storage options: first for in-memory (no persistence between runs), second for disk based.
#url@javax.sql.BaseDataSource=jdbc:hsqldb:mem:sakai
#url@javax.sql.BaseDataSource=jdbc:hsqldb:file:${sakai.home}db/sakai.db

Find the section: # MySQL settings and uncomment the 6+ lines in it

# MySQL settings
vendor@org.sakaiproject.db.api.SqlService=mysql
driverClassName@javax.sql.BaseDataSource=com.mysql.jdbc.Driver
hibernate.dialect=org.hibernate.dialect.MySQL5InnoDBDialect
url@javax.sql.BaseDataSource=jdbc:mysql://127.0.0.1:3306/sakai?useUnicode=true&characterEncoding=UTF-8
validationQuery@javax.sql.BaseDataSource=select 1 from DUAL
defaultTransactionIsolationString@javax.sql.BaseDataSource=TRANSACTION_READ_COMMITTED

Save the changes to the sakai.properties file

Create maven settings.xml file

Create a new xml file in your user home directory in the .m2 directory called settings.xml
- Note: This is probably c:\documents and settings\<username> in Windows
- Download a sample settings.xml file (setup for windows users)

Add the following lines:

<settings xmlns="http://maven.apache.org/POM/4.0.0"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                      http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <profiles>
    <profile>
      <id>tomcat5x</id> 
      <activation>
        <activeByDefault>true</activeByDefault> 
      </activation>
      <properties>
        <appserver.id>tomcat5x</appserver.id> 
        <appserver.home>/opt/tomcat</appserver.home> 
        <maven.tomcat.home>/opt/tomcat</maven.tomcat.home> 
        <sakai.appserver.home>/opt/tomcat</sakai.appserver.home> 
        <surefire.reportFormat>plain</surefire.reportFormat>
        <surefire.useFile>false</surefire.useFile>
      </properties>
    </profile>
  </profiles>
</settings>

Note: Unfortunately, Sakai does not use the standard appserver.home so you have to specify a sakai.appserver.home
Note: The sakai.appserver.home must be C:\opt\tomcat for windows users
Note: Do not include trailing / or \ slashes in the directory paths

(optional) You can specify the tomcat home to be an environment variable like so:
```
      <maven.tomcat.home>${env.CATALINA_HOME}</maven.tomcat.home>
```
(optional) Users who use a network proxy need to add the following to their file
```
...
  <proxies>
     <proxy>
        <active>true</active>
        <protocol>http</protocol>
        <host>www.your.proxy.host</host>
        <port>80</port>
        <username>your_username</username>
        <password>your_password</password>
        <nonProxyHosts>localhost</nonProxyHosts>
     </proxy>
  </proxies>
</settings>
```
- If you do not use a username or password for your proxy just leave out those 2 lines
- You only need the nonProxyHosts option if you have a local maven repo that does not require the proxy to be accessed
- Note: Maven 2 doesn't support Microsoft's NTLN authentification scheme. If you connect to a proxy like ISA you will need to use a tool, such as http://ntlmaps.sourceforge.net/ to proxy your traffic

Use Maven to build Sakai
1. Open a command line shell
2. Change directory to /opt/<sakai-src>
3. Execute mvn clean install to build Sakai using maven
  - Note: The build will take an extra 5-10 minutes to download dependencies the first time
4. Execute mvn sakai:deploy to deploy Sakai to your tomcat using maven
5. Partial builds are supported by the maven2 build system
  You can now do a "mvn clean install sakai:deploy" from any subdirectory and build just that code
6. Once you have downloded the jars you can run maven off-line with mvn -o clean install sakai:deploy
Start Tomcat and check to make sure Sakai runs
1. Start tomcat using $CATALINA_HOME/bin/startup
  Viewing an active launch log in Linux
  
  If you are in Linux you can view the activity log as its generated by using "# ./catalina.sh run" from Tomcat's bin directory instead of startup.sh. This is helpful for viewing potential errors as they happen or to know if Tomcat has locked-up during a launch. A launch will have completed when "INFO: Server startup in xxxx ms" is displayed in the terminal.
2. Allow 1 minute+ for tomcat to start up
3. Open http://localhost:8080/ to verify tomcat is running
4. Open http://localhost:8080/portal to verify sakai is running
5. Login to sakai as username:admin, password:admin
6. Shutdown Tomcat using $CATALINA_HOME/bin/shutdown
Install Eclipse 3.2+ stable
- Eclipse - http://www.eclipse.org/
  - Note: If you already have eclipse 3.1.x installed, your best best is to start over with a fresh install rather than attempting to upgrade
  - Note: If you have not installed eclipse already, you should download the WebTools Platform: All-in-one package for Eclipse 3.3+ which includes Eclipse and all of the necessary webtools packages, this is much faster and easier than downloading them seperately, you can skip the WebTools step if you do this, here are instructions to install the Eclipse WebTools
1. Download the newest stable version http://www.eclipse.org/
2. Extract the downloaded file to /opt
  - Windows users should extract to c:\
  - Mac users should use the installer and put it in Applications
3. Run eclipse to verify it works (/opt/eclipse/eclipse)
  - Windows users: c:\eclipse\eclipse.exe to run eclipse
  - Note: If it does not work, there is probably a problem with your java install
4. Set the memory settings for Eclipse
  - The default memory settings for Eclipse are much too low to handle the number of webapps in a full Sakai installation
  1. Shutdown eclipse if it is running
  2. Edit the eclipse.ini file in the directory you extracted/installed eclipse
  3. Change the settings from something like this:
    -vmargs -Xms40m -Xmx256m
    to something like this (leave any params that do not match these alone):
    --launcher.XXMaxPermSize 256M -vmargs -Xms128m -Xmx1024m -XX:+UseParallelGC
    - Windows users should not use notepad to edit this file, use wordpad or edit (command line)
    - Mac users will need to take additional steps to edit the eclipse.ini file
    1. Control-click on the Eclipse Application icon and select Show Package Contents
    2. Double-click on the Contents folder
    3. Double-click on the MacOS folder, the eclipse.ini file should be here
    4. Full path: eclipse/Eclipse.app/Contents/MacOS/eclipse.ini
5. Set the JVM
  - The default JVM will be the one in the JRE, things will work better if you change this to the one in the J2SE
  1. Startup eclipse if it is not running
  2. Select Window -> Preferences -> Java -> Installed JREs
  3. Select the current installed JRE
  4. Click the Edit button
  5. Click Browse and navigate to the home directory of your JAVA installation (e.g. /opt/java)
  6. Click OK to save and then OK to Finish
    - Running Eclipse with a different JVM from the default
      Add -vm /opt/java/bin/java after the eclipse executable
    - You must include the full path to the java executable file
- More Eclipse tips here: Eclipse notes
Add subclipse to Eclipse
- Subclipse - http://subclipse.tigris.org/
1. Startup eclipse
2. Select Help -> Software Updates
3. Select Available Software, click Next
4. Click Add Site
5. Enter http://subclipse.tigris.org/update_1.6.x for the URL
6. Click Finish
7. Check the Subclipse box and click Next
  - Ignore the integrations unless you have a need for them, they can stop the install from working
8. Accept terms, click Next, click Finish, click Install All
9. Restart the workbench when asked
- Add bin and target to global svn ignore in Eclipse (Optional)
  - This will keep you from committing bin and target directory files when you add projects to svn
  1. Click on Window -> Preferences
  2. Select Team -> Ignored Resources
  3. Click on Add Pattern and enter "bin"
  4. Click on Add Pattern and enter "target"
  5. Click on Add Pattern and enter "m2-target"
  6. Click on Apply and then OK
Add Maven Eclipse Plugin to Eclipse
- Maven Plugin for Eclipse - http://m2eclipse.codehaus.org/
1. Startup eclipse
2. Select Help -> Software Updates
3. Select Available Software, click Next
4. Click Add Site
5. Enter http://m2eclipse.sonatype.org/sites/m2e for the URL
6. Click Finish
7. Expand the Maven Integration for Eclipse update site
8. Check the Maven Integration box and click Next
  - Ignore the optional parts unless you have a need for them
9. Accept terms, click Next, click Finish, click Install All
10. Restart the workbench when asked
Add Lombok plugin to Eclipse [OPTIONAL]
- Lombok Library - http://projectlombok.org/download.html
1. Shutdown eclipse
2. Follow the steps on the page linked above
3. Restart eclipse
- NOTE: this is only required for certain parts of the code that use the lombok library (a limited set)
Import Sakai source code into Eclipse
1. [OPTIONAL] Run the following maven commands in your sakai-src root folder (e.g. /opt/cafe-2.5.x)
  This will clean out any eclipse files that are in there already:
```
mvn eclipse:clean
```
2. [OPTIONAL] Generate the eclipse files using maven (this is not required and not recommended)
  This will generate valid eclipse .project and .classpath files from the maven build
```
mvn eclipse:eclipse
```
  - Special Case: If your workspace is in a different location, I've found that this option is needed passed to maven, otherwise the dependencies get incorrect. -Declipse.workspace=..
    - Passing the full path to the local workspace or current directory might also work
  - If you are using the Maven Plugin for Eclipse then you should run this instead:
    mvn eclipse:m2eclipse
3. Startup eclipse if it is not running
4. Create a new workspace for Sakai
  1. Click on File -> Switch Workspace
  2. Enter "WS-Sakai" in place of the default "workspace" directory
  3. Click OK (eclipse will restart)
    - The following steps should be done in the WS-Sakai workspace
5. Add Maven Repository libraries to classpath
  - NOTE: This is for backwards compatibility with projects using the M2_REPO variable so it can be skipped
  1. Select Window -> Preferences -> Java -> Build Path -> Classpath Variables
  2. Add MAVEN_REPO classpath variable with the path to the local maven repository (Sakai 2.4 or lower)
    - The path should be: $USER_HOME/.maven/repository
  3. Add M2_REPO classpath variable with the path to the local maven repository (Sakai 2.5 or higher)
    - The path should be: $USER_HOME/.m2/repository
6. Switch to the Java perspective (Window -> Open Perspective -> Java)
  - Make sure you are in the package explorer
7. Turn off Automatic builds
  - Select Project and uncheck Build automatically
8. Select File -> Import -> Existing Projects into Workspace
  1. You can either import the entire sakai source tree (e.g. /opt/sakai-2.9.x) or selectively choose specific sub-projects you'll be working on (e.g. /opt/sakai-2.9.x/announcement)
  2. Click Finish to import the selected projects
    - This will take awhile, probably 5+ minutes
  3. Clean all Sakai projects
    1. Select Project -> Clean
    2. Select Clean all projects
    3. Check Start a build immediately
    4. Click OK
  4. Build All (Project -> Build All)
    - There will be hundreds of warnings, do not worry about these
9. Turn back on Automatic builds
  - Select Project and check Build automatically
  - Select project and then use the down triangle menu to add Filters (target, m2-target)

Development Environment Setup Walkthrough

Information

Setup Sakai DB (schema) and user

Download and setup Maven 2.2.x+ stable package

Use Subversion to download sakai code (Sakai 10 and earlier) or use Git/Github to download sakai code (Sakai 11 and later)

Setup sakai.properties file

Use Maven to build Sakai

Start Tomcat and check to make sure Sakai runs

Add subclipse to Eclipse

Add Maven Eclipse Plugin to Eclipse

Add Lombok plugin to Eclipse [OPTIONAL]