Page Comparison

...

The TestRunner is meant to support the writing of complex integration tests and performance checking tests. It is designed to support the goals from the Sakai Kernel RoadmapTechnical Goals (towards a roadmap) proposal. In particular, it is made to address these points without putting a heavy burden on the programmers:

...

Removes the need for Mock objects
- Difficult - Mocking up Sakai services is problematic and difficult to do correctly and is a barrier to writing tests
- Integration instead of Unit - Most tests of Sakai code cannot be written as actual unit tests and are actually integration tests because they have so many dependencies on other Sakai code
- Saves time - Creating all the mock objects needed by code which has many Sakai dependencies can take hours (or days), time which is better spent on writing actual code
- Invalid Mocks - The mock objects cannot completely emulate the real thing and code which may work when tested against a mock will actually fail when run in a real system
Valid testing results
- Tests are executed inside the Sakai environment and are therefore much more reliable than a test which was run in complete isolation
Load testing
- Load tests can be easily written and setup to run in a way that yields accurate results (inside the running system)
- Since a test is basically any code you want to run you can use them to artificially load your system and see where bottlenecks are
Data validation
- The test runner can be used to run tests which will check for data validity (and even correct it if desired)
- Allows a system to validate that the data it has is ok after or before a migration/upgrade or large scale export
Easy to use and highly configurable
- All test runner settings are configurable via spring (and also sakai.properties)
- Tests can be packaged as components or as a webapp and then dropped in (easily portable without rebuilding)
- Supports test types (Integration, Load, Data Validation) and types can be individually enabled or disabled
- Tests use standard syntax and are just an extension of the standard junit TestCase
- Test classes support annotation based injection via autowiring or explicit bean id injection (using the syntax that will be supported in Spring 2.5 when it is released)
- Can register your tests programmatically or via Spring beans using the TestExecutor
- Can disable all tests and control when tests are run or use the tool to run tests
Flexible and trustworthy code
- The code has extensive unit testing to test the TestRunner codebase
- Sample tests (small ones) are included with the TestRunner which are executed to verify it is working
- Maven 1 and Maven 2 are supported and it should run in Sakai 2.2+

...

Use svn to checkout the testrunner project from SVN (location at top of this page) and place it in your sakai source directory and then build it using maven 1 or maven 2

Create a test by extending either JUnit TestCase or TestRunner SpringTestCase

NOTE: You must use SpringTestCase if you want the ability to access Sakai services

Here is a simple test that extends the TestRunner SpringTestCase and gets the Sakai UserDirectoryService (using Autoiwiring automatic bean injection)

Code Block

	java
	java

public class SampleTestSakaiUser extends SpringTestCase {
   private UserDirectoryService userDirectoryService;
   @Autowired
   public void setUserDirectoryService(UserDirectoryService userDirectoryService) {
      this.userDirectoryService = userDirectoryService;
   }

   public void testCanGetSakaiUDSBean() {
      assertNotNull(userDirectoryService);
   }

   public void testCanUseUDS() {
      assertTrue(userDirectoryService.countUsers() > 1);
      assertTrue(userDirectoryService.getUsers(1, 1).size() == 1);
      User user = null;
      try {
         user = userDirectoryService.getUser(UserDirectoryService.ADMIN_ID);
      } catch (UserNotDefinedException e) {
         fail("Exception: " + e.getMessage());
      }
      assertNotNull(user);
      assertEquals(UserDirectoryService.ADMIN_ID, user.getId());
   }
}

Add the needed maven dependencies to a project which will run the tests,

Name it something like sakai-YOURAPP-tests if you are making a project specially for
running tests, otherwise you can just put it in the project with your implementation code

Maven 2 dependencies for your impl tests project (where you put your tests code)

Code Block

	xml
	xml

<dependency>
    <groupId>org.sakaiproject</groupId>
    <artifactId>sakai-testrunner-logic-api</artifactId>
    <version>${sakai.version}</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>3.8.1</version>
    <scope>provided</scope>
</dependency>

Maven 2 dependencies for your pack (where you put your spring configif building a component) or tool (if building a special test running webapp)

Code Block

	xml
	xml

<dependency>
    <groupId>org.sakaiproject</groupId>
    <artifactId>sakai-YOURTOOL-tests</artifactId>
    <version>${sakai.version}</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.sakaiproject</groupId>
    <artifactId>sakai-testexecutor</artifactId>
    <version>1.0</version>
    <scope>runtime</scope>
</dependency>

Register the test with the TestRunner

Register via Spring (using a TestExecutor bean definition in your components.xml file)

Code Block

	xml
	xml

<bean class="org.sakaiproject.testrunner.util.TestExecutor">
   <property name="testClassname"
      value="org.sakaiproject.testrunner.impl.tests.SampleTestSakaiUser" />
   <property name="testType" value="testrunner.integration" />
   <property name="registerTest" value="true" />
</bean>

Register programmatically via a service

Code Block

	java
	java

public class SampleTestRunner {
   private final static Log log = LogFactory.getLog(SampleTestRunner.class);

   private TestRunnerService testRunnerService;
   public void setTestRunnerService(TestRunnerService testRunnerService) {
      this.testRunnerService = testRunnerService;
   }

   private String myTestsId = "aaronz";

   public void init() {
      // register some tests
      testRunnerService.registerTest(myTestsId, TestRunnerService.TESTING_TESTS_INTEGRATION, SimpleTest.class);
      testRunnerService.registerTest(myTestsId, TestRunnerService.TESTING_TESTS_INTEGRATION, SampleTestSakaiUser.class);

      // call a method to run the tests
      if (runMyTests()) {
         log.info("My Sample tests running within a service passed!!");
      }

      // unregister my tests since I don't need them anymore
      testRunnerService.unregisterTests(myTestsId, null);
   }

   /**
    * This runs my tests and returns a boolean which indicates if they passed or failed,
    * you would probably want to do something useful with the output
    * @return true if tests passed, false otherwise
    */
   private boolean runMyTests() {
      Map<Class<? extends TestCase>, TestResult> m = testRunnerService.runTests(myTestsId, null);
      return TestRunnerUtils.checkTestsSuccess(m);
   }
}

This would also require a simple spring bean definition to initialize the SampleTestRunner

Code Block

	xml
	xml

<bean class="org.sakaiproject.testrunner.impl.SampleTestRunner" init-method="init">
   <property name="testRunnerService" ref="org.sakaiproject.testrunner.TestRunnerService" />
</bean>

Startup Sakai and the TestExecutor will run the test automatically (unless the TestRunnerService is configured to disable tests)

Code examples

There are some examples that use this in the following projects:

Versions Compared

Old Version 4

New Version Current

Key

Code examples