Open tasks to improve Sakai testing

WORK IN PROGRESS

Following on from the Paris BOF, here's a starting list of tasks which any developers in the community can take on. Please update this page as you see fit.

Add more sample tests

Revise existing tests to reflect "best practices"

Provide easier starts

It's a lot faster to fill in a form than to write an essay from scratch when faced with a blank sheet of paper. Similarly, time-crunched developers will be much more able to write tests if the basic setup is there already. For example, Aaron Zeckoski's App Builder encourages test development by generating placeholders for unit and project-bounded tests.

Possibilities include creating Maven archetypes and Eclipse templates for the various test styles, providing commented-out best-practices sample projects, adding EasyMock-based stubs to the App Builder templates, and adding optional generation of integration test areas to the App Builder.

Agree on Maven trigger to enable longer-running tests

Full front-to-back functional integration tests, Sakai service-level integration tests, and load tests are all too heavyweight to be run automatically on every build. Currently, there's no agreed-upon method for enabling all such tests. They have to be run manually for each project, and the method for running them varies. By agreeing on a Maven property or profile to be shared across projects, we can make sure that all tests pass as part of continuous integration and the release process.

As a first attempt, I'm using a "skipLongTests" property to enable or disable a few integration tests in Sakai 2.6.

Centralize support for user and request emulation

Much service-integrated tests and utilities need Sakai components to think they're being run by a particular user, and many also need an easy way to pretend to be in an HTTP request associated with a particular site and tool. Currently, we have many not-quite-copy-and-pasted variants of this functionality. It should be centralized in the test utilities package.

Write data loader for performance tests using ComponentContainerEmulator and compare its performance to the web services method

Script sample test-harness-powered integration tests using Groovy or Jython

Can Sakai service integration be used for all Spring tests without copy-and-paste?

To work with the emulated component manager, SakaiDependencyInjectionTests overrides two methods in AbstractDependencyInjectionSpringContextTests. Other Spring test-support classes could be used the same way by overriding the same two methods. Could AOP be used to overlay that logic rather than having to subclass each Spring class?

Continued improvements to test-harness

Jon Gorrono is coordinating this work at SAK-12334. Requirements were gathered at a Paris BOF.