...
Here's a list of possible projects for the QA WG to consider. Please add to and critique!
- A "stack" of open source tools for load testing and (if people can provide the hardware) nightly integration testing – that every Sakai production and development team can run.
- Continued work with Selenium IDE – in fact, after the talk about automation, I found what could be a very useful tool (I am very excited about this!): http://joker.linuxstuff.pl/documentation/make_selenium. This tool provides a way to (1) make python scripts out of recorded Selenium IDE macros (2) the reverse – make Selenium scripts from python scripts that regular users can run on their own.
- A load-testing repository and "cookbook" for open source tools (JMeter and Grinder) so that every team that wants to do load testing has examples and documentation they can use.
- Regular integration builds - make regression testing (1) automated (2) happens no less frequently than every 48 hours.
- Regression testing work flow – from a tester recording a selnium IDE macro --> a python script --> ant build --> incorporation of the test into the nightly integration build of trunk and supported maintenance branches
- From JIRA --> automated integration testing – getting to the point where no bug in JIRA is closed without an automated test that checks for that bug, a better way would be to have a test working when the issue is resolved and the best way would be to have a test even before somone works on the resolution. The goal would be to get to 50-80% coverage of all regression tests to be fully automated in an integration build. Not all issues can be tested in an automated way, AFAICT. If the community enforces this, we should have an available trusted team to distinguish the issues those testing cannot be automated.
- Further workshops on using Selenium and JMeter – for QA testers and developers.
- Process – ultimate getting to the point that no tool makes it into the core (stable) set of tests without not just a test plan, but a set of automated test scripts accompanying the test plan.