QA Test Automation

QA Test Automation

Introduction

This page is intended to be a resource for sharing ideas and tools for automating some (but not all!) of the QA process. With each subsquent release, there is more and more code to test. Automating some of the basic functionality of the tests will allow us to review the status of the code at any point in time.

What's your Interest?

Let's get a working group(s) going around these idea. Please sign up or contact me if you are interested. We need tools developers involved in the process. If you are a tool developer or represent one or more tool developers, please add the tool name with "(me)", "(<number>)" or "(me+<number>)" (eg "my neat tool (me+2)").

Background

Sakai Vancouver 2006 was a great start in getting ideas out there for how to use automation to improve QA efforts. Here are some ideas that came up. Please feel free to share ideas or critique. I understand that in a community not all of these can or should make it to fruition - but we might have a fun time trying. I look forward to sharing ideas, time, and learning and discovering better ways to make QA testing more fun, easier, take less time, and more effective for everyone - from the volunteer QA, to the production teams, to the developers. (Posted by Ari Consul)

List of possible projects

Here's a list of possible projects for the QA WG to consider. Please add to and critique!

• A "stack" of open source tools for load testing and (if people can provide the hardware) nightly integration testing – that every Sakai production and development team can run.
• Continued work with Selenium IDE – in fact, after the talk about automation, I found what could be a very useful tool (I am very excited about this!): http://joker.linuxstuff.pl/documentation/make_selenium. This tool provides a way to (1) make python scripts out of recorded Selenium IDE macros (2) the reverse – make Selenium scripts from python scripts that regular users can run on their own.
• A load-testing repository and "cookbook" for open source tools (JMeter and Grinder) so that every team that wants to do load testing has examples and documentation they can use.
• Regular integration builds - make regression testing (1) automated (2) happens no less frequently than every 48 hours.
• Regression testing work flow – from a tester recording a selnium IDE macro --> a python script --> ant build --> incorporation of the test into the nightly integration build of trunk and supported maintenance branches
• From JIRA --> automated integration testing – getting to the point where no bug in JIRA is closed without an automated test that checks for that bug, a better way would be to have a test working when the issue is resolved and the best way would be to have a test even before somone works on the resolution. The goal would be to get to 50-80% coverage of all regression tests to be fully automated in an integration build. Not all issues can be tested in an automated way, AFAICT. If the community enforces this, we should have an available trusted team to distinguish the issues those testing cannot be automated.
• Further workshops on using Selenium and JMeter – for QA testers and developers.
• Process – ultimate getting to the point that no tool makes it into the core (stable) set of tests without not just a test plan, but a set of automated test scripts accompanying the test plan.

Tutorials and infomation