JMeter
This page is intended for sharing resources and ideas relating to using JMeter for QA Testing
From Charles Hedrick on the sakai-dev list on using JMeter
jmeter is a web load tester. It looks pretty good. I set it up pretty quickly and got it to beat on samigo hard enough to break it consistently. I did it on windows, because the display looked a little odd on os x, through os x and Linux would probably work fine.
download it from http://jakarta.apache.org/site/downloads/
downloads_jmeter.cgi
docs come with the binary. It's a java application, so you have to have java on your system.
uncompress the distribution and run jmeter or jmeter.bat from bin
The docs are in docs. usermanual is probably the most useful. THere's a nice cookbook for starting in jmeter_proxy_step_by_step.pdf. This tells you how to set up the proxy, and use it to capture a session to play back.
I'm not going to repeat the instructions here. It's quite clear.
I used the instructions to capture a session that logs into sakai, takes a one-question test and submits it for grading, and logs out.
That session can be played back in multiple threads quite easily.
Caveats:
- you apparently can't record https:. I used http: for both recording and playback.
- for sakai you'll need to turn on cookie management. the document I pointed you to doesn't say how, although the main user manual does.
You pull down edit, choose add, add config element, http cookie manager.
I believe you want to add this to "test plan", although I added it to each thread group.
- you might want to turn on reporting. again, the main user manual gives info on this. In general you pull down edit, choose add, and choose a listener, e.g. aggregate report. That report is updated in real time.
- when you're capturing a session, one action in sakai will result in several things being captured. That's because each iframe generates a separate web query. You can get rid of some of them if you like, but I ended up leaving things exactly as the program had captured them.
Look at the samigo data to make sure the assessments are actually being graded properly. Before I set up cookie management, nothing was actually happening, but that fact wasn't obvious. You can also turn on detailed data collection in jmeter, and look at the response you're getting from the server.
The easiest thing to do is run multiple threads in parallel. When you select thread group, you can choose the number of threads and how many times to run the script.
It is pretty easy to break samigo with a script that takes a test and submits it for grading. 10 threads in parallel will do it for us.
I'm not sure whether running multiple parallel sessions for the same user is quite equivalent to a real load. Unfortunately doing several users is harder, because you pretty much have to capture a different session for each. The problem is that Sakai generates different URL's for each user. I found I could capture a session for a new user in 15 sec or so, so I just added 10 different users by hand.
Here's what I did:
create a separate "thread group" for each user. From "test plan", edit, add, thread group. Now we begin the trickiness. The web proxy likes to add what it captures to the first thread group. Once I had created 10 of them, it wasn't obvious how to put a different session in each. I just let it all go into the first thread group, then cut and pasted all the steps from the first thread group into the thread group where I wanted it. The confused the proxy slightly, so I had to kill IE and restart it for each user. Fortunately on a fast computer it just takes a second to restart IE. Once I had played around a bit I could do everything needed for a user in about 15 sec (given that I had already created the user in Sakai).
In "test plan", make sure you uncheck "run each thread group separately", since you want them all in parallel.
In the thread groups themselves, look at how many threads each spawns. You may want to do just one thread per user.
Once things are set up, you pull down "run" and do "start". You'll see a number in the upper right showing how many threads exist and how many are active. It ought to start out 10/10 (or however many you set up), and quickly go down to 0/10. It will almost certainly hang at 1/10 or 2/10, at least if you do it a few times.
Resources
SAK-7732