E-mailTesting Samigo - Charles Hedrick
In Vancouver I got the impression that debugging Samigo was suffering because people couldn't reproduce the problem. I heard of someone collecting 30 users and having them type by hand. Apparently automated load testers have a reputation of being expensive and hard to use.
Perhaps this is true if you want to do something complex, but I just found that Apache's free JMeter is quite sufficient to show the problems in Samigo, at least in our 2.1.x version. I also found it really easy to set up, once I found the secret PDF that explains how to create a script automatically by recording a web session. To help people get started, I'm attaching notes.
jmeter is a web load tester. It looks pretty good. I set it up pretty quickly and got it to beat on samigo hard enough to break it consistently. I did it on windows, because the display looked a little odd on os x, through os x and Linux would probably work fine.
download it from http://jakarta.apache.org/site/downloads/ downloads_jmeter.cgi docs come with the binary. It's a java application, so you have to have java on your system.
uncompress the distribution and run jmeter or jmeter.bat from bin
The docs are in docs. usermanual is probably the most useful. THere's a nice cookbook for starting in jmeter_proxy_step_by_step.pdf. This tells you how to set up the proxy, and use it to capture a session to play back. I'm not going to repeat the instructions here. It's quite clear.
I used the instructions to capture a session that logs into sakai, takes a one-question test and submits it for grading, and logs out. That session can be played back in multiple threads quite easily.
Caveats:
- you apparently can't record https:. I used http: for both recording and playback.
- for sakai you'll need to turn on cookie management. the document I pointed you to doesn't say how, although the main user manual does. You pull down edit, choose add, add config element, http cookie manager. I believe you want to add this to "test plan", although I added it to each thread group.
- you might want to turn on reporting. again, the main user manual gives info on this. In general you pull down edit, choose add, and choose a listener, e.g. aggregate report. That report is updated in real time.
- when you're capturing a session, one action in sakai will result in several things being captured. That's because each iframe
generates a separate web query. You can get rid of some of them if you like, but I ended up leaving things exactly as the program had captured them.
Look at the samigo data to make sure the assessments are actually being graded properly. Before I set up cookie management, nothing was actually happening, but that fact wasn't obvious. You can also turn on detailed data collection in jmeter, and look at the response you're getting from the server.
The easiest thing to do is run multiple threads in parallel. When you select thread group, you can choose the number of threads and how many times to run the script.
It is pretty easy to break samigo with a script that takes a test and submits it for grading. 10 threads in parallel will do it for us.
I'm not sure whether running multiple parallel sessions for the same user is quite equivalent to a real load. Unfortunately doing several users is harder, because you pretty much have to capture a different session for each. The problem is that Sakai generates different URL's for each user. I found I could capture a session for a new user in 15 sec or so, so I just added 10 different users by hand.
Here's what I did:
1. Create a separate "thread group" for each user. From "test plan", edit, add, thread group. Now we begin the trickiness. The web proxy likes to add what it captures to the first thread group. Once I had created 10 of them, it wasn't obvious how to put a different session in each. I just let it all go into the first thread group, then cut and pasted all the steps from the first thread group into the thread group where I wanted it. The confused the proxy slightly, so I had to kill IE and restart it for each user. Fortunately on a fast computer it just takes a second to restart IE. Once I had played around a bit I could do everything needed for a user in about 15 sec (given that I had already created the user in Sakai).
2. In "test plan", make sure you uncheck "run each thread group separately", since you want them all in parallel.
3. In the thread groups themselves, look at how many threads each spawns. You may want to do just one thread per user.
4. Once things are set up, you pull down "run" and do "start". You'll see a number in the upper right showing how many threads exist and how many are active. It ought to start out 10/10 (or however many you set up), and quickly go down to 0/10. It will almost certainly hang at 1/10 or 2/10, at least if you do it a few times.
JMeter and HTTPS - Casey Dunn, Aug 29, 2006
After recording via the Proxy, you can edit the results and change the transport to HTTPS. This will allow you to generate load with the encryption overhead. However, there is more to it ...
You must also uncomment the following sections of jmeter.properties:
--------------------------------------------------------------------------- # SSL configuration #--------------------------------------------------------------------------- #Classname of the ssl provider to be used (to enable testing of https urls) #And the package name where Stream Handlers can be found #These provided defaults can be uncommented, and they will work if you are using #Sun's JSSE implementation. ssl.provider=com.sun.net.ssl.internal.ssl.Provider ssl.pkgs=com.sun.net.ssl.internal.www.protocol #The location of the truststore (trusted certificates) and keystore ( if other than the defa\ ult. #you can uncomment this and change the path to the correct location. #javax.net.ssl.trustStore=/path/to/cacerts #javax.net.ssl.keyStore=/usr/java/jdk1.5.0_02/jre/lib/security/cacerts #The password to your keystore javax.net.ssl.keyStorePassword=changeit
If you are using the GUI verison of jmeter and it ever asks you for a password while you are testing, it's because you didn't uncomment the keyStorePassword. It's not sufficient to edit the properties file and try again, you must restart the GUI.
Behond that if you have exmulgated a host of Zombie Robots to stress Sakai, who are not freshmen, and you have not set the SSL properties correctly, you'll have a mess of lovely stale stuck Jmeter instances hung up on your box. The rmiserver will have connected to the first one and, well, you need to break out kill -9.
now, some one more unix clever than I could figure out how to create a template to allow community volunteers to host remote Zombie Robots to give to the QA effort by doing somthing unnatural with IP chains. Or Macs for that matter; a bit easier to administer.
Here's one: 171.64.137.23 it's an old G4.
Configuring JMeter for Sakai - Glenn Golden
I just spent hours today remembering this... so this is my email to remind me next time.
When using JMeter to test Sakai, the default HTML parser that it comes configured with does NOT load iframe contents, even if the Retrieve All Embedded Resources from HTML Files option is checked.
But JMeter ships with other HTML parsers, at least one of which does load our tool content. You can select the "regex" parser by uncommenting this line:
htmlParser.className=org.apache.jmeter.protocol.http.parser.RegexpHTMLPa rser
in the file "jmeter.propeties" found in "jmeter/bin" directory.
Then it works!