As a general rule, web applications should be tested thoroughly. Unfortunately, in practice, testing is spotty and often overlooked. As an
alternative to manual testing I often find myself creating small PHP scripts to parse my pages. I run them regularly to decide if my last changes did more harm then good. That's what regression testing is all about, checking to make sure your application hasn't taken steps backward. But it always seemed like this particular problem deserved a more elegant solution. That's where JMeter comes in. JMeter is slated as a "load tester", but I assure you it delivers much more than that. Originally developed to test load test Jserv (the forerunner of Apace Tomcat), JMeter has since expanded to load test FTP servers, database servers as well as java servlets and objects. Of particular interest to us, is JMeter's ability to run regression tests on web applications.
The first step is to retrieve the latest stable version of JMeter(currently 1.8). You'll also need the latest version of the java engine which you can find on Sun's site. Additionally, if you're going to be testing SSL pages, you might want the install the JSSE
To start JMeter, locate and run the jmeter.bat file (for the Windows flavor). When it starts up you'll see two items in the left tree. If you
right click on the "Test Plan" item you'll see all the modules available for our test. The first element we'll want to add to our test is the "Thread Group". The Thread Group simulates users. Since we're not doing performance testing right now, we'll set the number of threads and loops to 1. The next element we'll need is a Sampler. The Samplers control the requests made to your web application. Right clicking on the Thread Group icon, we add a "HTTP Request" off of the Sampler menu. Now if you click on the HTTP Request icon we can set it up to make requests to our application.
We don't have a web application handy, so we'll just make a pseudo application like so:
echo 'This is data that should show up on this page!';
JMeter will test for the existence of the echoed text. In practice, the echoed text should be out of the database, or another choke point in your code. Now, we'll configure a "Response Assertion" element to ensure the text "This is data" is present on the page (Note that you can use regular expressions in the assertion elements for more advanced validation). If the data doesn't show up on our page, the assertion element will mark the test as a failure.
Then, using the "Not" checkbox we'll add another Response Assertion to make sure that other text is not present, such as the common error strings "warning" and "Parse error".
Now, we could start the test, but we'd have no way of knowing what the results of the test were. So we add another element to the HTTP Request: a "View Results in Table" Listener. The Listener group gives feedback on your requests. This particular Listener doesn't need any additional configuration.
Finally we're ready to run our test plan. Choose "Start" from the "Run" menu to begin the test and highlight the "View Results in Table" element to see your results. You should get something similar to the following:
Of course, your run times will vary. The "Success" box should be checked if everything went as planned. If not, add an "Assertion Results" element to help debug your problem. Or, you can see the page exactly as JMeter sees it by adding a "View Results Tree" from the Listener menu. The View Results Tree shows the full request made to the page and the full response, which is
very useful when debugging your test plan.
Pay special care to how you nest the elements, It does make a difference. For example, if you have a Listener element on the same level in the tree as 2 sampler items, the Listener will get results for both of the samplers. While if the listener was added as a child item of the Sampler, the Listener would only get results from its parent element. The same can be said about assertion elements. So our generic assertion that looks for common error strings on any page can be added on the same level as the samplers. While the data specific assertions should be a child of the page they check.
You're already starting to see how the complexity of these tests can grow exponentially on large applications. When you have one sampler for each module in your web application, the tree fills up fast! One element that can help keep your test plan manageable is the "HTTP Request Defaults" element. Instead of filling in the server and port options on each Sampler, you can leave them blank and the sampler will automatically use values from a HTTP Request Defaults element on the same or higher level in the tree. This can save gobs of time not only when setting the plan up, but also if the application ever moves to a new server.
You're probably saying: "I could have developed this in PHP in half the time it took me to download the java run time!", which is probably true :). But we haven't touched on JMeter's more advanced features like the "HTTP Cookie Manager", which when added to your Thread Group, turns on Cookie support automatically! This allows us make complete test suites for applications that use expiring cookies for authentication. Or you could make a complete test - from login to checkout - for your new shopping cart application, without having to provide the POST and GET variables for every page. Let the cookie manager do the hard work!
But not all applications have cookies for persistent data. So the last element I want to touch on is the "HTTP URL Re-writing Modifier". This one is useful for applications that pass data (such as the $PHPSESSID) around in the URL. Add the re-writer on the same level as a group of Samplers that need a URL-passed variable between them, and the re-writer will automatically pass the value around for you!
I've barely scratched the surface of the possibilities that JMeter has for application testing (there is also a command line version for automated testing!), but I hope that I've sparked some interest in this great tool. If your looking for more on how to use the many modules I haven't mentioned, check out the great project documentation and the project Wiki. If you're feeling intrepid, you might even want to add something to the Wiki to help others out. After all, that's what this community is all about, right?