I have been playing around with Amazon's Elastic Compute Cloud (EC2). It allows you to provision virtual machines on-demand and configure and control your own compute clusters. To do external (over a WAN) performance\load testing against your web application or services, you can put together a cloud based test harnesses using some simple tools. My open source tool Pylot does the job well for simple web performance and load tests. Pylot generates concurrent load (HTTP Requests), verifies server responses, and produces reports with metrics. Tests suites are executed and monitored from a GUI or shell/console. You define your test cases in an XML file. This is where you specify the requests (url, method, body/payload, etc) and verifications.
It is incredibly easy to provision an instance and launch a virtual machine using the EC2 console. Once you have it up and running, you can just connect to your new virtual machine (via RDP or SSH) and get started.
Here is a screenshot of my terminal session using Pylot (click to enlarge):
You can see I took the following steps:
- remotely login to the EC2 instance using SSH and my private key
- download Pylot using wget
- unzip the distribution
- change to the Pylot directory
- launch the default test with 1 agent (virtual user)
Thats it! I ran a test today doing 1500 Virtual Users from one instance of 64-bit Fedora Linux, and it took me about 5 minutes to get it setup and running. Python 2.5 is already installed on the image, and no further configuration is needed to run a basic test. To run the Pylot GUI and generate results graphs, you need to install wxPython and Matplotlib.
You can upload your own test case files and run any load test scenario you want. Don't forget, you can create and save your own machine images and launch as many as you want to run a large distributed test.
I'd like to build some tools to make this easier and to create preconfigured machine images for other people to start with. It needs common results collection and remote control of user agents. I'm thinking of a cluster of virtual machines all running Pylot, each running thousands of virtual user agents. EC2 provides the ability to place instances in multiple locations, so this could be used to create a massive geo-distributed test bed with capacity on-demand.