July 7, 2009

Web Performance Tool Evaluation - lower end proprietary tools

I am in the middle of a Performance and Load tools selection process and wanted to get some feedback here.

I currently work in a shop that uses a mix of proprietary and open source tools for web performance & load testing. The bulk of our workload and analysis is currently done using SilkPerformer. As you all probably know, there is a class of tools that is *very* expensive (including SilkPerformer). Installations and maintenance can run into 7 figures ($$$) with yearly maintenance contracts upwards of 6 figures. Since SilkPerformer is in place and we are happy with it (besides price/maintenance), there is no point in moving to a similarly priced tool. Therefore I have ruled out the class of "high end" tools from my selection:

High-end tools
Borland/Segue - SilkPerformer
HP/Mercury - LoadRunner
IBM/Rational - IBM Rational Performance Tester
Microfocus/Compuware - QALoad
Oracle/Empirix - Oracle Load Testing For Web Applications (e-Load)

The tool I select will be used across several web applications.. pretty straight forward HTML/AJAX/JavaScript Web UI. Here is a basic list of requirements:


- distributed load generation
- reporting/analytics
- data driven testing
- 5000+ VU

I work on a very skilled team that is *very* proficient with programming, tools, and web technologies. Adapting to a new tool or programming language is not much of an issue.

I've searched the Open Source landscape pretty good. There are some fantastic tools (OpenSTA, JMeter, Pylot) to augment our testing, but no open source load generation tool completely meets our criteria.

Open Source tools

Now finally to the question/point....

I am looking at a class of tools that I will call "low-end performance tools". This includes all proprietary tools that are not listed above as "high-end tools". They tend be cheaper and more limited in functionality than the big guns, but are substantially cheaper and sometimes sufficient for complex web performance testing. This is where my interest lies. I have scoured the web and came up with a list of tools to evaluate.

Low-end tools
Microsoft - VSTS
Radview - WebLOAD
SoftLogica - WAPT
Facilita - Forecast
Zoho - QEngine
Neotys - NeoLoad

Does anyone have any feedback or experience reports using any of the "low-end" tools listed above? Are there other tools I am overlooking that I should definitely look into?

any comments/suggestions are appreciated.

Post a Comment