Mini Web Load Tester with Python and Pylot Core

Pylot is a performance tool for benchmarking web services/applications. I am working on exposing some of Pylot's internals so you can use it as a Python Module/API for generating concurrent HTTP load.

Below is a simple function that runs a mini [multi-threaded] load test against a single URL. It will return a dictionary containing runtime statistics. Results and timing information from each request is also logged to a file.

I use something like this to run performance unit tests rapidly (10-30 secs usually). I can bang on the URL for my application and quickly see how it performs and scales.

Here is a small Python script that uses Pylot as a module:

#!/usr/bin/env python

import pylot.core.engine as pylot_engine
import os
import sys
import time
        

pylot_engine.GENERATE_RESULTS = False

url = 'http://www.pylot.org'
num_agents = 5
duration = 10
runtime_stats = {}

original_stdout = sys.stdout
sys.stdout = open(os.devnull, 'w')

req = pylot_engine.Request(url)
lm = pylot_engine.LoadManager(num_agents, 0, 0, False, runtime_stats, [])
lm.add_req(req)

lm.start()
time.sleep(duration)
lm.stop()

sys.stdout = original_stdout

for agent_num, stats in runtime_stats.iteritems():
    print 'agent %i : %i reqs : avg %.3f secs' % \
        (agent_num + 1, stats.count, stats.avg_latency)

Output:

agent 1 : 46 reqs : avg 0.220 secs
agent 2 : 46 reqs : avg 0.218 secs
agent 3 : 46 reqs : avg 0.220 secs
agent 4 : 46 reqs : avg 0.221 secs
agent 5 : 46 reqs : avg 0.221 secs

Here is a slightly larger example with some more structure and features. This creates a small command line interface for running a mini load test.

Code:

#!/usr/bin/env python
# Corey Goldberg 2009

import pylot.core.engine as pylot_engine
import os
import sys
import time



def main():
    """
    Usage: >python pylot_mini_loadtest.py   
    """
    url = sys.argv[1]
    num_agents = int(sys.argv[2])
    duration = int(sys.argv[3])
    pylot_engine.GENERATE_RESULTS = False
    print '\nmini web load test \n---------------------------------'
    agent_stats = run_loadtest(url, num_agents, duration)
    throughput = sum([stat.count for stat in agent_stats.values()]) / float(duration)
    print '%.2f reqs/sec' % throughput
    for agent_num, stats in agent_stats.iteritems():
        print 'agent %i : %i reqs : avg %.3f secs' % \
            (agent_num + 1, stats.count, stats.avg_latency)
        


def run_loadtest(url, num_agents, duration):
    """
    Runs a load test and returns a dictionary of statistics from agents.
    """
    original_stdout = sys.stdout
    sys.stdout = open(os.devnull, 'w')
    
    runtime_stats = {}
    req = pylot_engine.Request(url)
    lm = pylot_engine.LoadManager(num_agents, 0, 0, False, runtime_stats, [])
    lm.add_req(req)
    
    lm.start()
    time.sleep(duration)
    lm.stop()
    
    sys.stdout = original_stdout
    
    return runtime_stats



if __name__ == '__main__':
    main()

Usage/Output:

C:\test>python pylot_mini_loadtest.py http://www.goldb.org 8 10

mini web load test
---------------------------------
19.20 reqs/sec
agent 1 : 24 reqs : avg 0.416 secs
agent 2 : 24 reqs : avg 0.418 secs
agent 3 : 24 reqs : avg 0.417 secs
agent 4 : 24 reqs : avg 0.418 secs
agent 5 : 24 reqs : avg 0.415 secs
agent 6 : 24 reqs : avg 0.419 secs
agent 7 : 24 reqs : avg 0.419 secs
agent 8 : 24 reqs : avg 0.419 secs

Of course, for more complex scenarios, you can use the full blown tool, available at: www.pylot.org/download.html

Questions? Hit me up.

Pylot - Total Downloads So Far

Here is a graph showing total downloads of Pylot since its first release:

Decent uptake so far. Keep the downloads coming!

Pylot is a web performance testing tool. It is Free Open Source Software.

Python - Redirect or Turn Off STDOUT and STDERR

Here is an easy way to temporarily turn off STDOUT or STDERR in your Python program.

First you create a class to replace STDOUT. This is just minimal class with a 'write()' method.

class NullDevice():
    def write(self, s):
        pass

Notice its 'write()' method does nothing. Therefore, when you write to the NullDevice, output goes nowhere and is dropped. All you need to do is assign sys.stdout to this class.

Here is an example of turning STDOUT off and back on:

#!/usr/bin/env python

import sys


class NullDevice():
    def write(self, s):
        pass


print "1 - this will print to STDOUT"

original_stdout = sys.stdout  # keep a reference to STDOUT

sys.stdout = NullDevice()  # redirect the real STDOUT

print "2 - this won't print"

sys.stdout = original_stdout  # turn STDOUT back on

print "3 - this will print to SDTDOUT"

You can also do the same thing with sys.stderr to turn off STDERR.

C# - Export Windows Event Logs

Here is a little C# program to export Windows Event Logs. It reads an Event Log and prints entries to STDOUT so you can pipe the output to a file or other application.

using System;
using System.Diagnostics;


class EventLogExporter
{
    static void Main(string[] args)
    {
        EventLog evtLog = new EventLog("Application");  // Event Log type
        evtLog.MachineName = ".";  // dot is local machine
        
        foreach (EventLogEntry evtEntry in evtLog.Entries)
        {
            Console.WriteLine(evtEntry.Message);
        }
        
        evtLog.Close();
    }
}

Dell Mini 10 Netbook with Linux == Graphics FAIL

If you are planning on buying a Dell Mini 10 (or Mini 12) to run Linux, read this...

I used to have the Dell Mini 9 that came with Linux (Ubuntu 8.04). As soon as I got it, I paved it and installed Ubuntu Intrepid instead. It worked like a charm. Then I decided to sell my Mini 9 and upgrade to the Mini 10. The Mini 10 is a better machine in terms of hardware, and is MUCH better in terms of screen resolution and keyboard size (best keyboard on any netbook).

So, the Mini 10 ships with Windows installed. Since I had such good luck with the Mini 9, I figured a Linux install would be a breeze. So with my shiny new Mini 10 netbook, I tried an install Ubuntu Intrepid. It worked great but no compatible graphics driver. OK, so I waited for the Ubuntu Jaunty release and then promptly installed that. Same prob.

Here is the deal: There is no Linux driver for the graphics card it uses (Intel GMA 500). So.. if you want to run Linux on it, your only choice is to run in a non-native resolution using the default driver. This totally sucks.

There appears to be a native Linux driver somewhere (Poulsbo), but it doesn't work right now and is not packaged.

I am just running Windows for now and waiting for a real native driver to be released. Shame on Intel for not providing one.

Scripted Testing Isn't Just Following Scripts

There is an ongoing (or dead horse, depending on your perspective) about "scripted" vs. "exploratory" testing.

I happen to refer to "scripted testing" as programmatic testing. You use programs, scripts, and tools to augment/enable your testing. You can explore a system with your toolset if you want. That is an example of doing exploratory testing with scripts/programs/tools.

The debate seems to overlook that definition and defines "scripted" as just following a number of predefined steps. I think this is the wrong definition and the wrong argument.. or maybe I just don't get it... or maybe I'm confused by the ambiguous definitions of scripting.

I don't see it as a boolean. I think of it terms of a spectrum and somewhere along that programmatic/manual continuum is where you work. Exploratory testing can fall in many areas of the spectrum and you can do it manually or programmatically.

That is where the argument breaks down (IMHO).