Selenium RC with Python in 30 Seconds

Selenium is a suite of tools to automate web app testing across many platforms. It has various pieces (Core, RC, IDE, etc), and I struggled trying to figure out how everything fits together and works. At the end of the day, all I wanted to do was use Selenium from my Python code to drive a browser session.

Selenium ships with full tests and some good sample code, but the driver examples all contain test frameworks/runners (JUnit, unittest, etc). All I wanted was a simple way to integrate with Python. To do this, the piece I need is Selenium RC (which includes Selenium Core).

So here is the beginners' 30-second guide to getting the Python client driver working with Selenium-RC:

1. Install Python (and add it to your path)
2. Install Java (and add it to your path)
3. Download Selenium RC
4. Unzip Selenium RC and search for 'selenium-server.jar' and 'selenium.py'
5. Copy them to a directory and run 'java -jar selenium-server.jar' to start the server
6. Start writing your Python driver code! (import selenium)

So what's actually going on when you run a driver script?

Selenium-server is the core program, which also contains an integrated web server. You send HTTP requests to the server to instruct it how to drive your browser. The 'selenium.py' module is just a wrapper around 'httplib' that provides a Python API for interacting with the Selenium-server. You need to import the 'selenium.py' module from your script and then you are ready to go.

now let's test it out and see if it works. Try the following Python script. It should open Firefox and navigate to www.google.com.

#!/usr/bin/env python

from selenium import selenium

sel = selenium('localhost', 4444, '*firefox', 'http://www.google.com/')
sel.start()
sel.open('/')
sel.wait_for_page_to_load(10000)
sel.stop()

Python - HTTP Request Profiler

Here is a script in Python that profiles an HTTP request to a web server.

to use, give it a url:

> python http_profiler.py www.goldb.org

output:

0.12454 request sent
0.24967 response received
0.49155 data transferred (9589 bytes)

It uses Python's httplib to send a GET request to the URL and adds some timing instrumentation between each step of the request/response.

Python Code:

#!/usr/bin/env python
# Corey Goldberg - September 2009


import httplib
import sys
import time



if len(sys.argv) != 2:
    print 'usage:\nhttp_profiler.py \n(do not include http://)'
    sys.exit(1)

# get host and path names from url
location = sys.argv[1]
if '/' in location:
    parts = location.split('/')
    host = parts[0]
    path = '/' + '/'.join(parts[1:])
else:
    host = location
    path = '/'

# select most accurate timer based on platform
if sys.platform.startswith('win'):
    default_timer = time.clock
else:
    default_timer = time.time

# profiled http request
conn = httplib.HTTPConnection(host)
start = default_timer()  
conn.request('GET', path)
request_time = default_timer()
resp = conn.getresponse()
response_time = default_timer()
size = len(resp.read())
conn.close()     
transfer_time = default_timer()

# output
print '%.5f request sent' % (request_time - start)
print '%.5f response received' % (response_time - start)
print '%.5f content transferred (%i bytes)' % ((transfer_time - start), size)

Web Page Profilers for Windows/Internet Explorer

A "Web Page Profiler" is a tool that is used to measure and analyze web page performance. It is usually implemented as a browser plugin or add-on, and lets you see performance of web pages/objects as they are transferred/loaded/executed/rendered.

For Firefox, choosing a profiler is a no-brainer. Firebug is an excellent developer tool that includes profiling capabilities.

• Firebug

Unfortunately, Firebug does not work with Microsoft's Internet Explorer. There is Firebug Lite, which "can" work with IE, but it is limited in functionality and requires you to install some server side code (which is not always feasible).

• Firebug Lite

There are some web profilers specifically for IE, but none of them live up to the functionality or stability of Firebug. Some to look at are:

• AOL Pagetest
• MySpace Performance Tracker

... also, Internet Explorer 8 has built-in "Developer Tools" (press F12) that include a basic performance profiler for JavaScript execution.

Does anyone know of any other profilers/performance tools that work with IE? Thoughts?

Segue/Borland/MicroFocus SilkPerformer - Who Owns It?

SilkPerformer is a popular performance and load testing tool. It has a confusing history and has bounced around between companies quite a bit. Here is where it came from and the current state of who owns it.

The history AFAIK:

• Segue buys a product from some (scandanavian?) company and names it Segue SilkPerformer.
• Borland buys Segue and names it Borland SilkPerformer.
• Micro Focus buys Borland and name it Micro Focus SilkPerformer.

Right now it is a little confusing because Borland still lists it as a product. The acquisition of Borland just went through a few days ago and now Micro Focus has it listed on its site also. I would expect the Borland site to eventually disappear.

To make it even more confusing; Micro Focus also bought Compuware which had its own performance/load testing application. So now Micro Focus is marketing QALoad as well as SilkPerformer, which overlap in most functionality.

It will be interesting to see where this goes and what happens to the product lines.

[Software] Tools Don't Make You a Mechanic

FYI: Learning LoadRunner does not mean that one has learned Performance Testing. LoadRunner is only a tool. Learning to use one of these:

...does not mean that one can disassemble and reassemble one of these:

[via JakeBrake]

Python - Zip Directories Recursively

This helped me out today with some backup scripts. Posting here so I can remember it. Idea and snippet adapted from: http://mail.python.org/pipermail/python-list/2007-February/596539.html

        
#!/usr/bin/env python

import os
import zipfile


def main():
    zipper('c:/test', 'c:/temp/test.zip')


def zipper(dir, zip_file):
    zip = zipfile.ZipFile(zip_file, 'w', compression=zipfile.ZIP_DEFLATED)
    root_len = len(os.path.abspath(dir))
    for root, dirs, files in os.walk(dir):
        archive_root = os.path.abspath(root)[root_len:]
        for f in files:
            fullpath = os.path.join(root, f)
            archive_name = os.path.join(archive_root, f)
            print f
            zip.write(fullpath, archive_name, zipfile.ZIP_DEFLATED)
    zip.close()
    return zip_file


if __name__ == '__main__':
    main()

* code updated. there was a bug in the original I posted (cmg - 07/13/09)

OpenSTA - SCL Code Boilerplate for HTTP Load Tests

(small code dump...)

OpenSTA (Open Systems Testing Architecture) is a popular open source web performance test tool. It uses a scripting language named SCL (Script Control Language), which seems to be heavily influenced by Fortran. It's a little bit dated and clumsy to program with, but suffices for writing scripts modeling complex web transactions.

Here is the basic structure I start with when modeling tests in OpenSTA:

!Browser:IE5


Environment
    Description "TEST SCRIPT"
    Mode HTTP
    WAIT UNIT MILLISECONDS


Definitions
    Include         "RESPONSE_CODES.INC"
    Include         "GLOBAL_VARIABLES.INC"
    CONSTANT        DEFAULT_HEADERS = "Host: www.goldb.org^J" &
                        "User-Agent: OpenSTAzilla/4.0"
    Integer         USE_PAGE_TIMERS
    Timer           T_Response
    CHARACTER*32768 logStuff, Local
    CHARACTER*512   USER_AGENT
    CHARACTER*256   MESSAGE


Code
    Entry[USER_AGENT,USE_PAGE_TIMERS]

    Start Timer T_Response

    PRIMARY GET URI "http://www.goldb.org/index.html HTTP/1.0" ON 1 &
        HEADER DEFAULT_HEADERS &
        ,WITH {"Accept: */*", "Accept-Language: en-us"}

    SYNCHRONIZE REQUESTS
    
    DISCONNECT FROM 1
    
    End Timer T_Response
  
Exit


ERR_LABEL:
    If (MESSAGE <> "") Then
        Report MESSAGE
    Endif

Exit

Nothing much to see here. If you use the OpenSTA recorder and record a simple HTTP GET request, it would generate a similar script for you.

Web Performance Tool Evaluation - lower end proprietary tools

I am in the middle of a Performance and Load tools selection process and wanted to get some feedback here.

I currently work in a shop that uses a mix of proprietary and open source tools for web performance & load testing. The bulk of our workload and analysis is currently done using SilkPerformer. As you all probably know, there is a class of tools that is *very* expensive (including SilkPerformer). Installations and maintenance can run into 7 figures ($$$) with yearly maintenance contracts upwards of 6 figures. Since SilkPerformer is in place and we are happy with it (besides price/maintenance), there is no point in moving to a similarly priced tool. Therefore I have ruled out the class of "high end" tools from my selection:

High-end tools
------------------------
Borland/Segue - SilkPerformer
HP/Mercury - LoadRunner
IBM/Rational - IBM Rational Performance Tester
Microfocus/Compuware - QALoad
Oracle/Empirix - Oracle Load Testing For Web Applications (e-Load)

The tool I select will be used across several web applications.. pretty straight forward HTML/AJAX/JavaScript Web UI. Here is a basic list of requirements:

Requirements
------------------------
Protocols:
- HTTP
- ODBC (SQL)

Features:
- distributed load generation
- reporting/analytics
- data driven testing
- 5000+ VU

I work on a very skilled team that is *very* proficient with programming, tools, and web technologies. Adapting to a new tool or programming language is not much of an issue.

I've searched the Open Source landscape pretty good. There are some fantastic tools (OpenSTA, JMeter, Pylot) to augment our testing, but no open source load generation tool completely meets our criteria.

Open Source tools
------------------------
OpenSTA
JMeter
Pylot

Now finally to the question/point....

I am looking at a class of tools that I will call "low-end performance tools". This includes all proprietary tools that are not listed above as "high-end tools". They tend be cheaper and more limited in functionality than the big guns, but are substantially cheaper and sometimes sufficient for complex web performance testing. This is where my interest lies. I have scoured the web and came up with a list of tools to evaluate.

Low-end tools
------------------------
Microsoft - VSTS
Radview - WebLOAD
SoftLogica - WAPT
Facilita - Forecast
Zoho - QEngine
Neotys - NeoLoad

Does anyone have any feedback or experience reports using any of the "low-end" tools listed above? Are there other tools I am overlooking that I should definitely look into?

any comments/suggestions are appreciated.