What is the Best Way To Learn Selenium?

(a small rant on asking answerable questions)

At a certain online QA/Testing Forum I regularly visit, the Selenium forum is one of the most active.

I continuously see newbie questions like: "How I do I learn Selenium?", with little or no other context given.

Then, helpful responders chime in with tangentially related answers. That's about as good as you can do, based on the lack of information given in the original post. This happens often and wastes significant brain activity in the responders, while generally giving no value to the OP or forum community.

So first... a clarification:

Selenium WebDriver is a system for automating browser interaction. It is accessed via programming library/API. If you want to use it without touching code, it is not possible (record/replay/export-madness via IDE aside). You can abstract away most of the programming aspects with higher level frameworks, but at that point you are no longer really dealing with Selenium anymore.

---

Here was my response to a recent forum example:

Question asked:
"Best way to start learning Selenium - I would like to know what is the best practice to start with Selenium for non programmers. Is there any books or ebooks?"

The responses compelled me to chime in with a rant:

---

To anyone on this thread looking for the "Best way to start learning Selenium":

Think of it this way:

Selenium (WebDriver) provides language bindings and a similar API for several programming languages. It gives you a programming library that can manipulate a browser and web elements/controls.

You can't "learn" a library for a programming language if you don't know the language itself (syntax, idioms, etc).

So the question shouldn't be: "How do I learn Selenium?", with no other information or context given.

Selenium should be used with either: the supported language that you are most comfortable with, or the language of the system under test you are working against.

If you are looking for information, be specific about what you want to learn. General understanding of the components that make up Selenium is great, but really, using the API's from any language is the best way to learn. If you specify a programming language along with your question, at least answers can be directed towards frameworks, code samples, documentation, tutorials, etc, that are *relevant* to you.

Diving into the main Selenium docs can be confusing at first, as it covers things from a general level... mixing in code samples from various languages.

So, my question to you: Which programming language are you going to use Selenium with? If you define that, someone can surely help point you in the right direction.

How are your programming skills? Which language do you want to learn in? If you know several, switching is trivial down the line, as the API's are similar between languages.

If you don't have skills in any Selenium supported programming language (java, c#, python, ruby, php, etc), then learn one of those before you even touch Selenium. If you have a language in mind, please phrase your question using that in context.

happy hacking,

-Corey

p.s. now go RTFM: http://seleniumhq.org/docs/

---

rant over.

Python Timer Class - Context Manager for Timing Code Blocks

Here is a handy Python Timer class. It creates a context manager object, used for timing a block of code.

from timeit import default_timer


class Timer(object):
    def __init__(self, verbose=False):
        self.verbose = verbose
        self.timer = default_timer
        
    def __enter__(self):
        self.start = self.timer()
        return self
        
    def __exit__(self, *args):
        end = self.timer()
        self.elapsed_secs = end - self.start
        self.elapsed = self.elapsed_secs * 1000  # millisecs
        if self.verbose:
            print 'elapsed time: %f ms' % self.elapsed

To use the Timer (context manager object), invoke it using Python's `with` statement. The duration of the context (code inside your `with` block) will be timed. It uses the appropriate timer for your platform, via the `timeit` module.

Timer is used like this:

with Timer() as target:
    # block of code goes here.
    # result (elapsed time) is stored in `target` properties.

Example script:
timing a web request (HTTP GET), using the `requests` module.

#!/usr/bin/env python

import requests
from timer import Timer

url = 'https://github.com/timeline.json'

with Timer() as t:
    r = requests.get(url)
    
print 'fetched %r in %.2f millisecs' % (url, t.elapsed)

Output:

fetched 'https://github.com/timeline.json' in 458.76 millisecs

---

`timer.py` in GitHub Gist form, with more examples:

My Evolving Droids

(Droid -> Droid X -> Droid Razr) [stock pics]

In the past few years, mobile devices have turned into amazing little pocket computers. I'm blown away by the hardware (ARM) and software (Android) evolution.

For smartphone wireless service, I was a Verizon customer (USA) for the past decade. I've owned the flagship Droid series devices by Motorola since their first launch (currently running on the 4G LTE network). It started with a clunky original Droid, followed by a Droid X, and then the sleek Droid Razr... every year another generation of better, faster devices.

Device Specs:

Motorola Droid - Nov 2009

• OS: Android v2.0
• CPU: 600 MHz ARM Cortex-A8 (TI OMAP3430, 65nm)
• RAM: 256 MB
• Internal Storage: 512 MB
• Screen Size: 3.7 inch
• Resolution: 480 x 854

Motorola Droid X - July 2010

• OS: Android v2.1
• CPU: 1 GHz ARM Cortex-A8 (TI OMAP3630, 45nm)
• RAM: 512 MB
• Internal Storage: 8 GB
• Screen Size: 4.3 inch
• Resolution: 480 x 854

Motorola Droid Razr - Nov 2011

• OS: Android v2.3
• CPU: 1.2 GHz dual-core ARM Cortex-A9 (TI OMAP4430, 45nm)
• RAM: 1 GB
• Internal Storage: 16 GB
• Screen Size: 4.3 inch
• Resolution: 540 x 960

However, my life with the Moto Droids is coming to an end. I just signed up with a new wireless carrier and pre-ordered a new Sammy GS3!:

(Samsung Galaxy S3) [stock pic]

Samsung Galaxy S3 - June 2012

• OS: Android v4.0
• CPU: 1.5 GHz dual-core Krait (Qualcomm Snapdragon S4 MSM8960, 28nm)
• RAM: 2 GB
• Internal Storage: 16 GB
• Screen Size: 4.8 inch
• Resolution: 720 x 1280

Home Audio Setup

I work from my apartment and listen to music or talk-radio nearly 24-hours a day (even at soft volume while I sleep). My home audio setup is pretty important to me.

Here is a description of the current rig for home listening:

---

The Gear:

• Media Server (Ubuntu, Logitech Media Server)
• Touchscreen UI (Android, Squeezebox Controller)
• Wifi Music Player (Squeezebox Touch)
• Amplifier (Harman Kardon)
• 5 Speakers (Polk Audio)

---

The Content:

Streaming Radio Networks:

• Slacker Radio Plus ($3.99/month) - slacker.com
• SiriusXM Internet Radio ($14.49/month) - siriusxm.com
• Pandora (Free account) - pandora.com
• SomaFM (Free) - somafm.com

MP3 Collection:

• 7010 Files
• 60 GB
• Track List: Corey's MP3s

---

Tablet interface (Logitech Squeezebox Controller):

Web interface (Logitech Media Server 7.7.2):

---

Rock on!

History of Python - Development Visualization - Gource

I made a new visualization. Have a look!

History of Python - Gource - development visualization (august 1990 - june 2012)
[HD video, encoded at 1080p. watch on YouTube in highest resolution possible.]

What is it?

This is a visualization of Python core development. It shows growth of the Python project's source code over time (August 1990 - June 2012). Nearly 22 years! The source code history and relations are displayed by Gource as an animated tree, tracking commits over time. Directories appear as branches with files as leaves. Developers can be seen working on the tree at the times they contributed to the Python project.

Video:
Rendered with Gource v0.37 on Ubuntu 12.04

Music:
Chris Zabriskie - The Life and Death of a Certain K Zabriskie Patriarch

Repository:
cpython 3.3.0 alpha, retrieved from mercurial on June 2 2012

---

for more visualizations and other videos, check out my YouTube channel.

SST 0.2.1 Release Announcement (selenium-simple-test)

SST version 0.2.1 has been released.

SST (selenium-simple-test) is a web test framework that uses Python to generate functional browser-based tests.

---

SST version 0.2.1 is on PyPI: http://pypi.python.org/pypi/sst

install or upgrade with:

pip install -U sst

Changelog: http://testutils.org/sst/changelog.html
SST Docs: http://testutils.org/sst
SST on Launchpad: https://launchpad.net/selenium-simple-test

---

SST downloads | (Jan 1 2012 - April 23 2012)

1600+ downloads from PyPI since initial release.

Python - Getting Data Into Graphite - Code Examples

This post shows code examples in Python (2.7) for sending data to Graphite.

Once you have a Graphite server setup, with Carbon running/collecting, you need to send it data for graphing.

Basically, you write a program to collect numeric values and send them to Graphite's backend aggregator (Carbon).

To send data, you create a socket connection to the graphite/carbon server and send a message (string) in the format:

"metric_path value timestamp\n"

• `metric_path`: arbitrary namespace containing substrings delimited by dots. The most general name is at the left and the most specific is at the right.
• `value`: numeric value to store.
• `timestamp`: epoch time.
• messages must end with a trailing newline.
• multiple messages maybe be batched and sent in a single socket operation. each message is delimited by a newline, with a trailing newline at the end of the message batch.

Example message:

"foo.bar.baz 42 74857843\n" 

Let's look at some (Python 2.7) code for sending data to graphite...

---

Here is a simple client that sends a single message to graphite.

Code:

#!/usr/bin/env python

import socket
import time


CARBON_SERVER = '0.0.0.0'
CARBON_PORT = 2003

message = 'foo.bar.baz 42 %d\n' % int(time.time())

print 'sending message:\n%s' % message
sock = socket.socket()
sock.connect((CARBON_SERVER, CARBON_PORT))
sock.sendall(message)
sock.close()

---

Here is a command line client that sends a single message to graphite:

Usage:

$ python client-cli.py metric_path value

Code:

#!/usr/bin/env python

import argparse
import socket
import time


CARBON_SERVER = '0.0.0.0'
CARBON_PORT = 2003


parser = argparse.ArgumentParser()
parser.add_argument('metric_path')
parser.add_argument('value')
args = parser.parse_args()


if __name__ == '__main__':
    timestamp = int(time.time())
    message = '%s %s %d\n' % (args.metric_path, args.value, timestamp)
    
    print 'sending message:\n%s' % message
    sock = socket.socket()
    sock.connect((CARBON_SERVER, CARBON_PORT))
    sock.sendall(message)
    sock.close()

---

Here is a client that collects load average (Linux-only) and sends a batch of 3 messages (1min/5min/15min loadavg) to graphite. It will run continuously in a loop until killed. (adjust the delay for faster/slower collection interval):

#!/usr/bin/env python
 
import platform
import socket
import time


CARBON_SERVER = '0.0.0.0'
CARBON_PORT = 2003
DELAY = 15  # secs


def get_loadavgs():
    with open('/proc/loadavg') as f:
        return f.read().strip().split()[:3]


def send_msg(message):
    print 'sending message:\n%s' % message
    sock = socket.socket()
    sock.connect((CARBON_SERVER, CARBON_PORT))
    sock.sendall(message)
    sock.close()


if __name__ == '__main__':
    node = platform.node().replace('.', '-')
    while True:
        timestamp = int(time.time())
        loadavgs = get_loadavgs()
        lines = [
            'system.%s.loadavg_1min %s %d' % (node, loadavgs[0], timestamp),
            'system.%s.loadavg_5min %s %d' % (node, loadavgs[1], timestamp),
            'system.%s.loadavg_15min %s %d' % (node, loadavgs[2], timestamp)
        ]
        message = '\n'.join(lines) + '\n'
        send_msg(message)
        time.sleep(DELAY)

---

Resources:

• Graphite Docs
• Graphite Docs - Getting Your Data Into Graphite
• Installing Graphite 0.9.9 on Ubuntu 12.04 LTS
• Installing and configuring Graphite

Installing Graphite 0.9.9 on Ubuntu 12.04 LTS

I just setup a Graphite server on Ubuntu 12.04 (Precise).

Here are some instructions for getting it all working (using Apache as web server).

It follows these steps:

• install system dependencies (apache, django, dev libs, etc)
• install Whisper (db lib)
• install and configure Carbon (data aggregator)
• install Graphite (django webapp)
• configure Apache (http server)
• create initial database
• start Carbon (data aggregator)

Once that is done, you should be able to visit the host in your web browser and see the Graphite UI.

---

Setup Instructions:

#############################
# INSTALL SYSTEM DEPENDENCIES
#############################

$ sudo apt-get install apache2 libapache2 libapache2-mod-wsgi /
    libapache2-mod-python memcached python-dev python-cairo-dev /
    python-django python-ldap python-memcache python-pysqlite2 /
    python-pip sqlite3 erlang-os-mon erlang-snmp rabbitmq-server
    
$ sudo pip install django-tagging

#################
# INSTALL WHISPER
#################

$ sudo pip install http://launchpad.net/graphite/0.9/0.9.9/+download/whisper-0.9.9.tar.gz

################################################
# INSTALL AND CONFIGURE CARBON (data aggregator)
################################################

$ sudo pip install http://launchpad.net/graphite/0.9/0.9.9/+download/carbon-0.9.9.tar.gz
$ cd /opt/graphite/conf/
$ sudo cp carbon.conf.example carbon.conf
$ sudo cp storage-schemas.conf.example storage-schemas.conf

###########################
# INSTALL GRAPHITE (webapp)
###########################

$ sudo pip install http://launchpad.net/graphite/0.9/0.9.9/+download/graphite-web-0.9.9.tar.gz

or

$ wget http://launchpad.net/graphite/0.9/0.9.9/+download/graphite-web-0.9.9.tar.gz
$ tar -zxvf graphite-web-0.9.9.tar.gz
$ mv graphite-web-0.9.9 graphite
$ cd graphite
$ sudo python check-dependencies.py
$ sudo python setup.py install

##################
# CONFIGURE APACHE
##################

$ cd graphite/examples
$ sudo cp example-graphite-vhost.conf /etc/apache2/sites-available/default
$ sudo cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
$ sudo mkdir /etc/httpd
$ sudo mkdir /etc/httpd/wsgi
$ sudo /etc/init.d/apache2 reload

#########################
# CREATE INITIAL DATABASE 
#########################

$ cd /opt/graphite/webapp/graphite/
$ sudo python manage.py syncdb
$ sudo chown -R www-data:www-data /opt/graphite/storage/
$ sudo /etc/init.d/apache2 restart
$ sudo cp local_settings.py.example local_settings.py

################################
# START CARBON (data aggregator)
################################

$ cd /opt/graphite/
$ sudo ./bin/carbon-cache.py start

Resources:

• http://graphite.readthedocs.org/en/latest/install.html
• http://graphite.wikidot.com/installation
• http://geek.michaelgrace.org/2011/09/how-to-install-graphite-on-ubuntu/

---

* works on my machine, Ubuntu 12.04

Python - Graphite: Storage and Visualization of Time-series Data

I'm doing some work with Graphite in Python. Here is a quick overview of what Graphite is...

---

Graphite provides real-time visualization and storage of numeric time-series data.

Links:

• Project: https://launchpad.net/graphite
• Docs: http://graphite.readthedocs.org

Graphite does two things:

• Store numeric time-series data
• Render graphs of this data on demand

Graphite consists of a storage backend and a web-based visualization frontend. Client applications send streams of numeric time-series data to the Graphite backend (called carbon), where it gets stored in fixed-size database files similar in design to RRD. The web frontend provides 2 distinct user interfaces for visualizing this data in graphs as well as a simple URL-based API for direct graph generation.

Graphite consists of 3 software components:

• carbon - a Twisted daemon that listens for time-series data
• whisper - a simple database library for storing time-series data (similar in design to RRD)
• graphite webapp - A Django webapp that renders graphs on-demand using Cairo

Python Book Giveaway - Boston Python User Group - April 12, 2012

I am bringing some of my [lightly read] Python books to give away at the next Boston Python User Group meetup.

Boston Python User Group - April Project Night
When: Thursday, April 12, 2012, 6:30 PM
Where: Microsoft NERD, Cambridge

Books:

• Pro Python (Alchin)
• Head First Python (Barry)
• Programming in Python 3 (Summerfield)
• Python Programming Patterns (Christopher)
• Hello World! (Sande)

If you are interested in getting a free book, come to #bostonpython Project Night on April 12!

Python Screencast: Install/Setup "SST Web Test Framework" on Ubuntu 12.04

I uploaded a 5 minute video/screencast showing how to install and setup SST Web Test Framework on Ubuntu (Precise Pangolin 12.04).

I step through: creating a virtualenv, installing SST from PyPI, and creating a basic automated web test:

http://www.youtube.com/watch?v=LpSvGmglZPI

the following steps are essentially a transcript of what I did...

---

install system package dependencies:

$ sudo apt-get install python-virtualenv xvfb

create a "virtualenv":

$ virtualenv ENV

active the virtualenv:

$ cd ENV
$ source bin/activate
(ENV)$

* notice your prompt changed, signifying the virtualenv is active

install SST using `pip`:

(ENV)$ pip install sst

Now SST is installed. You can check the version of SST:

$ sst-run -V

---

Codeswarm - Python Core Development Visualization

particle visualization of Python core development commits: Jan 1, 2010 - Mar 06, 2012

http://www.youtube.com/watch?v=IQPuU_YtN8Q

data source is the commit log from cpython mercurial trunk:

$ hg clone http://hg.python.org/cpython

... trimmed to show development activity since Jan 1, 2010.

(images making up this video were rendered with: Codeswarm)

SST - Automated Web Page Profiling (Python)

SST - Web Test Framework: http://testutils.org/sst

The latest release of SST (0.2.0) adds the ability to capture HAR (HTTP Archive format) output for pageload performance tracing/profiling.

New SST doc section: Performance tracing with Browsermob Proxy (HAR)

The HAR format is based on JSON, and is used by tools that consume/produce data collected by monitoring HTTP communication. These files contain a log of HTTP client/server conversation and can be used for additional analysis of page load performance.

The capture is achieved by routing browser requests through BrowserMob Proxy, which records web page loads while your tests run. SST will launch the proxy and save output to .har files if you enable the `--browsermob` command line option. HAR files are saved in the results directory for each page load.

HAR files can be viewed/analyzed with various tools, such as `harviewer`:

• http://www.softwareishard.com/har/viewer/
• http://pcapperf.appspot.com/
• http://code.google.com/p/harviewer/

I created a screencast demo showing it all together:

Automated Web Page Profiling : SST + BrowserMob

SST 0.2.0 Release Announcement (and codebase visualization)

SST (selenium-simple-test) is a web test framework that uses Python to generate functional browser-based tests.

Version 0.2.0 Released!

SST version 0.2.0 is on PyPI: http://pypi.python.org/pypi/sst

install or upgrade with:

pip install -U sst

Changelog: http://testutils.org/sst/changelog.html
SST Docs: http://testutils.org/sst
SST on Launchpad: https://launchpad.net/selenium-simple-test

---

Development Update

So... what's up with SST development? is it active?

I ran Gource against the trunk branch and created an awesome visualization (with soundtrack!) to show off development activity:

video: link (with soundtrack)
video: link (no soundtrack)

The magic incantation of `gource` to produce the video:

$ gource \
    -s .3 \
    -1280x720 \
    --auto-skip-seconds .3 \
    --multi-sampling \
    --stop-at-end \
    --hide mouse,progress \
    --file-idle-time 0 \
    --max-files 0  \
    --background-colour 222222 \
    --font-size 20 \
    --logo docs/assets/sst-logo_small.png \
    --title "SST Development - lp:selenium-simple-test" \
    --output-ppm-stream - \
    --output-framerate 30 \
    | ffmpeg -y -r 30 -f image2pipe -vcodec ppm -i - -b 2048K movie.mp4

(rendered on Ubuntu 11.10, audio mixed with Pitivi)

Python - Matplotlib and Numpy on Debian/Ubuntu

There are `python-matplotlib` and `python-numpy` packages in the Debian/Ubuntu repos.

However, if you want to run in a virtualenv (with no-site-packages), and pip install these packages from PyPI, you need some system dependencies installed first to build with:

$ sudo apt-get install build-essential python-dev libfreetype6-dev libpng-dev python-virtualenv

Then, you can create a virtualenv, and the installers for Numpy and Matplotlib will work:

$ virtualenv env
$ cd env
$ source bin/activate
(env)$ pip install numpy matplotlib

...
...
Successfully installed numpy matplotlib
Cleaning up...

(tested on Ubuntu Oneiric 11.10 and Ubuntu Precise 12.04 alpha)

Officially Introducing "SST" (Python Web Test Framework)

"SST (selenium-simple-test) is a framework built on Selenium WebDriver, using Python to make writing functional web tests easier with code."

Since early 2011, I have been working for Canonical on the Infrastructure Systems Development team (Core Dev Ops).

[pic of canonical-isd-hackers at UDS-P-Orlando]

A by-product of our recent development efforts is a web testing framework. It has been available on Launchpad for a while, but I've never really announced it in public. We are using SST internally, and I want to expose it to a wider audience.

What is Selenium WebDriver?

Selenium (WebDriver) is a popular open-source library for automating browsers. It can be used to create functional/acceptance tests of a web application. The Selenium client bindings provide API's that allow you to programatically drive a browser and access web content/elements. The bindings are available and supported for many languages and platforms.

While working directly with Selenium API's from code is fine for ad-hoc browser interaction, it is rather low-level and lacks things necessary for creating suites of automated web tests. For larger-scale testing, you will soon want to use a framework to help organize, execute, and report.

Introducing SST...

SST aims to keep things simple.

Tests are made up of scripts, created by composing actions that drive a browser and assert conditions. You have the flexibility of the full Python language, along with a convenient set of functions to simplify web testing.

SST framework consists of:

• user actions and assertions (API) in Python
• test case loader (generates/compiles scripts to unittest cases)
• console test runner
• data parameterization/injection
• selectable output reports
• selectable browsers
• headless (xvfb) mode
• screenshots on errors

Test output can be displayed to the console, saved as an HTML report, or JUnit-compatible XML for compatibility with CI systems.

SST is free open source software (Apache Licensed). SST is primarily being developed on Linux, specifically Ubuntu. It should work fine on other platforms, but any issues (or even better - patches) should be reported on the Launchpad project:

• https://launchpad.net/selenium-simple-test

I just uploaded SST 0.1.0 to PyPI:

• http://pypi.python.org/pypi/sst

go ahead, give it a try:

• `$ [sudo] pip install sst`

documentation and more info:

• http://testutils.org/sst

a sample test script in SST:

from sst.actions import *

go_to('http://www.ubuntu.com/')
assert_title_contains('Ubuntu homepage')

Here is the development progress of SST (shown as a code_swarm visualization) over the past 8 months:

Special thanks to all the SST code committers so far:

• Danny Tamez
• Julien Funk
• Kenneth Koontz
• Leo Arias
• Lukasz Czyzykowkski
• Rick McBride
• Sidnei da Silva

Extra special thanks to SST's initial creator:

• Michael Foord

Happy New Year!

-Corey Goldberg