July 2, 2008

Twitter API vs. Yahoo Web Services - Performance and Reliability

I've noticed the Twitter API is very slow and unreliable. Requests often take 10+ seconds to return. Sometimes they timeout with no response at all.

I put together some monitoring tools to see exactly how slow and unreliable the API really is (when it is even available!). This is obviously not a very scientific test; just a ballpark idea of performance. The monitoring tools are built with Python and RRDTool. To start with, I used a modified version of my rrdpy suite of tools (http://code.google.com/p/rrdpy).

I ran a test for 2 hours, hitting the API's Test Method every 30 seconds. The tools generated the following time-series graph of server response times:

The average response was 2.53 seconds, and we see a *lot* of variance in the data samples.


In comparison, here is the same test run against the Yahoo Search REST API:

Notice the small variance and the average server response time of 0.186 secs!


I would be interested in running these tests again in a few months to see if Twitter's API has become faster and more reliable.

4 comments:

Anonymous said...

Could you share your test script? I started learning rrd and have a good use of your script for own experiments.

Corey Goldberg said...

The script I used is this one (slightly modified) http://code.google.com/p/rrdpy/source/browse/trunk/rrd_feeder_http.py

check out:
http://code.google.com/p/rrdpy
for details

Anonymous said...

Wonderful! Thank you very much, I'll check it out...

Anonymous said...

Perhaps you should show the two graphs on the same scale to better emphasize your point. Looking at them quickly, it almost looks like identical statistics.