April 7, 2012

Python - Graphite: Storage and Visualization of Time-series Data

I'm doing some work with Graphite in Python. Here is a quick overview of what Graphite is...

Graphite provides real-time visualization and storage of numeric time-series data.


Graphite does two things:

  • Store numeric time-series data
  • Render graphs of this data on demand

Graphite consists of a storage backend and a web-based visualization frontend. Client applications send streams of numeric time-series data to the Graphite backend (called carbon), where it gets stored in fixed-size database files similar in design to RRD. The web frontend provides 2 distinct user interfaces for visualizing this data in graphs as well as a simple URL-based API for direct graph generation.

Graphite consists of 3 software components:

  • carbon - a Twisted daemon that listens for time-series data
  • whisper - a simple database library for storing time-series data (similar in design to RRD)
  • graphite webapp - A Django webapp that renders graphs on-demand using Cairo


esacteksab said...

How are you getting the data into Graphite? Have you decided on retention levels yet?

Corey Goldberg said...

I'm going to write a few more blog posts explaining my setup. I will be building some custom python clients to push data from various sources.

If you have any tips on best configuration of data retention, please let me know... I will be playing with that soon.

esacteksab said...

I've only started playing with Graphite recently myself. Retention is one of many thing still up in the air as I understand what it all means.