A Munin plugin to track http response times.

A Munin plugin to track http response times.

After two years of abandonment, I've finally managed to give some love to the server running this site. After all the trouble, I wanted to have a good monitoring setup, so I can evaluate the changes and how they're performing.

Munin has been running for while, but I had never tweaked it. Now, after running nginx and memcahed, I wanted to monitor those as well.

Turns out that Munin's plugin system is easy to extend. But it takes a while to understand how it all comes together: the various jobs, rdd integration, updates, permissions. Specially the autocongif and wildcards plugins, while a neat idea do make things more magical and harder to track.

I wanted to monitor is the total performance of the application. Not how apache, nginx or mysql are doing. Nor only disk or the bandwidth but the final response times, to measure how the entire stack is doing.

External tools, such as Pingdom and Monitor.us can handle it, but then you're also dealing with the internet connection, network conditions interfere with the measurements.

So I've hacked a quick plugin for Munin, that will measure response times for the urls you wish to monitor. This is a how it looks: The Http Response munin plugin graph.

In this particular case, I am monitoring the home url and the same one with a query string. That way, I can keep tabs on how the site is performing with memcached on and off.

Munin plugins are ran through the shell: they can be anything a perl script, c, python, anything else really. This plugin is written in python. It does require httplib2 though. I've tested this in python 2.5 and 2.4 (thanks for the patch Hermann Kase ) on linux, but it should really run on other setups without issues.

I am putting this up on github, as future updates will be easier to track. Installing is pretty easy, just copy the httpresonsetime file to where plugins are located (/etc/munin/plugins on ubuntu), chmod it appropriately and add the urls you need to monitor on the config file:

1
2
[httpresponsetime]
    env.urls http://www.example.com,http://www.othersite.com/somewhere

Restart the munin-node daemon, and you're all set.

Of course, just as I finished writing this, I found a similar plugin at the munin exchange.It takes a different approach though, as it allows for one url only to be monitored, and that url is set on the plugin file itself. The solution above seems cleaner.

If anyone else finds this useful, I'll put it up on munin exchange.

getting a third opinion

1.
Hessu says at

Hi,

If the URL contains a '-' character, it needs to be replaced with a '_' too, to make the munin plugin configuration valid:

parsed = urlparse.urlsplit (url)

  • return ("%s_%s_%s" % (parsed[1] , parsed[2], parsed[3])).replace("/", "_").replace(".", "_")

  • return ("%s_%s_%s" % (parsed[1] , parsed[2], parsed[3])).replace("/", "_").replace(".", "_").replace("-", "_")

Otherwise, nice work, thank you.

2.
Arthur Debert says at

Hi Hessu.

Thanks for the heads up, just committed your fix to the master branch.

Cheers

4.
Arthur Debert says at

Hi Alex.

Looking from your traceback, it seems your're running python 3.0. (you can check by running $python --version .

Python 3.0 is definitely not supported at this point (too different, will require two code bases), and it's not worth it. Check your system for python2.6 and set that on the shebang, should work.

Regards

6.
Arthur Debert says at

Hi Alex.

This should work on 2.4 (I don't have an install to double check right now).

The odd thing is that your trace back suggests that you are running the httplib2 for python 3.0 (that really won't work). Can you try to install version 0.5 ?

Regards

8.
Sachin Rekhi says at

Great Article!

Was looking for exactly this and you made my life easy.

I also love that you did it in Python. I was assuming that most of the plugins were written in Perl, since that's the platform's language, but after seeing this in Python I'm thinking about writing some of my own!

Thanks!

posted by
Arthur Debert

on

Tagged with:

8 comments so far.
Say something

Other months availabe in 2009

The complete blog archive

The latest entries

Subscribe to comments on this entry: rss feed

Cloud me:
  • Dane in The Web does not need flash. on

    To me, the key point in Flash's favor is productivity in creating rich experiences. Yes, javascript CAN create equivalent experiences, but it's a comparatively much more difficult process that requires ...

  • leef in The Web does not need flash. on

    Flash as a tool to develop cross platform applications is what I think is its future. And if it can also serve as a functional medium for users to use ...

  • Jesse in The Web does not need flash. on

    Despite the advances in Javascript, in many ways it is only catching up with actionscript, and while it is leading in some areas, javascript still needs a client. Whatever technology ...

  • Steven Wei in The Web does not need flash. on

    Very nice post. Your analysis was very thorough and I basically agree with all of your points.

    I have spent a couple years programming in Flash and have since moved ...

  • Steven Wei in The Web does not need flash. on

    Very nice post. Your analysis was very thorough and I basically agree with all of your points.

    I have spent a couple years programming in Flash and have since moved ...

Feeds: Entries rss feed Linksrss feed Worksrss feed

A Django joint. hosted on Slicehost