Friday, October 4, 2013

Puppet has memory leaks

Ever since I setup puppet, I've noticed that the memory just keeps growing. Searching on the web, I found that puppet has memory leaks. Some suggestions were to run puppet using cron.

I thought about it but I didn't want to do it. At first I thought about having Fabric restart puppet once a month. In theory, that should work. Then, when I was ready to put it in, all the possible problems of doing this popped in my head. So, I didn't proceed. What next??

Well, I thought, since we use Nagios to monitor all of our machines anyway, why not monitor the puppet process, get a rough estimate of its memory utilization and have Nagios automatically restart puppet if it hits a certain number? And so, here it is....


#!/usr/bin/perl
# Author: Gou
# Purpose: Checks and restarts puppet if the memory is too high

use strict;

my $default;
my $pcount;
my @Results;
my $memResults;

$default=10;
chomp($pcount=`ps aux | grep puppetd | grep -v grep | wc -l`);

if ($pcount < 2) {
  @Results=split(" ",`ps aux | grep puppetd | grep -v grep`);
  $memResults=$Results[3];
  if ($memResults > $default) {
  `sudo /etc/init.d/puppet restart`;
  print "WARNING - Memory utilization of $memResults\% is too high, restarting puppet";
  exit 1;
  }
  else{
    print "OK - Memory utilization is $memResults\%";
  exit 0;
  }
}
else{
  print "CRITICAL - Found $pcount puppet processes";
  exit 2;
}