Newfound Love

A few years ago, it seemed like not much could change with free monitoring systems. Despite its many weaknesses, Nagios had asserted itself as the untiring top dog, and the alternative solutions barely provided any new approaches. On the other hand, infrastructure growth accelerated by the cloud pushed the majority of monitoring systems to their performance limits even more quickly. The prospect of more fun with server monitoring seemed limited.

Then a light appeared rather unexpectedly at the end of the tunnel. It was 2012, and a Swedish system administrator called Ulf Mansson was so impressed by a new monitoring software that he circulated the hashtag #monitoringlove. At the DevOps Days in Rome, he related in detail how switching to this new software had put the fun back in his Ops team's work. Enthusiasm spread through the audience, and as my continuously growing hosting infrastructure was overtaxing Nagios (and my nerves) at the time, I was more than curious about what kind of system it was. The name of the project? Sensu [1].

About Sensu

Sensu enjoys the #monitoringlove of many Ops teams around the world. The Sensu program itself is written in Ruby, and its configuration files use the established JSON format. Sensu's architecture is very scalable, meaning you can monitor hundreds or even thousands of servers.

You can program Sensu plugins in nearly any language, and Sensu uses the well-known Nagios format for the check results, so you can reuse your existing Nagios checks with Sensu. Sensu also offers a large number of interfaces to external services, and you'll find that the Sensu community is very active and helpful.

Sensu is an open-core project; in other words, the core software is published under a free license, but that core also forms the basis for a proprietary product with additional features. Sean Porter initially developed Sensu as a part-time project at Sonian; Heavy Water Operations now pays him to devote 100 percent of his time to Sensu. Sensu Core is freely distributed under the MIT license.

The commercial product Sensu Enterprise expands on the Sensu Core underpinnings. Sensu Enterprise uses JRuby as a run-time environment that promises to improve processing performance. Other advantages of the commercial version include additional functions, such as the ability to convert metrics, easy integration of external services, and direct technical support from the developers at Heavy Water Operations.

If you are looking for free help with Sensu, you will find a very helpful developer community, which you can access via the mailing lists and the IRC channel #sensu on Freenode.

Architecture

Sensu is known for remarkably high performance and scalability. The key factor is the architecture (Figure 1). Sensu relies on a distributed architecture instead of packing the various components into a monolithic framework, where interfaces could quickly become a bottleneck.

Figure 1: The Sensu architecture overview.

Like any monitoring system, Sensu must initiate monitoring checks, process those results, and then perform actions, such as sending text messages or recording log entries. Sensu's modular manner means that the system controls these processes, so it is easy to understand why Sensu is sometimes called the "Monitoring Router."

The RabbitMQ messaging system supports asynchronous communication between system components on a Sensu machine. RabbitMQ always acts as the link for all processes in Sensu. A processing backlog on one component does not necessarily slow down all the other components.

The Sensu server simply publishes the request to Sensu clients to perform a specific check on the bus. Within a short time, each addressed client will comply with this request and will, in turn, publish the check results on the bus. It is now the server's turn again to process the check results and derive events from them. Event handlers ultimately implement these events as actions – for example, text message alerts or messages on the Ops team chat channel.

The Sensu API provides a REST-like interface to retrieve information about all monitored machines, their checks, and events from external sources. Using the API, it is possible to initiate checks and also prevent certain checks from being processed. The API can also acknowledge or delete alerts. All Sensu users will install at least one service that uses the API – namely, a website for interactively operating the system. Sensu's modular structure also allows for alternative user interface options, such as the sensu-admin tool, which is available at GitHub [2].

Server Installation

The Sensu monitoring tool is very easy to install. Sensu comes in the form of an omnibus package with all the necessary components; the various components install in a separate location on the system so that the dependencies related to the Ruby environment and all external libraries do not create a maintenance nightmare. A minimum Sensu installation is limited to adding the Sensu package sources and then running one of:

apt-get install sensu
yum install sensu

Only RabbitMQ and Redis are not included in the Sensu omnibus. They can be installed easily from the Linux distribution packages.

Startup scripts for /etc/init.d, which are enabled in the usual way, are included in the Sensu package. Scripts for other init alternatives are in /usr/share/sensu. Listing 1 shows a minimal server configuration that gives Sensu access to the necessary components.

01 ```Json
02 {
03   "Rabbit MQ": {
04     "ssl": {
05       "private_key_file": "/etc/sensu/ssl/client_key.pem",
06       "cert_chain_file": "/etc/sensu/ssl/client_cert.pem"
07     },
08     "port": 5671,
09     "host": "localhost",
10     "user": "sensu",
11     "password": "<mypass>",
12     "vhost": "/sensu"
13   },
14   "redis": {
15     "host": "localhost",
16     "port": 6379
17   },
18   "api": {
19     "host": "localhost",
20     "port": 4567
21   },
22   "dashboard": {
23     "host": "localhost",
24     "port": 8080,
25     "user": "admin",
26     "password": "<secret>"
27   }
28 }
29 ```

Anyone operating automated configuration management can rely on a Chef cookbook [3] or a Puppet module [4]; both modules are maintained directly by the Sensu team.