Log2Ram and frontail

Charly's Column – Log2Ram and frontail

Article from Issue 226/2019
Author(s):

If you run 25 Raspberry Pis at home, and an equal number of other IP devices, you might also think like Charly does when it comes to log management. The result is atomic technology and a logfile disk that is not permanently overloaded.

From time to time, I use nmap -sP 10.0.0.1-254 to check how many IP devices are online in my home network. There are now more than 50, half of them Raspberry Pis. The need for a central syslog server is slowly growing. An old miniature PC with an Intel Atom, which I retrofitted with an SSD, is the designated candidate for this permanent task. The syslog server comes courtesy of the standard rsyslogd. In its configuration file (/etc/rsyslog.conf), the following lines ensure that the server can receive syslog data from other hosts via UDP and TCP:

$ModLoad imudp
$UDPServerRun 514
$ModLoad imtcp
$InputTCPServerRun 514

On the other machines, I added an entry of *.* @10.0.0.254 to rsyslog.conf so that they all send their log data to the server on 10.0.0.254.

However, the incoming syslog messages generate huge numbers of writes, and I'm worried about the SSD service life. Enter Log2Ram [1] stage left. It creates a RAM disk on /var/log to which the central rsyslogd writes all the incoming data. Once an hour, the collected data are written to disk in one fell swoop.

Need to Talk

I installed Log2Ram by running the following command line on the log server:

git clone https://github.com/azlux/log2ram

I then changed to the directory created in the last step and executed the install.sh script. At first the installation failed because the Mailutils package was missing, and Log2Ram insists on the ability to mail to the admin in case of problems.

Also the size of the RAM disk, 40MB by default, was too small for my setup, but this can be adapted with a manual edit of the configuration file.

Now I just have one more wish: I don't want to be restricted to viewing the logs with tail -f on the log server console, instead I also want to inject them into a web page, just in case I feel the urge to inspect the files while I'm on the road. A small tool by the name of frontail [2] helps me do exactly this. It is based on Node.js, so you need to install the npm installer. You then install frontail and launch it like this:

npm i frontail -g
frontail /var/log/syslog

This starts a small web server on port 9001. Now, when I open the page in a web browser, I'm welcomed by the syslog (Figure 1). With just a little manual intervention, I can enjoy the view and an SSD that should survive for a couple of years.

Figure 1: frontail opens a viewing window into the log bucket.

The Author

Charly Kühnast manages Unix systems in the data center in the Lower Rhine region of Germany. His responsibilities include ensuring the security and availability of firewalls and the DMZ.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Charly's Column: lsof

    The shorter a command, the longer the list of support parameters. This rule applies to lsof, one of Charly’s favorite commands.

  • Charly's Column

    Using SQL to sift syslog data out of a database is an admittedly universal, but also fairly convoluted approach. phpLogCon, with its web interface, gives admins an easier option.

  • Charly's Column

    Users log on to services such as SSH, ftp, SASL, POP3, IMAP, Apache htaccess, and many more using their names and passwords. These popular access mechanisms are a potential target for brute-force attacks. An attentive bouncer will keep dictionary attacks at bay.

  • darkstat

    Thanks to its minimal footprint, 20-year-old darkstat hardly generates any noticeable load even on low-powered systems, making it the perfect monitoring tool for Charly's home utility room.

  • Charly's Column

    Well-used services write reams of log information to disk, which is not only bothersome from a storage perspective but also pushes grep and the usual group of statistics tools to their limits. Will hitching the syslog daemon up to a database help?

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95

News