The sys admin's daily grind: RSyslog

Where To Next?

Article from Issue 91/2008
Author(s):

Well-used services write reams of log information to disk, which is not only bothersome from a storage perspective but also pushes grep and the usual group of statistics tools to their limits. Will hitching the syslog daemon up to a database help?

Who said length doesn't matter? My spam filters alone give me a 3GB logfile daily, which would be fine if I just needed the beast to check up on the occasional error. Because I need to extract a whole bunch of statistics about spam and virus threats from the file, grepping such enormous files takes ages, besides creating far too much I/O overhead.

RSyslog [1] took me a giant leap closer to finding a solution – it logs directly to a MySQL or PostgreSQL database, meaning that I can replace my grep commands with fast SQL statements. RSyslog is included with many distributions and is the default application in Fedora 8, for example. By default, my Ubuntu lab environment runs sysklogd, making it easy for me to switch to RSyslog. For the time being, I can even keep my old syslog.conf. For the most part, RSyslog's configuration file is the same as the legacy format, although it does support a couple of additional options for linking up with the database.

First I need to set up a database. To do so, I run the createDB.sql script, which is provided with the RSyslog package:

[...]

Use Express-Checkout link below to read the full article (PDF).

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Security Lessons: Rsyslog

    Might as well do it properly – rsyslog.

  • Charly's Column

    Using SQL to sift syslog data out of a database is an admittedly universal, but also fairly convoluted approach. phpLogCon, with its web interface, gives admins an easier option.

  • Security Lessons: Windows Logging

    Windows 7 is pretty good at logging, but what do you do with all those log files? We look at some monitoring tools that can help you get the most out your logging data.

  • Charly's Column: lsof

    The shorter a command, the longer the list of support parameters. This rule applies to lsof, one of Charly’s favorite commands.

  • Just for the Record

    If you run 25 Raspberry Pis at home, and an equal number of other IP devices, you might also think like Charly does when it comes to log management. The result is atomic technology and a logfile disk that is not permanently overloaded.

comments powered by Disqus
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters

Support Our Work

Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.

Learn More

News