Lock down your logfiles with logrotate
Setting the Records Straight
The simple act of logging can create management and storage nightmares. Logrotate brings creative solutions to your logging needs.
When you are new to running servers and rightly worried about keeping downtime to a minimum, you inevitably find yourself facing a few problems – some predictable and others not so much. A particular aspect of systems that repeatedly causes consternation is logging and its concomitant storage.
To my mind, logging is one of the truly operational aspects of being a sys admin. Even with the best automation in the world, to get system logging right, you need to invest a little forethought along with very occasional housekeeping. After attempting to diagnose a few system problems, you soon come to realize that logs are absolutely essential. I'll reiterate that last statement by saying that they are not just important, they are truly key to keeping your servers functioning correctly.
Logfiles come in all different shapes and sizes and hold all kinds of information, both useful and relatively useless. Some logs are inconsequential if lost, whereas others are so critical to the operation of a server that their presence can mean the difference between a full server rebuild (sometimes adding several days of additional work to your busy week) or a five-minute health check.
With experience, you develop a level of trust in what your logs deliver and a degree of comfort in the level of detail you need to answer a number of questions relating to the kinds of problems you are often called on to investigate. How important is getting that level of detail correct? A few years ago, my least-favorite dialog box (on a certain popular GUI-based operating system) inconveniently popped up and reported the following: There has been an error.
No matter which way you squint, that dialog box isn't going to help much in solving what might be a life or death dilemma – in the sense of ongoing employment at least. The message I'm attempting to convey is: Logs are not only critical to the operation of your server, but they are also an aspect of your systems that needs more than just a flying blind, brief amount of consideration. Without a high level of logging detail, you are inevitably caught out unwittingly when presented with a large number of red herrings (i.e., clues) that lead you down the wrong path in your diagnosis.
Conversely, with a massive amount of detail your disks fill up so quickly that your remote server, at best, complains endlessly when trying to run its applications or, at worst, simply stops working, requiring expensive onsite engineers with hands-on access to recover it.
Doom-mongering aside, a Unix-flavored package called logrotate is so powerful that it swiftly takes care of many of the problems commonly associated with logging and storage without breaking a sweat. That said, it is still important to keep an eye on how your logs are behaving periodically, even with such a sophisticated tool in your toolkit.
Pause for Breath
Imagine a software package with functionality that lets you specify lots of well-considered parameters, including whether to send email after logfiles have reached a certain number, whether to compress logfiles to save disk space, and whether logs should be ignored or highlighted if they have no content (i.e., zero-byte files). The most important of all logrotate's features, however, is revealed by its name: The simple functionality of rotating logs is key to keeping your servers online.
Any additional bells and whistles won't be as useful to you as the simple act of purging old logfiles that are simply not useful any more to reduce disk space usage. Logrotate is so flexible that it can manage all of the above-mentioned tasks with the logfiles of almost any software package, making it a fantastic weapon in your sys admin arsenal.
The end result is peace of mind that your disks will never fill up, causing poor server performance. Coupled with the right level of logging (which is usually set within the configuration parameters of your applications and not within logrotate), you can, for all intents and purposes, strike logging off your daily housekeeping to-do list and relegate it to your monthly to-do list instead.
En Suite
To say the superhero that is logrotate is feature-filled is certainly no understatement. For a number of years now, logrotate has been so warmly embraced by the Linux community that many packages in the repositories ship conveniently with a ready-made logrotate profile that is dropped dutifully into the correct directory, with the path /etc/logrotate.d
.
The result is that, give or take some very infrequent tweaks, everything related to pruning and purging your critical logs is taken care of automatically for you. Additionally, I have seen some less mature packages include an optional profile, even if it's not dropped into place when the package is installed.
However, until you delve into the multiple configuration options, you might not realize exactly what you should be doing with your logs. Now that I've established why it's so important to keep logs ticking over correctly to prevent disaster, I'll run through some basic logging scenarios, as well as a few that might not have occurred to you.
Status Quo
My systems run my preferred flavor of Linux – namely, Debian on my servers and Ubuntu on my desktops. However, like so many Linux packages, logrotate is probably nearly identical to use across all the popular distros, barring a few file path changes. Logrotate likely already exists on your system, but you might want to check at this stage with:
# apt-get install logrotate
Before forging ahead, I'll briefly mention where to check the innards of the package. A "state" file remembers who did what and when, in terms of when it last checked, changed, or rotated a log. That file is /var/lib/logrotate/status
(Figure 1), which stores the last rotation recorded for each logfile by date.
Also, you don't have to worry about triggering logrotate, it simply runs as a cron job, sitting neatly inside the /etc/cron.daily/
directory. Thanks to cron's portability, then, you can simply move the 14 lines or so of configuration to another cron file and run it whenever you want.
The easiest way to dip your toe into the water with logrotate is to start looking at some of the bundled config files, which demonstrate how much of the hard work is already done for you by this well-written software package.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Halcyon Creates Anti-Ransomware Protection for Linux
As more Linux systems are targeted by ransomware, Halcyon is stepping up its protection.
-
Valve and Arch Linux Announce Collaboration
Valve and Arch have come together for two projects that will have a serious impact on the Linux distribution.
-
Hacker Successfully Runs Linux on a CPU from the Early ‘70s
From the office of "Look what I can do," Dmitry Grinberg was able to get Linux running on a processor that was created in 1971.
-
OSI and LPI Form Strategic Alliance
With a goal of strengthening Linux and open source communities, this new alliance aims to nurture the growth of more highly skilled professionals.
-
Fedora 41 Beta Available with Some Interesting Additions
If you're a Fedora fan, you'll be excited to hear the beta version of the latest release is now available for testing and includes plenty of updates.
-
AlmaLinux Unveils New Hardware Certification Process
The AlmaLinux Hardware Certification Program run by the Certification Special Interest Group (SIG) aims to ensure seamless compatibility between AlmaLinux and a wide range of hardware configurations.
-
Wind River Introduces eLxr Pro Linux Solution
eLxr Pro offers an end-to-end Linux solution backed by expert commercial support.
-
Juno Tab 3 Launches with Ubuntu 24.04
Anyone looking for a full-blown Linux tablet need look no further. Juno has released the Tab 3.
-
New KDE Slimbook Plasma Available for Preorder
Powered by an AMD Ryzen CPU, the latest KDE Slimbook laptop is powerful enough for local AI tasks.
-
Rhino Linux Announces Latest "Quick Update"
If you prefer your Linux distribution to be of the rolling type, Rhino Linux delivers a beautiful and reliable experience.