Managing log files with logrotate
Log Rolling
Regularly switching log files keeps them from filling up the filesystem, but the logrotate tool can do much more.
Linux log files do not take up much space. Even with systemd, many logs continue to be text files. All the same, if left unattended, log files would eventually fill their entire filesystem – and, if that filesystem is root, crash the system. To avoid this eventuality, Linux uses logrotate, a system of multiple logs that are switched regularly, either at a specified time or when the file size reaches the designated limit.
Logrotate is installed by default in every distribution of which I am aware. You can usually take for granted that logrotate is installed, but you can check by running the command ls /var/log
. If you see multiple files with numbered suffixes, then logrotate is installed. Alternatively, in some distributions, you can run less /var/lib/logrotate.status
to see the latest logrotate activity.
In fact, logrotate is required by a Linux system, because many standard applications use logs by default, including apache, aptitude, cups-daemon, and dpkg. Ordinarily, such logs exist as much for the system as the administrator, but they can be invaluable for troubleshooting. You may want to modify the default setups for a log or rotate a log for your own scripts.
For example, on my system with a solid state drive, I have a script with a log for fstrim, the utility run weekly to discard unused blocks and to keep the drive running at top efficiency. By setting up the fstrim script for logrotate, I can keep its size from ballooning out of control without any further action on my part.
Logrotate works with several files:
/etc/logrotate.conf
: The global configuration file./etc/logrotate.d
: A directory of logrotate settings for particular applications. Entries in this directory override the configuration in/etc/logrotate.conf
./etc/cron.daily/logrotate
: The scheduler in which an entry for logrotate is entered so that all the applications with an entry in/etc/logrotate.d
are processed. Usually, the scheduler is daily, but it can be hourly, monthly, or weekly instead.
The logs themselves are kept in /var/log
(Figure 1).
Note that, unlike many commands, the logrotate
command does not override the settings in /etc/logrorate.conf
or /etc/logrotate.d
. Instead, the command's options affect only how the command is run.
Setting Log Options
Values for log rotation can be set globally in /etc/logrotate.conf
(Figure 2) or in a separate file in /etc.logrotate.d
(Figure 3). Fields can be set in any order in the file, but if fields contradict each other, the one closest to the bottom of the field is used. Otherwise, fields can be in any order, with one per line, although the usual arrangement is to have global settings at the start of the file, and values that apply to specific logs contained within curly braces. For example, /etc/logrotate.d/
defines one log this way:
/var/log/apt/term.log { rotate 12 monthly compress missingok notifempty }
The same file, incidentally, defines another apt-related log in the same file. Most distributions further organize the settings by adding a comment above each field so that it is easily read. Field values can also be disabled by adding a hashtag (#) at the start of the line.
Regardless of where the log options are set, a couple of options must be set. Each set of values must have a regularity, such as monthly, weekly, or daily, corresponding to the cronjob in which it will run. Although not required, each set should also have a maximum number of rotations, which is set using rotate NUMBER
.
Alternatively, logs can be rotated whenever the current one reaches a current size with size SIZE
. Other options are maxsize SIZE
or minsize SIZE
. The size can be set using k
, M
, or G
(kilobytes, megabytes, gigabytes). Because logs are plain text, you can probably use kilobytes.
Regardless of which method is used to determine rotation, when a log is rotated, it is renamed by adding a number as a suffix, so that the current log becomes log.1
, and the previous one log.2
, and so on. When the maximum number of rotations is reached, the oldest log is deleted and replaced by the next oldest. If rotate is set to 0, then the older log is deleted, instead of being temporarily stored. If you prefer, with dateext
, older logs can have a date extension such as YYYYMMDD
, with the exact date format set using dateformat
. No matter what the naming method, there is no requirement about how many rotations to use, except that, the more important the information, the more rotations should generally be used.
Ordinarily, rotated logs are stored in the same directory as the current one – usually, /var/log
. However, you also have the option of using olddir Directory
to store a log rather than deleting it when its turn come. Another choice is to mail a log being rotated out of existence instead of renaming it, using mail ADDRESS
. If you prefer, use mailfirst
to mail the just-rotated file. If you want to be sure that no log is mailed, use nomail
instead.
By default, old logs are compressed using gzip
. However, you can store old logs without depression by using nocompress
. You can also use another command for archiving, such as tar
using compresscmd
, which requires a matching uncompresscmd
so that the logs can be properly accessed. With any compression method, compressionoptions
passes on to logrotate the options to use during compression. For example, when using gzip
, passing on the option -6
will maximize compression at the price of speed.
Still other options include missingok
, which ignores a missing log file and continues processing, and shred
and shredcycles NUMBER
, which overwrites deleted logs, making them unreadable. Further flexibility can be added to how a particular set of options is handled by adding scripts that begin with postrotate
, prerotate
, firstaction
, or lastaction
on a separate line, each of which ends with endscript.
Choosing Command Options
When logrotate runs, it works with all the logs defined in /etc/logrotate.conf
and /etc/logrotate.d/
. The options for the command are about how it runs, not how individual sets of logs are handled. If you mail any logs when they are rotated, use --mail (-mail) COMMAND
to change the email application from the default /usr/bin/mail -s
.
When you add new configuration entries, or if you have removed files manually instead of using logrotate, you may need to use --force (-f)
. In these situations or with any other troubleshooting, I suggest using --verbose (-v)
, especially when testing, so that you can be sure that the command and the configuration files are interacting in the way that you hope.
Setting Up a Cronjob
Although you might test your command as you set up logrotate, its main purpose is to add it as a cronjob so you can forget about log rotation. Typically, you will set it as a daily cronjob, running a command to confirm the existence of log rotation, then running logrotate itself, as shown in Figure 4. This command can be easily modified if you want to run logrotate at another frequency.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.