Managing log files with logrotate
Log Rolling
Regularly switching log files keeps them from filling up the filesystem, but the logrotate tool can do much more.
Linux log files do not take up much space. Even with systemd, many logs continue to be text files. All the same, if left unattended, log files would eventually fill their entire filesystem – and, if that filesystem is root, crash the system. To avoid this eventuality, Linux uses logrotate, a system of multiple logs that are switched regularly, either at a specified time or when the file size reaches the designated limit.
Logrotate is installed by default in every distribution of which I am aware. You can usually take for granted that logrotate is installed, but you can check by running the command ls /var/log
. If you see multiple files with numbered suffixes, then logrotate is installed. Alternatively, in some distributions, you can run less /var/lib/logrotate.status
to see the latest logrotate activity.
In fact, logrotate is required by a Linux system, because many standard applications use logs by default, including apache, aptitude, cups-daemon, and dpkg. Ordinarily, such logs exist as much for the system as the administrator, but they can be invaluable for troubleshooting. You may want to modify the default setups for a log or rotate a log for your own scripts.
For example, on my system with a solid state drive, I have a script with a log for fstrim, the utility run weekly to discard unused blocks and to keep the drive running at top efficiency. By setting up the fstrim script for logrotate, I can keep its size from ballooning out of control without any further action on my part.
Logrotate works with several files:
/etc/logrotate.conf
: The global configuration file./etc/logrotate.d
: A directory of logrotate settings for particular applications. Entries in this directory override the configuration in/etc/logrotate.conf
./etc/cron.daily/logrotate
: The scheduler in which an entry for logrotate is entered so that all the applications with an entry in/etc/logrotate.d
are processed. Usually, the scheduler is daily, but it can be hourly, monthly, or weekly instead.
The logs themselves are kept in /var/log
(Figure 1).
Note that, unlike many commands, the logrotate
command does not override the settings in /etc/logrorate.conf
or /etc/logrotate.d
. Instead, the command's options affect only how the command is run.
Setting Log Options
Values for log rotation can be set globally in /etc/logrotate.conf
(Figure 2) or in a separate file in /etc.logrotate.d
(Figure 3). Fields can be set in any order in the file, but if fields contradict each other, the one closest to the bottom of the field is used. Otherwise, fields can be in any order, with one per line, although the usual arrangement is to have global settings at the start of the file, and values that apply to specific logs contained within curly braces. For example, /etc/logrotate.d/
defines one log this way:
/var/log/apt/term.log { rotate 12 monthly compress missingok notifempty }
The same file, incidentally, defines another apt-related log in the same file. Most distributions further organize the settings by adding a comment above each field so that it is easily read. Field values can also be disabled by adding a hashtag (#) at the start of the line.
Regardless of where the log options are set, a couple of options must be set. Each set of values must have a regularity, such as monthly, weekly, or daily, corresponding to the cronjob in which it will run. Although not required, each set should also have a maximum number of rotations, which is set using rotate NUMBER
.
Alternatively, logs can be rotated whenever the current one reaches a current size with size SIZE
. Other options are maxsize SIZE
or minsize SIZE
. The size can be set using k
, M
, or G
(kilobytes, megabytes, gigabytes). Because logs are plain text, you can probably use kilobytes.
Regardless of which method is used to determine rotation, when a log is rotated, it is renamed by adding a number as a suffix, so that the current log becomes log.1
, and the previous one log.2
, and so on. When the maximum number of rotations is reached, the oldest log is deleted and replaced by the next oldest. If rotate is set to 0, then the older log is deleted, instead of being temporarily stored. If you prefer, with dateext
, older logs can have a date extension such as YYYYMMDD
, with the exact date format set using dateformat
. No matter what the naming method, there is no requirement about how many rotations to use, except that, the more important the information, the more rotations should generally be used.
Ordinarily, rotated logs are stored in the same directory as the current one – usually, /var/log
. However, you also have the option of using olddir Directory
to store a log rather than deleting it when its turn come. Another choice is to mail a log being rotated out of existence instead of renaming it, using mail ADDRESS
. If you prefer, use mailfirst
to mail the just-rotated file. If you want to be sure that no log is mailed, use nomail
instead.
By default, old logs are compressed using gzip
. However, you can store old logs without depression by using nocompress
. You can also use another command for archiving, such as tar
using compresscmd
, which requires a matching uncompresscmd
so that the logs can be properly accessed. With any compression method, compressionoptions
passes on to logrotate the options to use during compression. For example, when using gzip
, passing on the option -6
will maximize compression at the price of speed.
Still other options include missingok
, which ignores a missing log file and continues processing, and shred
and shredcycles NUMBER
, which overwrites deleted logs, making them unreadable. Further flexibility can be added to how a particular set of options is handled by adding scripts that begin with postrotate
, prerotate
, firstaction
, or lastaction
on a separate line, each of which ends with endscript.
Choosing Command Options
When logrotate runs, it works with all the logs defined in /etc/logrotate.conf
and /etc/logrotate.d/
. The options for the command are about how it runs, not how individual sets of logs are handled. If you mail any logs when they are rotated, use --mail (-mail) COMMAND
to change the email application from the default /usr/bin/mail -s
.
When you add new configuration entries, or if you have removed files manually instead of using logrotate, you may need to use --force (-f)
. In these situations or with any other troubleshooting, I suggest using --verbose (-v)
, especially when testing, so that you can be sure that the command and the configuration files are interacting in the way that you hope.
Setting Up a Cronjob
Although you might test your command as you set up logrotate, its main purpose is to add it as a cronjob so you can forget about log rotation. Typically, you will set it as a daily cronjob, running a command to confirm the existence of log rotation, then running logrotate itself, as shown in Figure 4. This command can be easily modified if you want to run logrotate at another frequency.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
NVIDIA Released Driver for Upcoming NVIDIA 560 GPU for Linux
Not only has NVIDIA released the driver for its upcoming CPU series, it's the first release that defaults to using open-source GPU kernel modules.
-
OpenMandriva Lx 24.07 Released
If you’re into rolling release Linux distributions, OpenMandriva ROME has a new snapshot with a new kernel.
-
Kernel 6.10 Available for General Usage
Linus Torvalds has released the 6.10 kernel and it includes significant performance increases for Intel Core hybrid systems and more.
-
TUXEDO Computers Releases InfinityBook Pro 14 Gen9 Laptop
Sporting either AMD or Intel CPUs, the TUXEDO InfinityBook Pro 14 is an extremely compact, lightweight, sturdy powerhouse.
-
Google Extends Support for Linux Kernels Used for Android
Because the LTS Linux kernel releases are so important to Android, Google has decided to extend the support period beyond that offered by the kernel development team.
-
Linux Mint 22 Stable Delayed
If you're anxious about getting your hands on the stable release of Linux Mint 22, it looks as if you're going to have to wait a bit longer.
-
Nitrux 3.5.1 Available for Install
The latest version of the immutable, systemd-free distribution includes an updated kernel and NVIDIA driver.
-
Debian 12.6 Released with Plenty of Bug Fixes and Updates
The sixth update to Debian "Bookworm" is all about security mitigations and making adjustments for some "serious problems."
-
Canonical Offers 12-Year LTS for Open Source Docker Images
Canonical is expanding its LTS offering to reach beyond the DEB packages with a new distro-less Docker image.
-
Plasma Desktop 6.1 Released with Several Enhancements
If you're a fan of Plasma Desktop, you should be excited about this new point release.