After decades of using Linux, I often imagine that I know my way around. Yet, humblingly, sometimes it can still surprise me. The latest example came with the sudden discovery that I could no longer install or delete packages, although applications continued to run normally. My search for a solution eventually led me to a full apt-cache archives, the directory in which removed packages are stored in case they are wanted. But let me start with the symptoms first so others can save time by going directly to ways to fix the problem.

I had never had such a problem before, so I was unprepared when it struck. In the last eight months, however, I had been exploring and deleting packages more often than usual, unknowingly setting up the condition that would cause the problem to strike. The first sign was a Flatpak package that suddenly refused to start. There was no notification of a newer version of the package, and at first I put the problem down to some vagary of Flatpak. To my bewilderment, I could not delete the package. Stranger still, I could not delete any DEBs, either. Instead, running apt-get dist-upgrade, alone of all the basic actions attempted, produced a "no space left" message. Yet typical productivity apps such as Firefox and LibreOffice continued to run as usual.

That was a clue, had I realized it at the time, but the message had misled me to the conclusion that the system as a whole had a memory problem. Yet managing packages is far from the most intensive action on a Debian installation, and the free command showed that the installed RAM had barely been touched, let alone the swap partition. Perhaps my use of six filesystems was the problem? Either /tmp or /var, the two most changeable partitions, could have filled without me noticing? I was quite prepared to crop a few gigabytes from the /home partition, the largest of my filesystems, but GNU Parted showed plenty of space remained on all of them. Clearly, the problem was not system-wide, but what was it then? I began searching through /var for deletable files, and there I discovered Apt's cache.

Cleaning the Apt Cache

The Apt cache is a relatively little-known part of Debian package management. Located in /var/cache/apt/archives, its purpose is to store old packages in case they are needed (Figure 1). Because dpkg generally looks for the more recent version of a package or the repository pinned during the addition of a package, the cache is easy to overlook. Users have to deliberately seek it out, and for many, there may be little reason to do so. On many systems, the archives continues to work quietly in the background, unnoticed until it is full and the installation no longer has the space to perform as it should. In fact, an installation may go its entire life span with the cache ever becoming full.

Figure 1: Apt archives contains deleted packages, or at least a list of them.

This obscurity explains why, like many users, I was caught by surprise when it filled. The obscure error message does not help, either, because nothing indicates that the space that is full is a specific part of the system, rather than the whole. The problem can be avoided if you always purge discard files, rather simply remove them, but it is easy to forget. Fortunately, there are at least four ways to restore functionality and prevent the problem from happening again.

When the problem occurs, the quickest course of action may be to create free space. If the filesystem that the /var directory is on is full, you can used Parted or GParted to increase the size of the filesystem, assuming that you have enough free space on other partitions. Alternatively, you can delete files on the partition. The /var directory contains several directories whose content can often be safely deleted, such as ./lost+found, ./logs, /mail, ./spool, and .tmp (Figure 2). Generally, though, you should check what you are deleting first, and avoid deleting files in ./cache, where more permanent directories such as ./fontconfig or ./dictionaries-common are stored. Provided you are cautious, any future inconvenience is likely to be minor.

Figure 2: The /var directory includes expired files that can be deleted to free up space, such as logs or spools.

However, to be completely safe, you can just delete the files in /var/cache/apt/archives. At worst, deleting the files in the archives will be a minor inconvenience, because Debian maintains an old stable repository that you an easily enable should you need to restore an older version of a package. Deleting the contents of the archives alone should restore the functionality of package management, although you may need to repeat the process in the future.

A more permanent solution is to regularly run Apt's housekeeping subcommands:

• apt-get purge PACKAGE: Removes all traces of a package from the system, including in the archive. This does a more thorough job of removal than apt-get autoremove.
• apt-get autoclean: Removes obsolete packages (Figure 3). However, deleted current packages are not deleted.
• apt-get clean: Removes install scripts and all packages in /var/cache/apt/archives.

Figure 3: The subcommand apt-get autoclean removes obsolete packages.

The easiest way to use these subcommands is to set up a cron job to run them as part of a regular maintenance routine. Set them up to run weekly or even daily, and you are unlikely to have full archives ever again.

Yet another solution is to customize how the cache is handled in a configuration file. If /etc/apt/apt.conf.d/99local-archive does not already exist, create it. On some Debian derivatives, you may need to use /etc/apt/apt.conf instead, especially with older releases. Either way, then add one of the following three options to the file:

• APT::Periodic::MaxSize "SIZE MiB"
• APT::Periodic::MaxAge "DAYS"
• APT::Periodic::MinAge "DAYS"

Of the three, MaxAge is probably the most useful, providing the same results as a cron job for apt-get clean. Note, though, that if you set MaxSize and the archives are already larger, some files will be deleted, starting with the largest files first. Whenever the archives' contents are deleted, the list of deleted packages remains in case you need to search, but the files do not.

A Relic of Memory Shortage

Setting a size limit for the package archives probably made sense in the days when memory was scarce. Yet now, in the five years or less of the average computer's working life, users are far less likely to be concerned with conserving memory. I have to wonder why a configuration setting is not installed by default, or perhaps an archive rotation, such as logrotate for logs, which switches logs as one is full, and then deletes the first file when the last one is full. As things are, solutions are obscure when a problem suddenly appears and an unnecessary complication occurs.