Command Line – BorgBackup

BorgBackup [1] was originally a fork of Attic [2], begun in 2015 by developers who wanted faster-paced development. Attic is no longer in development, although stable versions remain available in Debian and several other distributions. Meanwhile, BorgBackup has lived up to its mandate, not only enhancing the Attic code, but also gaining a reputation for its innovative approach. Although other compression and backup utilities have encryption, incremental, and remote options, what distinguishes BorgBackup is how it implements such features, making them more visible and easier to manipulate. For example, because an incremental backup is in a separate file from the original, it can be retrieved by itself. In many ways, BorgBackup is a modernization of long-time Linux archivers such as tar, gzip, and bzip2, offering increased efficiency and speed with its choice of compression formats and its own set of options. In addition, BorgBackup offers fine-tuned control of features, such as the compression level and the details stored in its logs, while using intelligent defaults that allow users to quickly get up and running.

BorgBackup's documentation [3] does warn that the application requires a reasonable amount of free space to operate. The exact amount, of course, depends on the size of the files it uses, but at least 2GB should be available. However, to ensure trouble-free backups, repositories should be created on filesystems with plenty of free space.

The Basics

Like an increasing number of modern command-line applications, BorgBackup's command structure consists of three parts: the basic command; a sub-command, such as create, extract, or delete; and the options and paths to a file. Many options can be used with more than one of the sub-commands and can be placed either after the sub-command or after the paths, although to avoid any confusion, users should probably use one position consistently.

Before archiving, BorgBackup requires the creation of a repository in which to store backups. This repository can be protected by a passphrase, as well as encrypted. The arrangement has the advantage of reducing the number of passphrases to remember, even though, technically, individual or additional passphrases would be more secure in some cases.

To create a repository, enter:

borg init  PATH

This command automatically creates a repository, prompting for a passphrase (Figure 1). BorgBackup does not prevent a weak passphrase. However, if you choose the option to display the password before continuing, it will give you the chance to reconsider your choice. The path can be relative or absolute, as well as local or remote and connected by SSH (ssh://PATH:2022/; USER@HOST:PATHcs). If necessary, a remote or external path can be activated using the mount sub-command, most of whose options are contained in the Filesystem in Userspace (FUSE) man pages.

Figure 1: To back up files, you must first create a repository. Viewing the new repository's passphrase can help you remember it or encourage you to reconsider a weak password.

In most cases, too, you may want to include the --encryption=repokey option, which stores the passphrase in the repository, or else the --encryption=keyfile option, which stores the passphrase in ~/.config.borg/keys. The keyfile option is most secure if your home directory is on a separate device from the repository. Later, the sub-command change-passphrase can be used to change the passphrase, although if the repository has been compromised, your data is more secure if you create an entirely new one.

In either case, the passphrase uses 256-bit AES encryption and is verified with HMAC-SHA256. The passphrase is stored in UTF-8 and should avoid any characters that have a specific meaning at the command line, such as a forward slash (/) or a regular expression like an asterisk (*).

After a repository is created, you can add files to it with the create sub-command:

borg create REPO::ARCHIVE SOURCES

Multiple sources can be specified in a space-separated list. You can create an incremental backup by specifying sources already archived, which will substantially reduce the amount of time required to update an archive. Once a repository and archives are created, you can use other sub-commands for basic functions (see Table 1).

Advanced Options for Creating Archives

When using the create sub-command, one of the most important settings is compression. By default, BorgBackup uses the LZ4 format, a fast choice, but one with a relatively low compression rate. If you have the disk space, you may prefer to specify --compression none (-C). Better control over compression can be had by specifying another format, each of which has its own scale for the level of compression: zstd (1-22), zlib (0-9), and lzma (0-9). In all of these scales, the lowest number gives the least compression but runs the quickest, and, if a compression level is not specified, BorgBackup uses a medium level. The format and compression level can be set using the option:

--compression FORMAT LEVEL

The preferable format and compression level is a matter for expert debate, but keep an eye on CPU activity and free disk space while running each format to help decide which one to use in a particular circumstance.

Unless the --verbose (-v) and --show-rc options are used, BorgBackup gives minimal output to avoid clutter. The output it gives is enough in routine circumstances and often includes helpful suggestions. However, at times you may need to look at the logfile for more detailed information. No logfile is written by default, but you can specify one as an environment setting (see "Automating the Command"), which may be advisable on a network or when dealing with any important files. You can set what is logged by creating an archive with one of four options:

• --debug: Logs everything in detail.
• --warning: Logs warning, error, and critical-level messages.
• --error: Logs error and critical-level messages.
• --critical: Logs only critical-level messages

While it might seem wisest to set the log information to --debug, too much information might make the log more difficult to read, so you should experiment to discover what level of information you actually need.

Other options are also available. The option --exclude PATH can be used to exclude items from an archive or else the backup can be limited to one filesystem with:

--one-file-system (-x)

the default 0077 permissions can be changed with:

--umask OCTAL-PERMISSIONS

The atime attribute, which records the last time a file was accessed, can be omitted with --noatime; the ctime attribute, which records the last time the file was changed, with --noctime; and the user and group identifiers can be restricted to numeric identifiers with --numeric-owner.

All these options can make the initially simple borg command considerably more complex, so, having set up a command, you might want to run:

--dry-run (-n)

to test your carefully crafted command before actually running it. Upon creating an archive, you should also run the check sub-command (Figure 3) to ensure that the archive is reliable. After all, the very worst time to find that an archive is unusable is when you need to extract it.

Figure 3: The check sub-command tests that an archive is usable.

Automating the Command

Another way to make using a complex command easier is to add the most commonly used options as environment variables using the export command [4]. The borg command will use the variables defined as part of the environment unless alternatives are specified. For example, BORG_REPO=PATH sets the location of the default repository, while BORG_PASSPHRASE=PASSPHRASE gives the passphrase for the default repository. Other possible environment variables store responses to warnings, such as

BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes

which responds to the warning, "Attempting to access a previously unknown unencrypted repository." As you might expect, none of the available environment variables will function if you use the sudo command to operate as another user, since the change of user is generally accompanied by a change in environment.

You can also use Bash scripts to automate backups and then use a crontab to run the script. BorgBackup's Quick Start provides an example of such a script that can be modified to suit your system [5].