File syncing with unison
Command Line – unison
Unison is a handy tool for file syncing, backups, and merging. To get the most out of unison, however, you need to invest time perfecting your preference files to meet your needs.
Although unison
has been around for years, chances are you have never heard of unison
. For one thing, rsync
and ssh
tend to be the default commands for file syncing. For another, unison
's documentation is maddeningly incomplete. In the absence of man or info files or usage instructions from the creators, the existing help focuses on building a command to use at the prompt – and, considering that unison
has 88 options, that is not a very attractive option, even without a command history, especially if you have several sets of files with which you regularly work. So far as I can see, the only distribution that mentions how to simplify the use of unison
with preference files is Arch Linux, and even it is incomplete – as is the original documentation that Arch references. Yet once preference files are created, the number of on-the-fly options is greatly reduced, and unison
becomes a handy tool for file syncing, backups, and even merging files.
unison
is a shell for the rsync
protocol; it also calls upon external programs like ssh
and diff
for its operations. It works with two sets of directories and files, updating one so that the two sets match. Its only drawback is that, if you are syncing in separate machines, each must have the same version of unison
installed. The basic command structure is:
unison PATH1 PATH2
Synchronization is two-way: The most recent version of a file in either directory will overwrite the version in the other. Similarly, any file that exists only in one directory will be copied to the other. The resulting output can have three parts:
A standard warning: This warning indicates that this is the first time you have run a synchronization or that archive files do not exist (Figure 1). Scroll past the warning to attempt the synchronization.
A list of proposed changes: Each must be confirmed unless the
-auto
or-batch
option is used to accept everything automatically. You must enter y (yes) to continue (Figure 2).A summary of operations: Arrows to the left show that a file is being transferred from the first root, while an arrow to the right shows a file transfer from the second root. At the end of all operations, a summary displays (Figure 3).
This basic structure is rapidly complicated by options. All unison
's options are prefixed by a single hyphen, with a name that is usually self-explanatory – a useful feature when dealing with so many options. To further simplify matters, options share identical or at least similar names with the fields in a preference file. However, once a preference file is debugged, no root needs to be specified at the prompt. Instead, the structure becomes:
unison PREFERENCE-FILE
Moreover, the only options you need to add are ones that affect the command's operation, which usually reduces the number of options to add. Often, no options need to be added to the command at all. In addition, preference files can be written to draw on a common file, making the creation of new preference files extremely easy.
Writing Preference Files
Preference files are the sanest way to use unison
. Preferences are stored in .unison
in your home directory, with names that end with a .prf
extension. Typically, you would have a default.prf
that is used when a preference is not specified with the command, and perhaps other preference files whose names are a description of their contents.
To make a profile, follow these four basic steps using commented lines to help keep track of the sections:
- Optionally, begin with
label=
, followed by a description of what contents the preference file is designed to sync. This field has no effect on howunison
interacts with the preference file, being intended only for human convenience. You could add a commented line instead. List the first root directory. For example, for user
bb
, it might be:#Root (source) root= /home/bb
List the second root directory to which files will be synchronized. On a separate line, you can also add any
ssh
argument, such as the port to use and the option to run in quiet mode. For example:#Target Directory root=ssh://nanday.com//home/bb/backup sshargs=-p4000 sshargs=-q
List, one per line, the subdirectories and files of the root or source directory to include in the operation. For instance:
#Directories and Files to include path=Projects path=Fonts path=Downloads path=to-do.txt
These first three sections (steps 2-4 and optionally step 1), in this order, must be in a preference file. Below them, you can add other sections:
List the files to ignore. Regular expressions may be useful in this section:
#Files to Ignore ignore=Virtualbox VMS ignore=*.tmp
As files are synced in the second root directory, they will be overwritten. As a precaution, you may want to back up the original files in the second root directory to another location, using these fields. If
backuplocation
is specified aslocal
, then backup files are stored in the second root directory; if the specification iscentral
, then backup files are stored in another directory that is specified inbackupdir
.Backups use the naming convention
.bak.VERSION.FILENAME
. The version begins with1
for the most recent and can be limited in number by using themaxbackups
field. The filename is the name of the original file, including any extension. This convention can be overridden using thebackupprefix
andbackupsuffix
fields, one of which must contain the version. For example:#Backups backuplocation=central backupdir=/home/bb/backups backup=$VERSION. backupprefix=backup backupsuffix= maxbackups=4
unison
can call on external commands likediff
anddiff -3
with the-diff
option and merge files using-merge PATTERN
. However, for either option to work, the path to the external command and any options to use with the command must be specified in a preference file. For instance:Diffs and merge diff=diff -y --suppress-common-lines merge = diff3 -m CURRENT1 OLD CURRENT2 > NEW
If you are syncing files on different servers or physical devices, you may want to log each use of
unison
. At the bottom of the preference file, add:#Log log=true
The logfile is located in the
.unison
directory.Advanced users can save complexity by using preferences another way:
In
default.prf
, include only the line:include common
- Create a file called
common
(notcommon.prf
). As in the first example,common
will define the two root directories. It must not contain anypath=
fields but may includeignore=
andbackup=
fields. Create preference files that contain only
path=
fields for backups, plus the statementinclude common
:#Directories to include path=Projects path=Fonts path=Downloads include common
Other than the directories to include, these stripped down preference files will take their characteristics from common
. Note that if you run unison
without specifying a preference file when using a common
file, then all subdirectories will be backed up.
Whether you use basic preference files or set up preference files with a common
file, the command is simplified to:
unison PREFERENCE-FILE
However, at least part of the time, you may still want or need to add other options. To give one obvious example, if you are syncing to a flash drive, then chances are you will want to add the -fat
option because many Linux users format flash drives to use FAT32.
Desktop vs. Command Line
unison
also supports a GTK+ graphical interface. This interface is often a separate package from unison
. In Debian, it is called unison-gtk
.
With unison-gtk
, you can create basic preference files using the included wizard. However, if you want to list files to ignore, or any other more advanced feature, you will have to open the preference file in a text editor. In particular, you might want to add keyboard shortcuts in the preference file. For example, the lines:
#Downloads Backup path=Downloads batch=true key=1 include common
This entry will customize unison
so that, when the GUI is open, pressing the 1 key will signal unison
to automatically sync any updates found in the Downloads
directory, using the information found in the common
file.
However, although unison-gtk
is convenient for average users, there may still be many times when you want to run unison
from the command line to take advantage of some of the options. Despite the availability of unison-gtk
, taking advantage of the prompt still requires a terminal.
Entering Options from the Prompt
Using preferences usually means that no options need to be entered at the prompt. Those that might be occasionally needed are normally general ones about how unison
operates, rather than about content. If manually entered options are used, they will override those specified in the preference file.
For example, when the output lists changed files, -sortbysize
, -sortfirst PATTERN
, or -sortnewfirst
will change the order of items on the list. You might also choose -auto
to accept non-conflicting default options without an interactive prompt. Alternatively, you might specify that you are prompted before deleting large numbers of files with -confirmbigdel
or -confirmmerge
before merging files. You might also add -links
to assure the copying of symbolic links.
If one root is on an external drive formatted with FAT32 – a common occurence in Linux – you will need to add the -fat
option. Similarly, if you are making a backup, you will not want to confirm each operation, so you use the -auto
option to disable confirmations.
Generally, the more you use unison
and create preference files to meet your needs, the less likely you are to need to add options manually. If you find yourself adding the same options all the time, check to see if they can be added to the preference file instead. Your use of unison
will be far easier if you can.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Latest Cinnamon Desktop Releases with a Bold New Look
Just in time for the holidays, the developer of the Cinnamon desktop has shipped a new release to help spice up your eggnog with new features and a new look.
-
Armbian 24.11 Released with Expanded Hardware Support
If you've been waiting for Armbian to support OrangePi 5 Max and Radxa ROCK 5B+, the wait is over.
-
SUSE Renames Several Products for Better Name Recognition
SUSE has been a very powerful player in the European market, but it knows it must branch out to gain serious traction. Will a name change do the trick?
-
ESET Discovers New Linux Malware
WolfsBane is an all-in-one malware that has hit the Linux operating system and includes a dropper, a launcher, and a backdoor.
-
New Linux Kernel Patch Allows Forcing a CPU Mitigation
Even when CPU mitigations can consume precious CPU cycles, it might not be a bad idea to allow users to enable them, even if your machine isn't vulnerable.
-
Red Hat Enterprise Linux 9.5 Released
Notify your friends, loved ones, and colleagues that the latest version of RHEL is available with plenty of enhancements.
-
Linux Sees Massive Performance Increase from a Single Line of Code
With one line of code, Intel was able to increase the performance of the Linux kernel by 4,000 percent.
-
Fedora KDE Approved as an Official Spin
If you prefer the Plasma desktop environment and the Fedora distribution, you're in luck because there's now an official spin that is listed on the same level as the Fedora Workstation edition.
-
New Steam Client Ups the Ante for Linux
The latest release from Steam has some pretty cool tricks up its sleeve.
-
Gnome OS Transitioning Toward a General-Purpose Distro
If you're looking for the perfectly vanilla take on the Gnome desktop, Gnome OS might be for you.