File syncing with unison
Command Line – unison

© Photo by pan xiaozhen on Unsplash
Unison is a handy tool for file syncing, backups, and merging. To get the most out of unison, however, you need to invest time perfecting your preference files to meet your needs.
Although unison
has been around for years, chances are you have never heard of unison
. For one thing, rsync
and ssh
tend to be the default commands for file syncing. For another, unison
's documentation is maddeningly incomplete. In the absence of man or info files or usage instructions from the creators, the existing help focuses on building a command to use at the prompt – and, considering that unison
has 88 options, that is not a very attractive option, even without a command history, especially if you have several sets of files with which you regularly work. So far as I can see, the only distribution that mentions how to simplify the use of unison
with preference files is Arch Linux, and even it is incomplete – as is the original documentation that Arch references. Yet once preference files are created, the number of on-the-fly options is greatly reduced, and unison
becomes a handy tool for file syncing, backups, and even merging files.
unison
is a shell for the rsync
protocol; it also calls upon external programs like ssh
and diff
for its operations. It works with two sets of directories and files, updating one so that the two sets match. Its only drawback is that, if you are syncing in separate machines, each must have the same version of unison
installed. The basic command structure is:
unison PATH1 PATH2
Synchronization is two-way: The most recent version of a file in either directory will overwrite the version in the other. Similarly, any file that exists only in one directory will be copied to the other. The resulting output can have three parts:
A standard warning: This warning indicates that this is the first time you have run a synchronization or that archive files do not exist (Figure 1). Scroll past the warning to attempt the synchronization.
A list of proposed changes: Each must be confirmed unless the
-auto
or-batch
option is used to accept everything automatically. You must enter y (yes) to continue (Figure 2).A summary of operations: Arrows to the left show that a file is being transferred from the first root, while an arrow to the right shows a file transfer from the second root. At the end of all operations, a summary displays (Figure 3).
This basic structure is rapidly complicated by options. All unison
's options are prefixed by a single hyphen, with a name that is usually self-explanatory – a useful feature when dealing with so many options. To further simplify matters, options share identical or at least similar names with the fields in a preference file. However, once a preference file is debugged, no root needs to be specified at the prompt. Instead, the structure becomes:
unison PREFERENCE-FILE
Moreover, the only options you need to add are ones that affect the command's operation, which usually reduces the number of options to add. Often, no options need to be added to the command at all. In addition, preference files can be written to draw on a common file, making the creation of new preference files extremely easy.
Writing Preference Files
Preference files are the sanest way to use unison
. Preferences are stored in .unison
in your home directory, with names that end with a .prf
extension. Typically, you would have a default.prf
that is used when a preference is not specified with the command, and perhaps other preference files whose names are a description of their contents.
To make a profile, follow these four basic steps using commented lines to help keep track of the sections:
- Optionally, begin with
label=
, followed by a description of what contents the preference file is designed to sync. This field has no effect on howunison
interacts with the preference file, being intended only for human convenience. You could add a commented line instead. List the first root directory. For example, for user
bb
, it might be:#Root (source) root= /home/bb
List the second root directory to which files will be synchronized. On a separate line, you can also add any
ssh
argument, such as the port to use and the option to run in quiet mode. For example:#Target Directory root=ssh://nanday.com//home/bb/backup sshargs=-p4000 sshargs=-q
List, one per line, the subdirectories and files of the root or source directory to include in the operation. For instance:
#Directories and Files to include path=Projects path=Fonts path=Downloads path=to-do.txt
These first three sections (steps 2-4 and optionally step 1), in this order, must be in a preference file. Below them, you can add other sections:
List the files to ignore. Regular expressions may be useful in this section:
#Files to Ignore ignore=Virtualbox VMS ignore=*.tmp
As files are synced in the second root directory, they will be overwritten. As a precaution, you may want to back up the original files in the second root directory to another location, using these fields. If
backuplocation
is specified aslocal
, then backup files are stored in the second root directory; if the specification iscentral
, then backup files are stored in another directory that is specified inbackupdir
.Backups use the naming convention
.bak.VERSION.FILENAME
. The version begins with1
for the most recent and can be limited in number by using themaxbackups
field. The filename is the name of the original file, including any extension. This convention can be overridden using thebackupprefix
andbackupsuffix
fields, one of which must contain the version. For example:#Backups backuplocation=central backupdir=/home/bb/backups backup=$VERSION. backupprefix=backup backupsuffix= maxbackups=4
unison
can call on external commands likediff
anddiff -3
with the-diff
option and merge files using-merge PATTERN
. However, for either option to work, the path to the external command and any options to use with the command must be specified in a preference file. For instance:Diffs and merge diff=diff -y --suppress-common-lines merge = diff3 -m CURRENT1 OLD CURRENT2 > NEW
If you are syncing files on different servers or physical devices, you may want to log each use of
unison
. At the bottom of the preference file, add:#Log log=true
The logfile is located in the
.unison
directory.Advanced users can save complexity by using preferences another way:
In
default.prf
, include only the line:include common
- Create a file called
common
(notcommon.prf
). As in the first example,common
will define the two root directories. It must not contain anypath=
fields but may includeignore=
andbackup=
fields. Create preference files that contain only
path=
fields for backups, plus the statementinclude common
:#Directories to include path=Projects path=Fonts path=Downloads include common
Other than the directories to include, these stripped down preference files will take their characteristics from common
. Note that if you run unison
without specifying a preference file when using a common
file, then all subdirectories will be backed up.
Whether you use basic preference files or set up preference files with a common
file, the command is simplified to:
unison PREFERENCE-FILE
However, at least part of the time, you may still want or need to add other options. To give one obvious example, if you are syncing to a flash drive, then chances are you will want to add the -fat
option because many Linux users format flash drives to use FAT32.
Desktop vs. Command Line
unison
also supports a GTK+ graphical interface. This interface is often a separate package from unison
. In Debian, it is called unison-gtk
.
With unison-gtk
, you can create basic preference files using the included wizard. However, if you want to list files to ignore, or any other more advanced feature, you will have to open the preference file in a text editor. In particular, you might want to add keyboard shortcuts in the preference file. For example, the lines:
#Downloads Backup path=Downloads batch=true key=1 include common
This entry will customize unison
so that, when the GUI is open, pressing the 1 key will signal unison
to automatically sync any updates found in the Downloads
directory, using the information found in the common
file.
However, although unison-gtk
is convenient for average users, there may still be many times when you want to run unison
from the command line to take advantage of some of the options. Despite the availability of unison-gtk
, taking advantage of the prompt still requires a terminal.
Entering Options from the Prompt
Using preferences usually means that no options need to be entered at the prompt. Those that might be occasionally needed are normally general ones about how unison
operates, rather than about content. If manually entered options are used, they will override those specified in the preference file.
For example, when the output lists changed files, -sortbysize
, -sortfirst PATTERN
, or -sortnewfirst
will change the order of items on the list. You might also choose -auto
to accept non-conflicting default options without an interactive prompt. Alternatively, you might specify that you are prompted before deleting large numbers of files with -confirmbigdel
or -confirmmerge
before merging files. You might also add -links
to assure the copying of symbolic links.
If one root is on an external drive formatted with FAT32 – a common occurence in Linux – you will need to add the -fat
option. Similarly, if you are making a backup, you will not want to confirm each operation, so you use the -auto
option to disable confirmations.
Generally, the more you use unison
and create preference files to meet your needs, the less likely you are to need to add options manually. If you find yourself adding the same options all the time, check to see if they can be added to the preference file instead. Your use of unison
will be far easier if you can.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Direct Download
Read full article as PDF:
Price $2.95
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
News
-
KDE Plasma 6 Looks to Bring Basic HDR Support
The KWin piece of KDE Plasma now has HDR support and color management geared for the 6.0 release.
-
Bodhi Linux 7.0 Beta Ready for Testing
The latest iteration of the Bohdi Linux distribution is now available for those who want to experience what's in store and for testing purposes.
-
Changes Coming to Ubuntu PPA Usage
The way you manage Personal Package Archives will be changing with the release of Ubuntu 23.10.
-
AlmaLinux 9.2 Now Available for Download
AlmaLinux has been released and provides a free alternative to upstream Red Hat Enterprise Linux.
-
An Immutable Version of Fedora Is Under Consideration
For anyone who's a fan of using immutable versions of Linux, the Fedora team is currently considering adding a new spin called Fedora Onyx.
-
New Release of Br OS Includes ChatGPT Integration
Br OS 23.04 is now available and is geared specifically toward web content creation.
-
Command-Line Only Peropesis 2.1 Available Now
The latest iteration of Peropesis has been released with plenty of updates and introduces new software development tools.
-
TUXEDO Computers Announces InfinityBook Pro 14
With the new generation of their popular InfinityBook Pro 14, TUXEDO upgrades its ultra-mobile, powerful business laptop with some impressive specs.
-
Linux Kernel 6.3 Release Includes Interesting Features
Although it's not a Long Term Release candidate, Linux 6.3 includes features that will benefit end users.
-
Arch-Based blendOS Features Cool Trick
If you're looking for a Linux distribution that blends Linux, Android, and web apps together, blendOS might be what you're looking for.