How to synchronize files between two servers bidirectionally

Last updated on October 7, 2020 by Dan Nanni

Suppose you have a collection of files which are replicated on two different servers. The two replicas are then modified independently, and you want whatever changes made in one replica to be propagated to the other, so that both replicas remain in sync.

There are several file mirroring tools on Linux, such as rsync or duplicati. However, these tools are meant for uni-directional file sync (i.e., pushing or pulling incremental updates in one direction), and so two-way sync would require running such tools twice, one for each direction.

Unison is an open-source file synchronization tool that natively supports bi-directional file synchronization. Unison is available on multiple platforms including Linux, FreeBSD, Windows and MacOS X. In Linux, Unison is available as a command-line tool as well as a GUI program with GTK+ interface.

In this tutorial, I will describe how to synchronize files between two servers with Unison command-line utility.

Install Unison on Linux

For Ubuntu, Debian or Linux Mint:

$ sudo apt-get install unison

For Fedora, CentOS or RHEL:

$ sudo yum install unison

Create Unison Profile

The first thing to do is to create a Unison profile on both servers which have replicas to sync. A Unison profile is a text file (with .prf extension) that specifies file sync settings such as directory roots, include/ignore paths or patterns, etc.

You can create a Unison profile anywhere in your system, in which case you must define UNISON environment variable pointing to the directory path to the profile. If UNISON variable is not defined, Unison looks for profiles in $HOME/.unison directory by default.

The following snippet is a Unison profile example.

# Two root directories to sync.
# You can use ssh:// to sync over SSH
root = /home/alice/sync_folder
root = ssh://[email protected]//home/alice/sync_folder

# If you want one-way mirroring from one replica to the other, specify the source replica using "force" as follows.
# force = /home/alice/sync_folder

# If you want Unison to run without any user input, try "batch" mode.
batch = true

# If you don't want to be prompted, and just accept Unison's recommendation:
auto = true

# Optionally, you can sync specific sub directories only (under the root).
# path = dir1
# path = dir2

# Optionally, you can ignore specific files or directories that are matched with regular expressions.
# ignore = Name *.o
# ignore = Name *~
# ignore = Path */temp/archive_*

# If you want to ignore difference in file props:
perms = 0

The most important information in the above are the two root directories to sync. Unison supports SSH, RSH or socket to sync files over network. So you can specify the root directory on a remote host by using one of the formats below:

SSH:

root = ssh://user@remote_host//absolute/path/to/root
root = ssh://user@remote_host/relative/path/to/root

RSH:

root = rsh://user@remote_host//absolute/path/to/root
root = rsh://user@remote_host/relative/path/to/root

Socket:

socket://remote_host:port_num//absolute/path/to/root
socket://remote_host:port_num/relative/path/to/root

Sync Files with Unison

Once a Unison profile has been created on both servers, simply run Unison from either server.

$ unison

If there are multiple Unison profiles (e.g., sync1.prf, sync2.prf) in Unison directory, you can specify the profile name as follows.

$ unison sync1

If you have not set up key-based SSH authentication for the remote host, Unison will ask you to log in to the remote host via SSH before performing file sync. If you have set up passwordless SSH login, Unison will automatically start file sync over SSH.

Contacting server...
Connected [//local//home/alice/sync_folder -> //remote_host//home/alice/sync_folder]
Looking for changes
  Waiting for changes from server
Reconciling changes
new file -->            document1.pdf
         <-- new file   my.jpg  
Propagating updates
UNISON 2.40.63 started propagating changes at 21:19:13.65 on 20 Sep 2013
[BGN] Copying document1.pdf from /home/alice/sync_folder to //remote_host//home/alice/sync_folder
[BGN] Copying my.jpg from //remote_host//home/alice/sync_folder to /home/alice/sync_folder
[END] Copying my.jpg
[END] Copying document1.pdf
UNISON 2.40.63 finished propagating changes at 21:19:13.68 on 20 Sep 2013
Saving synchronizer state
Synchronization complete at 21:19:13  (2 items transferred, 0 skipped, 0 failed)

Troubleshooting Unison

Symptom: When you run Unison, you see the following message:

  Waiting for changes from server - document1.pdf
Reconciling changes

local          remote_host             
props    <-?-> props      /  [] 

This means that Unison attempts to synchronize file properties (e.g., file permission). You can press I to ignore this path permanently, and proceed. Alternatively, you can permanently ignore props by adding the following line to your Unison profile file.

perms = 0

Support Xmodulo

This website is made possible by minimal ads and your gracious donation via PayPal or credit card

Please note that this article is published by Xmodulo.com under a Creative Commons Attribution-ShareAlike 3.0 Unported License. If you would like to use the whole or any part of this article, you need to cite this web page at Xmodulo.com as the original source.