Automated backup

Backup Box

Article from Issue 185/2016
Author(s):

Use the power of Bash to transform a Linux machine into a device for automatic backup of storage cards and cameras.

Keeping your snapshots and photos safe when you are out and about is as important as staying dry and warm on a cold and rainy day. After all, losing your photos, like going down with the flu, is no fun at all. If you already pack a laptop with you, you can use it to back up images from storage cards and cameras. But, lugging a full-blown machine just for this purpose is not very practical.

You can, of course, splurge on a dedicated storage device with a built-in card reader, but they tend to be rather expensive and limited. Instead of spending money on something like this, you can build your very own automatic backup device that can handle both cards and cameras and will cost only a fraction of what you would pay for a similar product on the market. All you need is a single-board computer (SBC) capable of running a regular Linux distribution and a dash of Bash scripting. Choosing the DIY approach will not only save you money, it will also allow you to build a much more versatile device and learn several useful techniques in the process.

It may come as no surprise that a Raspberry Pi makes a perfect platform for the mobile backup device: It's cheap and small, and it can run a Debian-based Linux distribution. Although any model will do, you might want to use the Raspberry Pi Model B for more flexibility, because it offers two or more USB ports. Instead of Raspberry Pi, you can use any other SBC that can run Linux. This project assumes that you are using Raspberry Pi 2 Model B running the latest release of Raspbian. The machine is connected to the local network and accessible via SSH.

Besides the SBC, you'll also need a USB card reader and a high-capacity USB stick. You could replace the latter with an external hard disk as the backup storage, but it would make the entire setup awkward for use on the move.

Use another Linux machine on the same network to establish an SSH connection to Raspberry Pi, and you are ready to start.

The Camera Backup Script

All functionality in the backup device is implemented using Bash scripting, and you can start with the script that transfers photos from a camera connected via USB. Before you proceed, make sure that the camera you are using is supported by gPhoto2 software [1]. The first step is to install the gPhoto2 tool that will do all the work. Run the command below and wait until it finishes:

sudo apt-get update && sudo apt-get install gphoto2

Connect the camera to the Raspberry Pi, and issue the gphoto2 --auto-detect command. The output returns the name of the detected camera along with the port it's connected to:

Model Port
--------------------------------
USB PTP Class Camera usb:001,012

The important part here is the first word in the camera's name. In this case, it's USB, but it could be a camera maker name like Nikon or Sony. The script uses this specific word to determine whether the camera is connected or not.

Use the nano camerabackup.sh command to create a text file and open it for editing. Paste the code in Listing 1 into the text file. If necessary, replace the USB value of the CAMERA variable with the actual camera maker name returned by the gphoto2 --auto-detect command.

Listing 1

Camera Backup Bash Script

 

Although the script is relatively short and simple, it does several things. When the script runs, it waits for the backup storage device to be connected. It does so using:

DEVICE=$(sudo ls /dev/* | grep $STORAGE_DEV | cut -d"/" -f3)

which consists of three separate commands. The sudo ls /dev/* command lists all devices in the dev directory. The output is then piped to the grep $STORAGE_DEV command, which finds the line with the name specified in the $STORAGE_DEV variable. Finally, cut -d"/" -f3 extracts the device name.

Here is how this works in practice. When you plug in a USB stick, it appears as the /dev/sda1 device in the output returned by the sudo ls /dev/* command. The grep $STORAGE_DEV command passes the obtained /dev/sda1 result to the cut -d"/" -f3 command that extracts the sda1 part. This value is then assigned to the DEVICE variable. If this variable is empty (i.e., the storage device is not connected), the script enters the whiledo loop that continues to run until the sda1 device is detected.

When the storage device is detected, the script mounts it using the sudo mount /dev/$STORAGE_DEV $STORAGE_PATH statement. Also, the command mounts the device on the /media/storage mountpoint, and you need to create it manually if it doesn't already exist. You can do this using the sudo mkdir /media/storage command.

To simplify the camera backup script, you can configure the system to mount the storage device automatically on boot. To do this, plug the storage device into the Raspberry Pi and run ls -l /dev/disk/by-uuid/. This returns a list of all connected devices, which should look something like what is shown in Listing 2:

Listing 2

List of Connected Devices

 

The sda1 device in the listing is the USB stick and b5b53ac0-1869-42c8-bc12-e870d30ffce2 is its UUID identifier.

Next, create a mountpoint for the USB storage device using sudo mkdir /media/storage. Open the fstab file for editing using the sudo nano /etc/fstab command and add the following (replace the example UUID with the actual value):

UUID=1b5b53ac0-1869-42c8-bc12-e870d30ffce2 /media/storage ext2 defaults 0 0

Reboot Raspberry Pi to mount the USB storage device on the /media/storage mountpoint. This approach has two drawbacks, though. First, the USB storage device must be plugged in before you power up the Raspberry Pi; otherwise, the machine will refuse to boot. Second, you can't just replace the specified USB storage device with another one without manually updating the UUID string in the /etc/fstab file.

Back to the camera backup script: Once it has detected and mounted the storage device, it checks whether the camera is connected using:

gphoto2 --auto-detect | cut -d ' ' -f 1 | grep $CAMERA

The cut -d ' ' -f 1 command extracts the first words from each line of the output returned by the gphoto2 --auto-detect command, and grep $CAMERA tries to find the specified camera name in the resulting word list. If this statement returns the empty value, the script enters another whiledo loop that runs until the camera is detected.

When the camera is connected and detected, the script creates a directory specified in the BACKUP_PATH variable, switches to it, and downloads all the photos from the camera using

gphoto2 --get-all-files --skip-existing

but skipping those that already exist in the destination directory. Finally, the sudo halt command powers down the Raspberry Pi when the script has finished.

Save the file and quit the editor, then make the script executable using:

sudo chmod 755 camerabackup.sh

Finally, you need to configure the Raspberry Pi to run the script on boot. There are several ways to do that, but probably the easiest way is to create a cron job. Run the crontab -e command and specify a cron job as follows:

@reboot sudo </path/to>/camerabackup.sh

Don't forget to replace </path/to> with the actual path to the script (e.g., /home/pi/camerabackup.sh). To save the script's output in a logfile (which can be useful for troubleshooting the script), modify the cron job as follows:

@reboot sudo </path/to>/camerabackup.sh >> </path/to>/camerabackup.log

Save the changes and reboot the Raspberry Pi. Plug in the storage device first, then connect the camera. If everything works properly, your DIY backup device should transfer photos from the camera and power itself down.

Storage Card Backup

The camera backup script transfers photos directly from the camera, but there are situations when this is not practical. First, the camera must be powered up during the backup process, and because the backup operation can take some time, this can drain your camera's battery. Second, not all cameras are supported by gPhoto2, and if your particular camera model is among them, the script is useless. In this case, a Bash script that can transfer photos from a storage card connected to the backup machine via a card reader should do the trick.

If you take a look at the script in Listing 3, you'll notice that it shares some similarities with the camera backup script. It uses a combination of

Listing 3

Card Backup Bash Script

 

DEVICE=$(ls /dev/* | grep $STORAGE_DEV | cut -d"/" -f3)

and a whiledo loop to detect a storage device and a card. When devices are detected, the script mounts them on the specified mountpoints.

So far, this is nothing out of the ordinary, but the code block that starts with

UUID=$(ls -l /dev/disk/by-uuid/ | grep $CARD_DEV | cut -d" " -f9)

deserves a closer look. In most cases, you'd likely use several storage cards, so it's necessary to have a mechanism in the script to identify each card and transfer its content to separate directories. The code block

UUID=$(ls -l /dev/disk/by-uuid/ | grep $CARD_DEV | cut -d" " -f9)

uses the ls, grep, and cut tools to extract the unique UUID value of the mounted storage card.

Sometimes a card doesn't have a UUID, though. For example, cards used with certain Sony camera models don't have UUIDs, so if the statement returns an empty value, the script uses the read -r ID < $CARD_PATH/id command to read the first line in the id text file on the card. You will need to create this file manually and specify the desired name for the card on the first line. Finally, the script uses the Rsync tool to transfer photos from the card to the storage device.

To deploy the script, make it executable using the sudo chmod 755 cardbackup.sh command. Add a cron job as described earlier to make the script start at boot, and you are done.

Add Some Blink

The scripts are designed to run automatically: Boot up the Raspberry Pi and plug in the storage device and the card reader or camera, and the scripts take care of the rest. That's fine, but it would be even better if you had some sort of visual feedback. You can add a simple LED circuit to the Raspberry Pi and modify the scripts to use the LED for notifications (e.g., continuous light when the script is running, three blinks when a storage device is mounted, etc.); however, an LED and resistor dangling out of the Raspberry Pi is neither convenient nor practical.

BlinkStick Nano [2] provides a much more elegant and flexible solution. This tiny device features two multicolor LEDs, and it plugs directly into a USB port. Better still, BlinkStick Nano can be controlled from the command line, so you can easily integrate it into the scripts. To enable BlinkStick Nano, install the blinkstick package using PIP:

sudo apt-get update && sudo apt-get install python-pip
sudo pip install blinkstick

To make BlinkStick Nano blink, use the blinkstick command with the --repeats (number of blinks) and --blink option followed by the LED color. The command below, for example, makes BlinkStick Nano blink green three times:

blinkstick --repeats 3 --blink green

You can also use the --index 1 option to use the second LED:

blinkstick index 1 --repeats 3 --blink yellow

Now, all you have to do is strategically place the appropriate blinkstick commands in each script. For example, adding blinkstick --repeats 3 --blink green in the beginning of the script will notify you when the script is up and running.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95

News