Adding scripts and tools to SystemRescue

Custom Repair Toolkit

© Lead Image © Maksym Yemelyanov, 123RF.com

© Lead Image © Maksym Yemelyanov, 123RF.com

Article from Issue 274/2023
Author(s):

You can do more with SystemRescue than just repair broken systems. By adding tools and scripts, you can create a custom rescue environment that meets your needs.

Computers crash. Whether because of human error, software bugs, or hardware dying from old age, workstations and servers alike will eventually break. For this reason, every system administrator should have a digital first-aid kit to get broken systems back to working condition as soon as possible. For this task, I keep SystemRescue in my toolkit.

SystemRescue, formerly known as SystemRescueCd, is a Live Linux distribution designed to repair broken systems and handle a number of maintenance tasks. It includes tools to recover deleted files, repair broken boot managers, fix inconsistent filesystems, and more. The default set of installed programs is impressive (see Table 1 for a sample of included tools or visit the SystemRescue website [1] for the complete list). If one of your workstations or servers is rendered unbootable, you can simply boot your SystemRescue DVD or USB and have a full recovery environment available in a matter of minutes. With SystemRescue, you can repair damaged boot managers, partition tables, or filesystems. More importantly, you can extract your files from the failing computer and safely save them to a USB-attached hard drive or over a network.

Table 1

SystemRescue Tools (an Incomplete List)

Tool

Description

GNU ddrescue

Copies data from devices with bad blocks (such as damaged hard drives)

FSArchiver

Saves and restores filesystem images

OpenSSH server

Controls the recovery environment from a remote machine and easily transfers files in and out of the damaged computer

Minicom

Connects to serial consoles

cryptsetup

Accesses Linux encrypted disks

chntpw

Resets Windows passwords

TestDisk

Recovers lost partitions and repairs boot sectors

PhotoRec

Recovers files deleted accidentally (or otherwise)

wipe

Deletes data securely

Rclone

Transfers and syncs files to a number of storage systems, including FTP and Nextcloud

Despite SystemRescue's impressive feature set, you will eventually find yourself in a situation where the default features fall short. SystemRescue's maintainer has recognized this possibility and made it easy to create your own SystemRescue solutions to suit your needs. This article explains how to create a custom SystemRescue environment.

The Need for Customization

Two real-life scenarios call for creating a custom SystemRescue-based distribution. The most common scenario involves needing a tool that SystemRescue does not include by default. Because SystemRescue is based on Arch, it has access to Arch's repositories. If you ever need a tool, you can simply install it in your rescue environment as you would on a regular operating system. In practical terms, you can use the pacman package manager from within a running SystemRescue environment. For example, if you prefer Emacs over the default Vim, you can install it with the following commands:

# pacman -Sy
# pacman -S emacs

Packages installed this way will be placed in a filesystem overlay in RAM. The packages will be available only to the rescue environment and won't touch the computer's hard drive that you are rescuing. On the downside, you will have to manually install these packages every single time you boot SystemRescue, because packages you install with pacman are not kept across reboots. Therefore, if you use an application often or expect to be performing maintenance tasks without access to Arch's repositories, you will need to include the desired packages in your custom version of SystemRescue.

Another scenario that calls for a custom SystemRescue involves a recovery environment that allows running custom scripts, performing tasks automatically, or executing services without the intervention of an administrator. For example, a server at your office breaks down and the responsible system administrator is out of town, requiring the sys admin to instruct an untrained co-worker to perform recovery via phone, which may not end well. An easier solution would be to have the sys admin fix the problem remotely by telling the co-worker to grab a recovery DVD from the shelf, put it in the server's DVD tray, reboot, and have SystemRescue offer a shell to the sys admin.

USB or DVD?

The easiest way to create a custom SystemRescue is to install it on a USB drive using a backing store [2], which is an area on the USB drive where changes to the running SystemRescue environment are written. You can install SystemRescue to a USB with a backing store, boot it, install all the packages you need, and configure all the desired scripts, and then shut it down. Your changes will be preserved across reboots this way.

While this approach is serviceable, I personally avoid using USB drives for recovery environments for a number of reasons. DVDs offer a reasonable guarantee that you will get a pristine recovery environment every time you boot, and there is value in that. (Your little brother is more likely to borrow a USB and replace your valuable data with B-movies.)

Because I want my rescue environment stored to read-only media, I can't use a backing store. Instead, I will create an ISO image containing my custom software and configurations and then burn it to a DVD.

Adding SSH Access and Persistent Keys

To illustrate how to include additional files and custom auto-started services on a SystemRescue DVD, I'll use the example of a bunch of servers in a remote office. One of the servers breaks, and the sys admin needs to make it boot SystemRescue and offer an SSH shell to connect to it from home.

While SystemRescue includes an SSH server, it is not accessible by default for two reasons: A proper password needs to be configured first, and the default firewall blocks incoming SSH connections. To remedy this, you need to create a custom SystemRescue image that does the following:

  • Set a password to enable login over SSH
  • Remove the firewall
  • Use a persistent set of SSH fingerprints (because I want to verify the SSH server's identity when connecting to it remotely)

This custom SystemRescue image will then allow the senior sys admin to instruct a junior sys admin in the server room to boot SystemRescue, which will allow the sys admin to connect to the rescue environment without the junior admin performing any complex task.

To implement this setup, you need to download a copy of SystemRescue using a standard Linux system:

$ curl -L https://sourceforge.net/projects/systemrescuecd/files/sysresccd-x86/10.00/systemrescue-10.00-amd64.iso/download > systemrescue-10.00-amd64.iso

A helpful script by the SystemRescue creator lets you unpack the ISO image you just downloaded. You can use the script to extract the contents of the ISO file and modify them before repackaging. You can download the script as follows:

$ curl -L https://gitlab.com/systemrescue/systemrescue-sources/-/raw/main/airootfs/usr/share/sysrescue/bin/sysrescue-customize?inline=false> sysrescue-customize

Once the script, sysrescue-customize, has been downloaded, extract the image's contents with the following command:

$ ./sysrescue-customize --unpack --source=systemrescue-10.00-amd64.iso --dest=custom

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Specialty Distros

    In the Linux world, form follows function. A specialty distro is a Linux-based system designed to serve a specific role. We look at some classic examples.

  • This Month's DVD

    SystemRescue 10.0 and Linux Lite 6.4

  • On the DVD: Linux Mint 14 & Knoppix 7.0.4

    Linux Mint 14 “Nadia”

    Linux Mint is a wildly popular Ubuntu-based desktop Linux with an emphasis on simplicity. Along with a strong assortment of open source applications and utilities, Mint takes pains to provide proprietary drivers and other helper tools for seamless-as-possible operations. The Mint 14 release features a new Windows Quick-list app for managing windows across multiple workspaces, as well as Alt+Tab window-switcher virtualization and a handy notification tool. Also debuting with this release is the new Nemo file browser, a fork of Gnome’s Nautilus file manager based on the Nautilus 3.4 codebase.

  • Spoiled for Choice: Assorted Linux from Ultilex 4.0.0

    Ultilex 4.0.0 promises to be the ultimate Linux experience: the multi-boot live Linux distribution offers a whole bouquet of Linux distributions to prove it.

  • Trinity Rescue Kit

    Trinity Rescue Kit is driven by the practical requirements of the admin’s daily work, integrating a full set of tools for maintaining and rescuing Linux and Windows PCs.

comments powered by Disqus
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters

Support Our Work

Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.

Learn More

News