Experimental package management with Nix and NixOS

Brand New Package

© Photo by Jess Bailey on Unsplash

© Photo by Jess Bailey on Unsplash

Author(s):

NixOS and the Nix package manager offer a promising new approach to the challenge of managing packages in Linux.

In most Linux distributions, the files associated with the OS end up in specific places. Most Linux distributions subscribe to the filesystem hierarchy defined in the Linux Standards Base (LSB), which specifies the familiar directory names you are accustomed to if you work in Linux (/etc, /dev, /bin, etc.). On most Linux systems, if you do not have the files in the right places, the applications will not find them. But experimentation is at the heart of Linux, and every rule has an exception. An innovative project called NixOS [1] takes a different approach to system configuration and package management.

On NixOS, all files are in the Nix store. For applications to find them, NixOS links to the correct locations with symlinks. This approach makes all kinds of things possible. For instance, you can have several versions of any library and let the application know which link to use. For development environments, you can even test an application using system files from different distributions, just using the package manager.

On a NixOS system, everything is reproducible. A good backup of your home directory and a few *.nix files is all you need to get back up after a crash.

At the heart of NixOS is the Nix package manager. Nix is a cross-platform package manager that can run on other distros as well, but NixOS was created to provide a native environment for testing and experimenting with Nix.

When you make a change to your system, NixOS creates a new generation. A generation contains all the links to the software you want to run. When you come into your favorite boot manager (GRUB?), it will give you the default boot or alternatives. The alternatives contain all versions of the system you have created since you installed. Yes, as you will learn later in this article, saving all previous versions of the system can waste a lot of disk space, so you will want to take the time to retire older generations you are no longer using.

Advantages and Disadvantages

From a practical point of view, changing your configuration is easier on NixOS than on other distributions. Let's say you want to use another window manager but still keep the first one for backup. If you make the install and something goes wrong, or if you change your mind, you can roll back. To roll back, reboot and pick the earlier generation.

You might be missing a few packages, but Nix is popular enough to boast 80 thousand packages in the default repository. You can create your own package if you are capable enough. There is also support for the AppImage and Flatpak systems, although Snap support seems a little lacking.

NixOS does not have a graphical installer. You need to change the settings for keyboard, locale desktop environment, and similar by changing the configuration.nix file. Keep in mind, also, that the Nix package manager does not have a graphical interface, which could be a disadvantage for some users.

When you first try NixOS, all you need to do is partition your drives, or label the existing ones, and run nixos-generate-config. Next, you change one file (configuration.nix), run nixos-install, and you are done.

NixOS lets you put all the packages that you want to use in the configuration.nix file. You can then use configuration.nix to recreate the same configuration on any other machine.

Nix Package Manager

The real strength of NixOS is the Nix package manager. The package manager has many unique features, such as rollback, support for multiple versions, and the ability to reproduce the install. The Nix package manager is an interesting option if you are developing software and you are getting tired of the multitude of virtual environments.

To fetch the Nix package manager only, you need to use the link available at the project website. The installer will ask you for sudo access to create the /nix directory as root.

sudo mkdir /nix
curl -L https://nixos.org/nix/install | sh

The more cautious of you will download the script and read it through before running it. Bear in mind though, that if you run the install with a non-privileged user, the script will ask for root, so it can create and write to the /nix directory.

You can also create a multi-user environment with the installer script. To do this, use the --daemon option:

sh <(curl -L https://nixos.org/nix/install) --daemon

Notice that you are still piping the script to the shell. Many seasoned administrators will not bat an eyelid over this, except for the security implications of running a script straight from a URL. You can see the start of the script in Figure 1.

Figure 1: The Nix package manager is very informative. If you pay attention, you will know every detail of your system.

The installer is aware of whether you have already installed Nix before and warns you about it.

The installer has several different configuration options, depending on what you want to do, but the main idea is to support any version of software at any time. One option is to install the Nix package manager using the installer and then pick a package and see if it has a version you like.

nix-env -i gimp

After having installed Gimp, you use the which command to discover the version:

which gimp

It is a good idea to check through the applications you are already using and see if they are present and maintained in the NixOS repositories. NixOS is still not very common, so some packages may not have the latest available version. If you want to check what software is available before installing, the NixOS website has a great searchable list (Figure 2).

Figure 2: The NixOS website includes a convenient package search feature.

After you have installed and tested a few packages, you should consider the space on your system. Due to the store and the way Nix stores the files, you may have many binary files that are identical. To fix this, search through the Nix store and re-link the files. Don't worry – there is a simple command line tool for this task. Check your Nix store using the nix-store command:

nix-store --optimise

Best practice is to have NixOS do this automatically on a regular basis. What the command does is go through all the binaries and compare them. When it finds a match, it creates a new link and erases the, now unnecessary, file. When you have tried your favorite applications, you can list them using the nix-env command again. Any applications that you cannot find in the Nix repositories might be available using AppImage and Flatpak. Support for Snap packages is a bit more dubious. To store your choices in a separate file, redirect the result to a text file:

nix-env -q >>Nix-applications.txt

You can use this list when you decide to switch your entire system to NixOS.

Dev Environments

Once you install the Nix package manager, you can use it to keep your development environments clean and consistent. With the nix-shell command, you can specify exactly what versions you have available. If anyone else is sharing the project, they can use the default.nix and shell.nix files to create an environment identical to yours.

To install an individual package, use the -p option. If you want to run Python without installing on your system, or a version other than your current system, try the following:

nix-shell -p python3

You will get a prompt (Figure 3).

Figure 3: The nix-shell command shows that the Python version is different.

Theoretically, you could install your whole environment with the -p option, but that would be counter productive. Instead, you can put all the complicated stuff in the shell.nix and default.nix files.

Inside these files, you can define your installation. The nix-shell command calls the shell.nix file first. Listing 1 shows an old example that starts an environment for running the Teensy development board, courtesy of Richard Zetterberg [2].

Listing 1

Teensy Environment

01 shell.nix:
02
03 { system ? builtins.currentSystem }:
04
05 let
06   pkgs = import <nixpkgs> { inherit system; };
07 in
08   pkgs.callPackage ./default.nix {}
09
10
11 default.nix:
12
13 { stdenv, lib, gcc, gcc-arm-embedded-, teensy-loader-cli }:
14
15 stdenv.mkDerivation rec {
16
17   buildInputs = [
18     gcc
19     gcc-arm-embedded-4_7
20     teensy-loader-cli
21
22   ];
23 }

In Listing 1, the curly brackets at the top contain the environment you need for this shell. In most cases, you will put stdenv here. You can see that this file needs gcc, lib, and the special gcc version for ARM devices. This does not tell you how to compile the package, though, which you do with stdenv.mkDerivation. In Nix parlance, the word derivation means building something from something else.

To make this work, you create the directory and place the files inside. Only one of the files is required, but most people mix in a separate default.nix to separate the parts of the system. In the file, the most important part you want to know about is the curly brackets: {...}. The curly brackets contain the standard environment you want to take from. The <nixpkgs> references the list of files that exist in the repository. For a shell to work, you only need to put those curly brackets in and then add pkgs.mkshell.

The little script file in Listing 2 gives you the ability to start Python 3.8 in a shell. Since the flask packages are also added, you also have those modules available. To find the packages you want, see the NixOS website package page [3]. When you run nix-shell, the install happens on its own. You do not need to install it first.

Listing 2

Python 3.8

01 { pkgs ? <nixpkgs> , ... }:
02
03 with pkgs;
04 mkshell {
05         buildInputs = [
06         python38Packages.flask
07         ];
08 }

Preparing Your System

The first issue you'll face when transitioning to NixOS is which applications to enable. Most users will find all the applications they need in the NixOS repositories. (By the way, there are also some alternative repositories.)

Even though the installer is command-line only, it is surprisingly easy to use. You can use any standard partition scheme. The install manual has examples that you can copy if you don't have any special demands. You do have to know how to handle a partition program. They suggest you use partition labels, which is really useful. You'll need to partition your disks, mount the root partition, boot your new system, and run the nixos-generate-configuration command:

nixos-generate-configuration --root /mnt

Use the manual that is included in the install ISO. The manual looks long and complicated, but it is actually comprehensive and clear. When this command has finished, you will need to edit your configuration.nix file.

configuration.nix

The nixos-generate-configuration command generates two files: configuration.nix and hardware.nix. In most cases, you will do nothing with the hardware file, since the installer has set everything according to the current system you are on. The interesting part is the configuration.nix file. Inside, you will have all the settings you need and the applications you want on the general system.

Starting from the top, the file has the curly brackets that define your environment (Figure 4). You change this setting for extreme cases; on a desktop, leave it alone. Second, you have an import statement that imports the hardware file. You can use this section if you want to break out some of your configuration. Any Nix file that you reference will be added to your configuration.

Figure 4: The top of the configuration.nix file points to the hardware file; you can add your own files for special purposes.

Next, there should be a few lines about your boot loader. The Nix installer detects UEFI and BIOS; don't change the boot loader configuration unless you have very particular circumstances. As you go down the file, most options are clear and follow standards for locale and time zone. When you want to choose a desktop system, however, you are left in the dark – the configuration file has nothing for you.

Fear not! The NixOS website has a very nice interface for looking for options and applications [3]. On this page, you can enter in the application that you want. If you search for Gnome, you get a long list of options (Figure 5). Click on the option you want and the Name given in the list is the name you put in the file. The Gnome entry needs the setting:

Figure 5: Exploring options and applications.
services.xserver.desktopManager.gnome3.enable = 1;

The list of options has more than 10,000 entries! You can use them when you want, but don't let the shear number hold you back. All options you need, initially, are in the original file. You will only need to add a window or desktop manager. Once you have written the file, you can install. The procedure is one script. At the end of the script, you need to set the root password.

The script compiles all programs that are not available in binary form, so it might take a while. You have options for distributing the compile to other computers. If you have problems, you can also choose the --show-trace option to see a diagnostic trace.

After the installation, reboot into your new system.

Updating Applications

At some point a package might require an upgrade. You can use the --upgrade action for this job. More surprising is that you can upgrade to the same versions. As the manual states, this might seem useless. However, you can use it to recompile your current applications to newer libraries and also cut out duplicates of libraries. Enter the following command:

nix-env --upgrade --leq

You can also skip the leq parameter, in which case the command will upgrade everything without optimizing the libraries you use. An upgrade leads to another concern; you may have many versions of any file in your system, which means you'll have many generations to deal with.

Generation Handling

Creating many generations can waste large amounts of file space. The utility that handles this is problem is nix-env, but you should use the wrapper nix-collect-garbage. nix-collect-garbage has the standard delete option that takes away all old generations. You can do this manually, but more useful is the --delete-older-than option. You should set this as a regular script:

nix-collect-garbage --delete-older-than 10d

Using Nix in the Cloud

Nix is not supported by many cloud companies, but many still provide ISO options. NixOS has a page describing which ones are friendly and which ones you can still get started on. The procedure is similar, though getting the disks created is a bit of a chore when full support is not included. The list of services is available at the project website [4].

Conclusion

Nix is very powerful tool for handling diverse configurations. Many developers appreciate the ease at which they can just run the Nix shell and have all dependencies correctly configured every time. Ordinary users will appreciate the power of rollbacks and the possibility to run old software without bending over backwards.

The Author

Mats Tage Axelsson keeps aging hardware for the challenge of handling crashes. Oh, and he also is a cheapskate.