Improving Linux package management
Hooks and Triggers
Fedora is also slow because its unpacked package list alone is 60MB, while Alpine's list is a lean 734KB. Fedora offers over 20,000 packages, which is three times more packages than the very small Alpine, but the difference is still striking.
However, Stapelberg thinks even the best results in the test are too slow. He sees another reason for this in the frequently used hooks and triggers that the package manager executes during installation, which trigger the aforementioned maintainer scripts, create daemon user accounts (such as an FTP or WWW account), or create cache files.
One of the most commonly used triggers, the man package trigger ensures that a man page for the package is included on the system with each package installation. In his blog, Stapelberg explains why all this should not happen during installation and how it prevents parallel installation of packages [8].
In Stapelberg's opinion, these interruptions of the actual installation should preferably take place when the app is first launched. If an application does not start between installation and the first or even further updates, the adjustments would only be executed once instead of several times, for example.
Image Instead of Archive
In Stapelberg's opinion, a package manager should only do what is absolutely necessary to anchor a package in the system so that it is ready for use (i.e., start the program or load a kernel module). Unpacking during installation is not necessary if packages are available as filesystem images that the distribution mounts at startup, as is the case with AppImage or Snap.
According to Stapelberg, no package manager in a Linux distribution currently uses this scenario, although it could still increase the speed to above the level achieved by Alpine's apk, the fastest package manager in his test series. Images are currently only used by the Haiku operating system project.
In Stapelberg's experimental distribution distri, he seeks to experiment with reducing the complexity of package management. He concludes that distributions like Fedora or Debian could also run faster given less complexity. That doesn't mean it's technically easy to implement, but it would be feasible.
For example, distri uses read-only SquashFS images as the package format instead of the usual TAR archives (Figure 2). In addition to increased speed, this has the advantage that applications cannot be modified, which protects them from accidental or malicious modifications.
Distri organizes all files provided by a package under the /ro/
mount point, each in its own directory. The usual data exchange between software packages, which takes place via the specified directories of the Filesystem Hierarchy Standard (FHS) in conventional distributions, is handled by the system via exchange directories, which are provided by FUSE.
For example, the exchange directory /ro/share/
provides the union of the share/
subdirectory of all packages in the package store. The global exchange directories map the FHS with sufficient accuracy to allow third-party software, such as Google Chrome or Spotify, to work. Using /ro/
also prevents conflicts when installing multiple versions of a package.
Distri also streamlines package building. Unlike conventional distributions' builders, the distri package builder does not install packages in the build environment. Instead, the system provides a filtered view of the package store in /ro/
in the build environment. Even with large dependency trees, setting up a build environment this way takes a fraction of a second.
Distri's website provides information about the various ways to use the distribution [9]. There is no installer yet, but the maintainer has future plans for one. Distri can be started from a USB stick or in a Docker or LXD container, as well as in a virtual machine with VirtualBox or Qemu. Since it is in IMG
format [10] and not an ISO
, you first need to convert it to a Virtual Disk Image (VDI) for VirtualBox (Figure 3).
First, you unpack the image. Since the developers have packaged it with the relatively new Zstandard (Zstd) compression algorithm, you will probably need to install zstd up front. On Debian, you can do this with:
apt install zstd
On Fedora, use:
dnf install zstd
Then extract the file using the call:
$ unzstd ~/Downloads/distri-disk.img.zst
In the process, the compressed file loses the .zst
attachment and grows to 8GB. An attempt to start distri on an 8GB USB stick failed as expected; you need a stick with at least 16GB capacity. After starting distri, a login prompt opens where you can enter the password, which is peace for root (Figure 4).
You are greeted by a very simple Z shell prompt that lets you explore the system. At first, I thought the cd
command had failed, because the prompt does not show the new location after the change – but pwd
will help here. Entering cd /ro/
takes you to the directory with all installed packages; switching to /ro/share/
takes you to the exchange directory (Figure 5).
For common distri commands, and their equivalents in Debian, see distri's documentation [11]. This is also where you can learn more about Distri's package format and how to create your own packages. The distri update
command (Figure 6) replaces
apt update && apt full-upgrade
and upgrades the entire distribution at an impressive pace. With a 1Gbps connection, the system downloads and installs around 275MB of data in less than four seconds.
Package installations are also completed in the blink of an eye (Figure 7). In the case of the Nano editor
distri install nano-amd64-4.9.5-2
it took just over a millisecond. You need to specify the package version because multiple versions can be installed in parallel. Available packages can be found in the distri repository [12].
Conclusions
Because distri is an experiment in package management, it is likely to interest only a limited user base. If you want to dig deeper, you should read all of Stapelberg's blog posts on the topic [13] and watch his keynote at the Arch Developer Conference 2020 [14]. While there is no official support for distri, Stapelberg will answer questions on the mailing list [15] and in the #distri chat room on the legacy.irc-robustirc IRC server. However, you may have to be patient.
Infos
- distri: https://distr1.org
- Stapelberg's critique of Debian: https://michael.stapelberg.ch/posts/2019-03-10-debian-winding-down/
- Lintian: https://en.wikipedia.org/wiki/Lintian
- Package manager comparison: https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/
- Maintainer scripts: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html
- Debian flowcharts: https://www.debian.org/doc/debian-policy/ap-flowcharts.html
- Fedora scriptlets: https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/
- Hooks and Triggers: https://michael.stapelberg.ch/posts/2019-07-20-hooks-and-triggers/
- Using distri: https://distr1.org/getting-started/
- distri download: https://repo.distr1.org/distri/supersilverhaze/img/
- distri documentation: https://repo.distr1.org/distri/jackherer/docs/rosetta-stone.html
- distri repository: https://repo.distr1.org/distri/supersilverhaze/pkg/
- Michael Stapelberg's blog: https://michael.stapelberg.ch/posts/tags/distri/
- Stapelberg's keynote at Arch Developer Conference 2020: https://media.ccc.de/v/arch-conf-online-2020-6387-distri-researching-fast-linux-package-management
- distri mailing list: https://www.freelists.org/list/distri
« Previous 1 2
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.
-
ZorinOS 17.1 Released, Includes Improved Windows App Support
If you need or desire to run Windows applications on Linux, there's one distribution intent on making that easier for you and its new release further improves that feature.
-
Linux Market Share Surpasses 4% for the First Time
Look out Windows and macOS, Linux is on the rise and has even topped ChromeOS to become the fourth most widely used OS around the globe.
-
KDE’s Plasma 6 Officially Available
KDE’s Plasma 6.0 "Megarelease" has happened, and it's brimming with new features, polish, and performance.
-
Latest Version of Tails Unleashed
Tails 6.0 is based on Debian 12 and includes GNOME 43.
-
KDE Announces New Slimbook V with Plenty of Power and KDE’s Plasma 6
If you're a fan of KDE Plasma, you'll be thrilled to hear they've announced a new Slimbook with an AMD CPU and the latest version of KDE Plasma desktop.
-
Monthly Sponsorship Includes Early Access to elementary OS 8
If you want to get a glimpse of what's in the pipeline for elementary OS 8, just set up a monthly sponsorship to help fund its continued existence.