Universal Package Systems and competing standards
Status Report
Billed as the future of package management, universal package systems like Snappy and Flatpak have failed to live up to their promise.
Remember universal package systems? Although AppImage [1], the earliest universal package system, was first released in 2004, the concept did not capture much attention until a decade later, when Canonical released Snappy [2] and Red Hat released Flatpak [3]. Each was presented as the next generation of package managers, usable by any distribution, and as a means to reduce the number of rival technologies. Yet in 2020, both Snappy and Flatpak have receded into the background, and the deb and RPM package management systems continue to dominate Linux, leaving the question of why Snappy and Flatpak did not fulfill their promises.
Two quick searches on DistroWatch reveal that, out of the 273 active distros listed, 39 support Flatpak [4], and 35 support Snap packages [5]. At first, those may sound like respectable numbers, until you realize that a much more arcane deviation from the norm, like distros that do not ship systemd, can boast 99 distros. Moreover, those figures consist mainly of major distros that support Flatpak and Snap – often both – but still depend primarily on traditional package managers.
Theory vs. Practice
A serious drawback to universal packages is that, to be truly universal, they require that each distribution be structured the same as others. Despite efforts like the Linux Standard Base, this requirement is simply not met. Many distros continue to place key files in different positions. For this reason, the promise that universal packages would reduce the amount of work needed to ship packages has no practical chance of being realized.
Similarly, although still in discussion, Alexander Larsson, one of the original Flatpak developers, has championed the placement of Flatpak in containers, which has also fallen short of theory. For one thing, containers are optional, and the level of security often varies. Just as importantly, a study at North Carolina State University in 2017 [6] showed that over 356,000 community-contributed container images averaged 156 vulnerabilities, while 3,800 official images averaged 76. In most cases, these vulnerabilities were rated high severity. In other words, while containers might be secure in themselves, what is in them may not be secure.
As Adrian Coyler, who summarized the North Carolina State study, pointed out, new packages often perpetuate vulnerabilities by borrowing dependencies from older ones for convenience. In fact, even a containerized package that is up to date when created may later prove to have vulnerabilities but continue to be used. With the introduction of automatic updates and tools such as Docker Security Scanning, these problems may be mitigated, but, even so, much still depends upon the conscientiousness of a package's maintainer. Consequently, the promised advantages of universal packages often are yet to be realized.
Going Against Custom
The technical challenges are only part of the reason for the lukewarm reception of universal package managers. It is true that the deb and RPM package managers were designed in eras of limited memory, rather than today's abundance. However, for the average user, that hardly matters. In fact, when installing on older or limited systems, like many bottom-of the-line laptops, the efficient memory use can still be relevant. Snaps, for example, may be better suited for use with containers, but, overall, the incentive to move away from traditional packages simply hasn't been there once the universal package manager's novelty subsided.
Part of the problem may be infrastructure. When universal packages were introduced, the rationale was that they would be built by upstream developers. By removing the distributions from the delivery, in theory, packages could get into the hands of users more quickly. The trouble is that was a new role for upstream developers, and one they have not always undertaken. This should not be surprising, because upstream developers' operations are simply not geared for it. Often, they may not have the numbers to provide such service. As a result, packaging has remained largely in the hands of distro developers, who have the experience and infrastructure for it. While distro developers appear perfectly willing to produce universal packages of different formats, the role tends to be a sideshow, secondary to maintaining traditional packages.
Even more importantly, despite what the makers of universal packages maintain, a functional distribution is not a matter of technology so much as policy – specifically, of quality control. Josh Triplett [7], a long time Debian contributor, explains the reason for Debian's dominance among currently active distributions: "Debian without the .deb
format would still be Debian; Debian without Debian Policy would just be SourceForge or rpmfind" – that is, repositories of packages and source code that you could find on a web page with no overall quality control.
The Debian Policy Manual [8] is a lengthy, much revised document that explains what goes in to both a Debian package and a general release. It explains what a package must or may contain, and how it interacts with other packages. It includes where files and logs should be placed, and dozens of other details. No other distribution is so specific about such matters, although almost all have similar documents.
As a Debian package moves from the Unstable repository to Testing and Stable, it is closely examined for compliance with Debian Policy. Moreover, as John Goerzen [9], another veteran Debian contributor, notes, this initial quality control is reinforced throughout a package's life history through "unattended-updates, needrestart, debsecan, and debian-security-support [...] Debian's security team generally backports fixes rather than just say 'here's the new version,' making it very safe to automatically apply patches. As long as I use what's in Debian Stable, all layers mentioned above [everything] will be protected using this scheme."
Originally, universal packages lacked most of these quality control measures. Today, ones like automatic updates are mostly standard, but the kind of quality control offered by Debian is not instituted over night. Quality control in major distros like Debian is the result of decades, and universal packages cannot be expected to equal them in a few years, especially since quality control is a specialist role that many developers do not favor.
The Future of Universal Packages
Universal packages do have one advantage: They make having multiple versions of a package on the same system easier. However, when traditional packages are numbered, multiple versions of libraries and even desktop packages like LibreOffice can coexist on the same system. Besides, multiple versions are a special case that do not affect many users unless they mix package repositories. Nor is there any reason why users should not mix package systems as they choose.
In a sense, universal packages are a modern version of static tarballs, which include all the dependencies needed to install a package. However, the majority of distributions rejected that model for package management years ago, and the improvements offered by universal packages are not enough to make them preferable to traditional systems.
Neither Flatpak nor Snap are about to go away, especially since they are backed by major Linux corporations. Free software has never been slow to use new technologies, and universal packages are no exception. Still, their current status is far from "the future of application deployment" promised on Flatpak's front page or "the new bullet-proof mechanism for app delivery and system updates" [10] announced by Snappy. Instead, as a much-reprinted famous xkcd comic observed [11], the attempt to reduce the number of competing standards has only added to the confusion, and the benefit is small.
Infos
- AppImage: https://appimage.org/
- Snappy: https://snapcraft.io/
- Flatpak: https://www.flatpak.org/
- 39 support Flatpak: https://distrowatch.com/search.php?ostype=All&category=All&origin=All&basedon=All¬basedon=None&desktop=All&architecture=All&package=Flatpak&rolling=All&isosize=All&netinstall=All&language=All&defaultinit=All&status=Active#simple
- 35 support Snap packages: https://distrowatch.com/search.php?ostype=All&category=All&origin=All&basedon=All¬basedon=None&desktop=All&architecture=All&package=Snap&rolling=All&isosize=All&netinstall=All&language=All&defaultinit=All&status=Active#simple
- North Carolina State University Study 2017: http://dance.csc.ncsu.edu/papers/codaspy17.pdf
- Josh Triplett: https://lists.debian.org/debian-devel/2016/06/msg00287.html
- Debian Policy Manual: https://www.debian.org/doc/debian-policy/
- John Goerzen: https://changelog.complete.org/archives/date/2017/04/29
- "the new bullet-proof mechanism for app delivery and system updates": https://www.markshuttleworth.com/archives/1434
- xkcd comic: https://xkcd.com/927/
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Latest Cinnamon Desktop Releases with a Bold New Look
Just in time for the holidays, the developer of the Cinnamon desktop has shipped a new release to help spice up your eggnog with new features and a new look.
-
Armbian 24.11 Released with Expanded Hardware Support
If you've been waiting for Armbian to support OrangePi 5 Max and Radxa ROCK 5B+, the wait is over.
-
SUSE Renames Several Products for Better Name Recognition
SUSE has been a very powerful player in the European market, but it knows it must branch out to gain serious traction. Will a name change do the trick?
-
ESET Discovers New Linux Malware
WolfsBane is an all-in-one malware that has hit the Linux operating system and includes a dropper, a launcher, and a backdoor.
-
New Linux Kernel Patch Allows Forcing a CPU Mitigation
Even when CPU mitigations can consume precious CPU cycles, it might not be a bad idea to allow users to enable them, even if your machine isn't vulnerable.
-
Red Hat Enterprise Linux 9.5 Released
Notify your friends, loved ones, and colleagues that the latest version of RHEL is available with plenty of enhancements.
-
Linux Sees Massive Performance Increase from a Single Line of Code
With one line of code, Intel was able to increase the performance of the Linux kernel by 4,000 percent.
-
Fedora KDE Approved as an Official Spin
If you prefer the Plasma desktop environment and the Fedora distribution, you're in luck because there's now an official spin that is listed on the same level as the Fedora Workstation edition.
-
New Steam Client Ups the Ante for Linux
The latest release from Steam has some pretty cool tricks up its sleeve.
-
Gnome OS Transitioning Toward a General-Purpose Distro
If you're looking for the perfectly vanilla take on the Gnome desktop, Gnome OS might be for you.