Insecure updates are the rule, not the exception
Mixed Signals
Kurt looks at the practice of code signing and examines why so few upstream open source projects actually do it.
First, the good news: Most major Linux vendors sign their software and source code packages, thereby allowing end users and administrators to verify that the code has indeed originated from the vendor and hasn't been modified or tampered with after being signed.
Most tools like Yum and RPM check package signatures by default and will refuse to install or upgrade packages unless they are properly signed (which you can override manually). So, why am I worried about upstream code signing of updates? Because very few open source projects actually sign their code properly, if at all. This means that if an attacker breaks into a distribution site, they can modify existing source code packages or upload new ones with malicious code. Users cannot easily detect a problem unless they have a copy of the code for comparison or they compare the changes in the new version against an old version. And yes, people do break into major sites (e.g., the Linode hack earlier this year) [1].
Why People Don't Sign Code
Why don't upstream open source projects sign their code properly? Doing so would allow vendors like Debian and Red Hat and end users to verify easily that the code was signed by the project in question. There are several reasons: the first is that signing code correctly, even minimally, is a chore and requires some setup. But, more importantly, it requires ongoing discipline; you must protect the signing keys forever, you must sign and verify the signed code, and you must handle key management. Additionally, code signing only proves that a file containing certain content was signed; it does nothing to attest to the quality of the code (everyone has security flaws in their software). For all you know, the upstream project has taken code contributions containing a back door.
The third and, in my opinion, biggest reason is that unsigned code and all the problems that come with it don't cost the code author anything. Virtually all the costs of distributing malicious code are borne by end users; the author rarely incurs any cost or penalty.
A perfect example is the Social Media Widget incident [2]. The company responsible for this WordPress plugin outsourced development to a company that inserted malicious PHP code. Fortunately (as far as is known), the code inserted just displayed an ad and didn't attack the server or users' systems. The WordPress team accepted this explanation and reinstated the plugin. So, the cost in terms of money, time, and effort was basically zero for the Social Media Widget plugin.
How to Sign Code
Signing code is easy; you just use a tool like GPG and create a signature, usually external, so you have foo.tar.gz
and foo.tar.gz.asc
. You then provide the signature file, typically by putting it into the same directory or download system as the files. Then, you need to make the signing key available in a secure manner. You can get your key signed, but the chances of getting it signed by widely trusted keys (e.g., vendor distribution keys) are minimal. Your best bet is to make the GPG signing key available on the project website via HTTPS so that users have a reasonable chance of downloading it without an attacker modifying or spoofing it. Simple, right?
Well, you also have to handle key security. The signing key must be as secure as possible, which ideally means placing it on a hardware module that is external to your computer and only plugging it in and using it as needed, thereby minimizing the window of opportunity for an attacker to compromise it.
In a perfect world, your signing server is offline so that attacks cannot be launched against it. The cheapest way to put your signing key into a secure hardware module is to use an OpenPGP-compatible card [3]. These cards are not too expensive; Kernel Concepts sells them for about US$ 20 [4], and you'll also need to buy a reader. You'll also need to choose a key lifespan. Note that keys should expire. This ensures that, say, 20 years from now, when attackers can crack 4096-bit keys on their quantum-enabled smartphones, the key you created cannot be used for nefarious purposes.
Attacks Against Signed Code
So, now the code is signed, and everything is good, right? Nope. Even if packages are signed properly and clients verify the signatures properly (which won't happen unless the process is automated and fails to close when the signature is invalid, as with Yum and RPM), a number of attacks are still possible. The simplest and most direct attack is to provide older signed packages that have known security issues. This attack is especially easy if the file signature doesn't also confirm that the filename has not been changed. Do you ever read ChangeLog
or VERSION
to confirm the package version?
Another attack is to prevent any updates of software by deleting new versions, or making downloads incredibly slow, or by feeding infinitely large files to clients. Unfortunately, this approach is more effective than you'd think. For example, when was the last time your update software warned you that no updates had been installed for a few months?
Addressing such attacks is not simple and depends on the client having information available to check. For example, Yum downloads a copy of the repository data and then gets the packages, which prevents replaying of older packages (unless the user forces a downgrade manually) and also lets the client know whether updates should be available. Note, however, that if the attacker can modify the RPMs, chances are they can mangle the repo data as well.
For software that doesn't take this approach, such as WordPress, I suggest keeping a copy of the software that you download so you can compare newer versions to it and make sure that the version you're downloading is actually newer than what you have.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
New Steam Client Ups the Ante for Linux
The latest release from Steam has some pretty cool tricks up its sleeve.
-
Gnome OS Transitioning Toward a General-Purpose Distro
If you're looking for the perfectly vanilla take on the Gnome desktop, Gnome OS might be for you.
-
Fedora 41 Released with New Features
If you're a Fedora fan or just looking for a Linux distribution to help you migrate from Windows, Fedora 41 might be just the ticket.
-
AlmaLinux OS Kitten 10 Gives Power Users a Sneak Preview
If you're looking to kick the tires of AlmaLinux's upstream version, the developers have a purrfect solution.
-
Gnome 47.1 Released with a Few Fixes
The latest release of the Gnome desktop is all about fixing a few nagging issues and not about bringing new features into the mix.
-
System76 Unveils an Ampere-Powered Thelio Desktop
If you're looking for a new desktop system for developing autonomous driving and software-defined vehicle solutions. System76 has you covered.
-
VirtualBox 7.1.4 Includes Initial Support for Linux kernel 6.12
The latest version of VirtualBox has arrived and it not only adds initial support for kernel 6.12 but another feature that will make using the virtual machine tool much easier.
-
New Slimbook EVO with Raw AMD Ryzen Power
If you're looking for serious power in a 14" ultrabook that is powered by Linux, Slimbook has just the thing for you.
-
The Gnome Foundation Struggling to Stay Afloat
The foundation behind the Gnome desktop environment is having to go through some serious belt-tightening due to continued financial problems.
-
Thousands of Linux Servers Infected with Stealth Malware Since 2021
Perfctl is capable of remaining undetected, which makes it dangerous and hard to mitigate.