Paul Frields's Update and Report on Fedora August 2008 Intrusion on the fedora-announce-list reads like a detective novel. It all started on August 12, 2008, when a cron job on a Fedora host reported an error. While reviewing the logs, Fedora admins found a change in the package complement that no one could explain. On short notice, the changes turned out to be tampering by an intruder. The project notified the community of the break-in and promptly pulled the server off the net.

It's now become clear how the rogue entered the server structure: he used no hacker tools, but simply authenticated himself using a copy of an SSH private key that was not passphrase-protected. The key belonged to a Fedora admin and in the log entries it showed that the intruder also cracked or knew the admin's password. How the intruder got to the SSH private key, however, nobody knows.

One of the compromised computers also contained the Fedora package signing key. The intruder created modified versions of the two packages OpenSSH and RPM to get to user passwords and, eventually, the password for the package signing key. Had he been successful, he could have introduced fraudulent packages into the repository. Fortunately the investigation found that
Fedora admins discovered the modified packages before anyone could use the server for package signing.

To mitigate any risk of this ever happening again, the Fedora project quickly rebuilt their entire infrastructure, generated new package signing keys and came up with a new security policy. In a week the most essential systems were back to normal and all admins got new SSH keys. A new repo security policy also required Fedora admin groups to use passphrases on their private keys, a definite break from the past.

Frields assured users that no compromised packages were ever delivered as a result of this break-in, either from the master repository or the mirror sites. He went on to thank the Red Hat security response team for their timely assistance.

Comments

Server

Susan Jul 23, 2009 3:16pm GMT

I recently came accross your blog and have been reading along. I thought I would leave my first comment. I dont know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.

Susan

<a href="http://8080proxy.com">http://8080proxy.com</a>

Debian timing?

Mackenzie Apr 07, 2009 8:55pm GMT

Given the timing, I wonder if he didn't have his passphrase-less SSH key stored on a machine to which a Debian/Ubuntu user had SSH access. This was *right* after the Debian OpenSSL thing, and a lot of people thought "that's a Debian problem, I'm fine" ignoring that they had Debian-sourced SSH keys on their systems from other users.

SELinux is still secure

loupgaroublond Apr 06, 2009 5:07pm GMT

This kind of attack can't be mitigated with SELinux. The credentials of a user with administrator access was compromised. Don't start spreading FUD that SELinux is insecure, when no other security layer would have protected Fedora.

Re: Corrections

Britta Wuelfing Apr 06, 2009 3:50pm GMT

Sorry about the mistake, it got in with the translation. Originally Marcel described it differently, and we've changed that paragraph according to his text. The true crime story obviously was a temptation to make up a slightly different novel. We apologize!

Fedora compromised. Ironic since it is presumably SELINUX protected?

pgmer6809 Apr 05, 2009 11:52pm GMT

I have been running a Centos based server for years, but avoided Fedora, in part because I cant stand the hassle that SELINUX imposes by default. I find it very ironic that the one distro which pushes SELINUX is the one that was compromised.

pgmer6809

Corrections.

Paul W. Frields Apr 03, 2009 5:00pm GMT

In the third paragraph, you appear to refer to the Fedora package signing key, claiming that the intruder "used this key to create modified versions of OpenSSH and RPM." This is false, and our announcement plainly states that our investigation has supported that the intruder did not gain access to this key. Moreover, even had the intruder gained such access, he would need a different passphrase to use it to fraudulently sign packages. Again, our investigation showed this was not the case.

I still love fedora

lily Apr 02, 2009 6:29pm GMT

I really dont care. I love fedora and all its features:
The cron may report an error but that hardly matters to my servers

was reading how to Set up RPM Fusion with Fedora to shore up multimedia support
http://www.techunits.com/linux/list/fedora

Admin Magazine

(Update:) Fedora: Chronicle of a Server Break-in