Putting the Cart Before the Horse

Rick P. Edgecombe asked Linus Torvalds for some advice about "ecosystem compatibility." In fact, Rick was concerned about features that were becoming very popular elsewhere, but that were not yet included in the Linux kernel. Specifically, he was interested in Intel's hardware-based support for Control-flow Enforcement Technology (CET). This creates a hidden memory stack that prevents certain attacks that might otherwise deduce the memory location containing the next CPU opcodes to execute.

It's a real danger! The kernel's program is just a bunch of opcodes in memory, with a hardware register that continually points to the next one to be executed. Function calls and goto statements are always updating where that register points. If attackers can predict that, they might be able to replace those opcodes with their own hostile code.

However, Rick wasn't concerned about CET itself so much as the interactions between other tools and the kernel. Specifically, he said:

"The issues all have a root cause of support for CET in tools spreading widely while kernel support was still in development. This has lead to:

1. Some existing binaries (node.js, PyPy, CRIU) that will break when glibc updates to use the kernel CET APIs.

2. GCC C++ exception stack unwinding code expecting old development versions of the kernel ABI."

Rick pointed out that "once there is kernel support, glibc plans to immediately update in such a way that some existing distro binaries will break against it."

He also said that meanwhile, "The gcc CET support has preceded the kernel changes and the unwinding code assumes things about the kernel shadow stack signal frame ABI that have changed over the course of CET kernel development."

Rick was trying to come up with a way to cushion these future incompatibilities somewhat, and he asked Linus for advice.

Linus, however, gave an elaborate shrug. He said:

"I'm disgusted by glibc being willing to just upgrade and break existing binaries and take the 'you shouldn't upgrade glibc if you have old binaries' approach.

"But hey, I guess that's part for the course for glibc, and there's nothing I can do about that.

"But yes, once people complain, I'll just make sure that old binaries continue to work, and at that point the glibc and tooling people will presumably have to fix their broken situation to get CET at all.

"Because no, the kernel doesn't enable CET if it breaks binaries. That's how we roll."

Florian Weimer replied:

"We've been in this position for years. Every time we use a new system call to implement existing functionality in glibc, some applications break. Mostly due to seccomp filters. They break even if there would be no observable differences for applictions in the way the new system calls would be invoked if the seccomp filter wouldn't block them.

"I proposed a new ENOSYS handshake between userspace and kernel to reduce the amount of breakage (but not all of it). Senior kernel developers rejected it, so we didn't implement it in glibc.

[...]

"Instead, we work with distributions and upstreams to make sure the applications are ready before the next distribution glibc update. Fortunately, there seems to be a pretty broad overlap between seccomp-using applications and applications with frequent, more-or-less mandatory updates, so the transition periods are relatively short. You didn't seem to have noticed, so maybe we aren't doing such a bad job after all."

There was no "official" conclusion during the discussion, although the kernel developers did get into some serious detail about exactly what breakage to expect, who would experience it, and where any potential fixes might go.

Ultimately, Linus probably does mean business in terms of ensuring that user binaries don't break, even if the kernel must accomplish this at the expense of inconveniencing other tools and other groups of developers. Breaking existing binaries and allowing known security holes to persist in the kernel are probably two of the least tolerated things in all of kernel development. Linus would rather accept a huge loss of efficiency and prune off whole branches of kernel features rather than tolerate a security bug or break an existing user binary.