Cleaning asm Out of syscall Slow-Path Selection Code

Linus Torvalds pointed out an issue with the x86-64 and 32-bit system call code. He said the code that chose which slow path to execute for any given system call was a mish-mash nightmare of Frankensteinian proportions, mixing assembler code with C code in complex ways that were vile and grotesque.

He posted some code that laid the groundwork for one day replacing the whole mess with crisp, clean C code. He remarked, "I didn't bother trying to clean up the 32-bit code similarly (no test-cases, and trust me, if you get this wrong, it will fail spectacularly but in very subtle and hard-to-debug ways)." He added that his code "is actually tested, and seems to work (including limited testing with strace, gdb etc), and while it adds a few more lines than it removes, the removed lines are mostly asm, and added lines are C."

His vision for the future of this particular batch of code, he said, was to address each case of assembler code one by one, until they were all replaced with calls to a single wrapper function. Then, that wrapper function could be replaced with some well-written C code.

Al Viro pointed out some additional technical issues; in particular, he pointed out some problems they should expect to encounter when porting these changes to 32-bit kernels. Linus remarked, "Yes, the 32-bit code I didn't want to touch, partly because I no longer have a test-case. And it does end up having some more interesting cases."

H. Peter Anvin was very excited to see Linus working on this. He said, "this code is incredibly ugly and getting it cleaned up would really, really help, of course." Peter Zijlstra said he also was thrilled that Linus was doing this stuff. He said, "its a nice cleanup, entry.S always hurts my brain."

Al continued pointing out deep assembler horrors and trying to indicate ways to fix them. He, Linus, Andy Lutomirski, and others discussed the technical details. Part of the difficulty remained the open question of exactly how much to even bother trying to fix versus the quantity of soul-death each portion of such fixes would cause.

At one point (to give a semi-readable example of the conversation), Al said, "The problem with validation is that we'll suddenly become sensitive to hard-to-estimate pile of hardware bugs ;-/ E.g. which AMD-specific errata is that comment in entry_64.S about? The one I kinda-sorta remember is Intel-specific, and that was about uncanonical RIP; looking for AMD one has turned #353 (with suggested workaround being 'have bit 16 set in whatever you put into R11'), but I've no idea whether that's the only potential issue there … ."

The discussion went on for a bit and was inscrutably technical. The difficulties of eradicating assembly language from the Linux kernel seem to be immense. Ultimately, Linus, Al, and the rest will probably not try to get rid of assembler code absolutely everywhere. In some cases, the speed advantage of assembly language is highly significant, and other cases involve initialization code that occurs very early in the boot process, where very specific opcodes are expected and required.

The whole issue is reminiscent of the effort to get rid of the Big Kernel Lock (BKL) some years back. In that case, it had proven extremely difficult even to approach the problem. The BKL was everywhere! Rewriting it to be smaller, sleeker, and more targeted was simply not possible with things as they stood.

The solution ended up being to push the BKL code outward from where it was defined and duplicate that code wherever it might be needed, instead of keeping it as a single centralized function call. That project alone proved time consuming and difficult. Once the BKL had been thoroughly distributed, however, each separate case could be examined and recoded to perform only the locking behavior that was actually needed. In many cases, the desired locks were simple and could be reimplemented as a new semaphore or similar type of lock that could be called from diverse portions of the kernel. But, as more and more occurrences of the BKL were replaced in this way, the remaining cases proved more and more difficult; and so, it was years before the last of them could be rooted out and recoded with specialized locking mechanisms that met the particular needs of those parts of the kernel.

The current situation with assembler in the syscall slow-path selection code is similar. Linus wants to address the easy cases consistently and get them out of the way, then tackle the progressively more difficult cases as folks identify possible solutions for them. But, as with the BKL, it seems as if this may turn out to be a long slog.