Anatomy of a kernel attack


Article from Issue 250/2021

A vulnerability in an operating system kernel is a security nightmare. This article analyzes some well known kernel security problems, explains how they are exploited, and gives real-life examples of attacks that used these time-honored techniques.

Some security issues remind me of Groundhog Day – they just keep coming back. One example of a problem that won't go away is buffer overflow, which was first described in 1972 [1], got a fair bit of attention in 1996 in the oft-quoted Phrack e-zine article "Smashing the Stack for Fun and Profit" [2], and is still one of the most prevalent programming mistakes that can lead to a code injection. And it isn't due to the lack of media coverage that integer overflows are still around – long after they caused the Ariane 5 rocket explosion [3] and the massive Stagefright security issue in Android. The fact is that many of the same few problems continue to turn up, and most could have been prevented by code analysis [4] and defensive programming.

This article takes a close look at some of the techniques attackers use to crack the Linux kernel.

Stein and Shot

You can perform a simple test at home for a graphic example of a buffer overflow: Grab a shot glass and a comfortably sized Munich beer stein, fill the stein to the top, then pour all of its contents into the shot glass. Take note of the overflow. To prevent this situation, the bartender must compare the source's size to the size of the destination buffer.

A buffer overflow simply sends too much data into a too small of a buffer. In the case of a data buffer, the overflow does not just pour out onto the floor but actually overwrites the adjacent data structures. If the buffer is on the stack, the overflow overwrites other data on the stack – which might be other variables, constants, and data used to control an instruction pointer, such as the return address. The return address is used by the final ret command in any subroutine, so that the CPU can find the next instruction to execute. A change to this address can alter the control flow of the process.

In many buffer overflow attacks, the instruction pointer is set to a memory region the attacker controls and is thus able to fill with the code he wants to execute; however, an attacker can also set the instruction pointer to a library function, such as a call to libc, and provide this function with a properly crafted stack in order to make the library execute the instructions for the attack. The case of sending the attack to libc is called a "return-to-libc" attack, and it is used as a workaround on CPUs supporting non-executable flags for memory regions.

The non-executable flag was introduced because the memory region attackers control is usually the stack, which means that the data sent to overflow the buffer could also provide the code to be executed. If the stack is marked as a "data" region (i.e., non-executable), the code on the stack would fail. However, calling a system-provided function, i.e. in libc, avoids this problem – this workaround is known as "return-to-libc".

Setting the Stage

Listing 1 has a simple example of a vulnerable C program. To test it, compile it with

gcc -o buf -mpreferred-stack-boundary=2 buf.c

Listing 1

Buffer Overflow Example

01 #include "stdio.h"
02 void read_and_print (void)
03   {
04     char text[160];
05     gets(text);
06     printf("Your input:\n%s\n", text);
07   }
09 int main ()
10   {
11     read_and_print();
12     return 0;
13   }

which, even on a 64-bit system, configures data on the stack to be aligned at two bytes.

A quick test run with the correct size of the input string won't show anything suspicious; however, entering more than 160 characters will mostly likely result in a segmentation fault. If typing 160+ characters isn't your favorite way to pass the time, Perl might offer a helping hand:

perl -e 'print "A"x162' | ./buf

A side effect of this rather uniform input is that it is easy to spot when looking at the stack with a debugger such as gdb. The setup is shown in Listing 2.

Listing 2

Setting Up for gbd

01 (gdb) list
02 4         {
03 5           char text[160];
04 6
05 7           gets(text);
06 8           printf("Your input:\n%s\n", text);
07 9         }
08 10
09 11      int main ()
10 12        {
11 13          read_and_print();
12 (gdb) break 8
13 Breakpoint 1 at 0x804839b: file buf.c, line 8.
14 (gdb) display/200b $esp

Once the program runs, with

run < <(perl -e 'print "A"x162')

in gdb, it will break in line 8 of the C code (Listing 2, line 12) and display the stack contents. The 162 "A"s (0x41) are obvious (Figure 1). Next to the "A"s, the return address is stored – and easily discovered once main() is disassembled (Figure 2). The return address is stored according to the architecture endianess, hence the bytes seem to be swapped to 0xbb 0x83 0x04 0x08.

Figure 1: Stack contents after sending a few bytes too many. The 162 "A"s (0x41) stick out, null terminated and aligned to two bytes, followed by the return address.
Figure 2: main() disassembled: The subroutine call at main+3 is obvious, as is the subsequent command at main+8.

Go Back

A simple first test would just set the return address to main+3 and rerun the function again. Note that a string in C is null terminated, and I have a stack alignment of two bytes (Figure 2); thus, to attack, I now need to send 164 "A"s plus the new return address:

run < <(perl -e 'print "A"x164 ."\xb6\x83\x04\x08"')

You will notice that the breakpoint is triggered twice, which should not happen under regular circumstances. The process then crashes because the stack is corrupted.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Kernel Protection

    Security vulnerabilities in the kernel often remain undetected. The kernel hacker initiative, Kernel Self-Protection, promotes safe programming techniques to keep attackers off the network, and, if they do slip through the net, mitigate the consequences.

  • Rdesktop: Remote Control with Security Holes

    Security researchers iDefense have disclosed three vulnerabilities in the Rdesktop Remote Client.

  • Security and SOHO Routers

    Home and small office networks typically place their security in the hands of an inexpensive device that serves as a router, DHCP server, firewall, and wireless hotspot. How secure are these SOHO router devices? We're glad you asked …

  • Vulnerabilities in Xine-Lib and Mplayer

    Vulnerabilities have been discovered in two major media players for Linux. A Xine-Lib vulnerability also affects Mplayer.

  • Apache 2.2.13 with Overflow Protection

    With Apache 2.2.13, developers have closed security holes in the popular webserver.

comments powered by Disqus
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters

Support Our Work

Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.

Learn More