Better Bash
Shell scripts from hell: Shebang
ByIn the beginning was the double pound sign and the exclamation mark – or at least shell scripts always start this way. The inventor, Dennis Ritchie, really didn’t know how much pain this was going to cause users.
When a user opens an interactive program, the shell unfolds more magic than you might imagine, especially if you happen to be talking about scripts. Bash is so omnipresent that people tend to forget it is also code that follows certain rules.
Bash has very little to do with accepting key presses. This job is handled by the terminal driver, which feeds a pseudo-terminal either locally at the console or via a detour through SSH or an X client. The pseudo-terminal passes the input on to the interactive shell, which sends a prompt to the terminal beforehand.
The shell waits until it receives an EOL and then interprets the string up to that point according to Bash syntax. If the string is an internal command – such as a while loop, an if condition, or an assignment that uses = – the shell executable can take the necessary action directly. This also applies to the many shell built-ins, such as ulimit or history, or any shell functions you defined yourself.
If none of these cases applies, Bash assumes the user wants to launch an external program, but it first needs to find the program. To do so, it iterates against the content of the PATH environmental variable, which it first separates by the delimiting colons.
Bash opens each element as a path and searches for a file with the execute flag set and with the name of the command input. Recent versions of Bash use a cache for this to avoid the need to search the physical filesystem for the complete path, but this doesn’t really change the approach just described.
External Binaries
If the command interpreter finds something, it initially leaves the rest of the job to the kernel by enabling the execve() call in a new process. It passes in the full path, the command name input, and – if command-line arguments exist – the arguments to the process. The kernel opens the file and checks the first 2 bytes. If they reveal that the file is a genuine binary (e.g., in ELF format), the kernel launches it directly. In the meantime, the shell waits for the program to terminate and then proceeds to process the next command.
#! All Change!
However, if the first two bytes are #!, the control flow takes a convoluted path back to linux/fs/binfmt_script.c in the kernel; Listing 1 shows some simplified code. This triggers a complicated mechanism:lines 10 through 19 delimit the #! line with an end marker and remove any white spaces that occur before this. Line 20 eliminates any blanks that directly follow the Shebang, because Unix creator Dennis Ritchie explicitly permitted this behavior in an email back in 1980.
Listing 1: Kernel Parses Shebang
01 static int 02 load_script(struct linux_binprm *bprm, struct pt_regs *regs) { 03 const char *i_arg, *i_name; 04 char *cp; 05 struct file *file; 06 07 if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!')) 08 return -ENOEXEC; 09 10 if ((cp = strchr(bprm->buf, '\n')) == NULL) 11 cp = bprm->buf+BINPRM_BUF_SIZE-1; 12 *cp = '\0'; 13 while (cp > bprm->buf) { 14 cp--; 15 if ((*cp == ' ') || (*cp == '\t')) 16 *cp = '\0'; 17 else 18 break; 19 } 20 for (cp = bprm->buf+2; (*cp == ' ') || (*cp == '\t'); cp++); 21 if (*cp == '\0') 22 return -ENOEXEC; /* No interpreter name found */ 23 i_name = cp; 24 i_arg = NULL; 25 for ( ; *cp && (*cp != ' ') && (*cp != '\t'); cp++); 26 while ((*cp == ' ') || (*cp == '\t')) 27 *cp++ = '\0'; 28 if (*cp) 29 i_arg = cp; 30 [...] 31 }
If trimming in line 21 fails to create a meaningful string, there is no interpreter name and the function reports an error in line 22. If this is not the case, the interpreter name is now available in i_name; this is typically /bin/sh. Lines 24 through 29 then take precisely one argument from the remaining line – if it exists – and store it in i_arg. The kernel ignores the rest of the first line.
Finally, the function calls itself recursively within the kernel using the command interpreter it extracted and reappends the original command name and the arguments. For example, if /tmp/runme contains an initial line of #!/foo/bar --myarg, PATH=/tmp is true, and if the user types runme alpha beta, the kernel actually calls execve("/foo/bar", "bar", "--myarg", "/tmp/runme", "alpha", "beta"). But if the kernel fails to determine an interpreter by following this procedure, Bash takes control of the execution, assumes that the file contains valid shell code, opens the file, and interprets its content.
Weird
You might be wondering why the kernel takes this roundabout approach. Ritchie’s response to this was that it allows shell scripts to be launched by exec() calls, that the process display and accounting show more intuitive names, and – this might surprise you – that you can assign set UID flags to shell scripts, too.
Unfortunately, this idea caused administrators no end of security worries. If a Unix user links to a set UID script that starts with #!/bin/sh, and cunningly names the link -i, the resulting call is /bin/sh -i, which gives the user a very convenient, interactive root shell. This explains why Linux dumps the privileges that result from additional flags in the extra Shebang loop.
At the same time, there is also an attack vector for a race condition in which the file changes between parsing and execution. All told, this mechanism – no matter how well-meant it was on Ritchie’s part – has mainly caused confusion. In a survey, Sven Mascheck investigated this aspect in 48 Unix derivatives, finding very little common ground, but a number of vulnerabilities.
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
So Long Neofetch and Thanks for the Info
Today is a day that every Linux user who enjoys bragging about their system(s) will mourn, as Neofetch has come to an end.
-
Ubuntu 24.04 Comes with a “Flaw"
If you're thinking you might want to upgrade from your current Ubuntu release to the latest, there's something you might want to consider before doing so.
-
Canonical Releases Ubuntu 24.04
After a brief pause because of the XZ vulnerability, Ubuntu 24.04 is now available for install.
-
Linux Servers Targeted by Akira Ransomware
A group of bad actors who have already extorted $42 million have their sights set on the Linux platform.
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.