Better Bash
Shell scripts from hell: Shebang
ByIn the beginning was the double pound sign and the exclamation mark – or at least shell scripts always start this way. The inventor, Dennis Ritchie, really didn’t know how much pain this was going to cause users.
When a user opens an interactive program, the shell unfolds more magic than you might imagine, especially if you happen to be talking about scripts. Bash is so omnipresent that people tend to forget it is also code that follows certain rules.
Bash has very little to do with accepting key presses. This job is handled by the terminal driver, which feeds a pseudo-terminal either locally at the console or via a detour through SSH or an X client. The pseudo-terminal passes the input on to the interactive shell, which sends a prompt to the terminal beforehand.
The shell waits until it receives an EOL and then interprets the string up to that point according to Bash syntax. If the string is an internal command – such as a while loop, an if condition, or an assignment that uses = – the shell executable can take the necessary action directly. This also applies to the many shell built-ins, such as ulimit or history, or any shell functions you defined yourself.
If none of these cases applies, Bash assumes the user wants to launch an external program, but it first needs to find the program. To do so, it iterates against the content of the PATH environmental variable, which it first separates by the delimiting colons.
Bash opens each element as a path and searches for a file with the execute flag set and with the name of the command input. Recent versions of Bash use a cache for this to avoid the need to search the physical filesystem for the complete path, but this doesn’t really change the approach just described.
External Binaries
If the command interpreter finds something, it initially leaves the rest of the job to the kernel by enabling the execve() call in a new process. It passes in the full path, the command name input, and – if command-line arguments exist – the arguments to the process. The kernel opens the file and checks the first 2 bytes. If they reveal that the file is a genuine binary (e.g., in ELF format), the kernel launches it directly. In the meantime, the shell waits for the program to terminate and then proceeds to process the next command.
#! All Change!
However, if the first two bytes are #!, the control flow takes a convoluted path back to linux/fs/binfmt_script.c in the kernel; Listing 1 shows some simplified code. This triggers a complicated mechanism:lines 10 through 19 delimit the #! line with an end marker and remove any white spaces that occur before this. Line 20 eliminates any blanks that directly follow the Shebang, because Unix creator Dennis Ritchie explicitly permitted this behavior in an email back in 1980.
Listing 1: Kernel Parses Shebang
01 static int 02 load_script(struct linux_binprm *bprm, struct pt_regs *regs) { 03 const char *i_arg, *i_name; 04 char *cp; 05 struct file *file; 06 07 if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!')) 08 return -ENOEXEC; 09 10 if ((cp = strchr(bprm->buf, '\n')) == NULL) 11 cp = bprm->buf+BINPRM_BUF_SIZE-1; 12 *cp = '\0'; 13 while (cp > bprm->buf) { 14 cp--; 15 if ((*cp == ' ') || (*cp == '\t')) 16 *cp = '\0'; 17 else 18 break; 19 } 20 for (cp = bprm->buf+2; (*cp == ' ') || (*cp == '\t'); cp++); 21 if (*cp == '\0') 22 return -ENOEXEC; /* No interpreter name found */ 23 i_name = cp; 24 i_arg = NULL; 25 for ( ; *cp && (*cp != ' ') && (*cp != '\t'); cp++); 26 while ((*cp == ' ') || (*cp == '\t')) 27 *cp++ = '\0'; 28 if (*cp) 29 i_arg = cp; 30 [...] 31 }
If trimming in line 21 fails to create a meaningful string, there is no interpreter name and the function reports an error in line 22. If this is not the case, the interpreter name is now available in i_name; this is typically /bin/sh. Lines 24 through 29 then take precisely one argument from the remaining line – if it exists – and store it in i_arg. The kernel ignores the rest of the first line.
Finally, the function calls itself recursively within the kernel using the command interpreter it extracted and reappends the original command name and the arguments. For example, if /tmp/runme contains an initial line of #!/foo/bar --myarg, PATH=/tmp is true, and if the user types runme alpha beta, the kernel actually calls execve("/foo/bar", "bar", "--myarg", "/tmp/runme", "alpha", "beta"). But if the kernel fails to determine an interpreter by following this procedure, Bash takes control of the execution, assumes that the file contains valid shell code, opens the file, and interprets its content.
Weird
You might be wondering why the kernel takes this roundabout approach. Ritchie’s response to this was that it allows shell scripts to be launched by exec() calls, that the process display and accounting show more intuitive names, and – this might surprise you – that you can assign set UID flags to shell scripts, too.
Unfortunately, this idea caused administrators no end of security worries. If a Unix user links to a set UID script that starts with #!/bin/sh, and cunningly names the link -i, the resulting call is /bin/sh -i, which gives the user a very convenient, interactive root shell. This explains why Linux dumps the privileges that result from additional flags in the extra Shebang loop.
At the same time, there is also an attack vector for a race condition in which the file changes between parsing and execution. All told, this mechanism – no matter how well-meant it was on Ritchie’s part – has mainly caused confusion. In a survey, Sven Mascheck investigated this aspect in 48 Unix derivatives, finding very little common ground, but a number of vulnerabilities.
Issue 269/2023
Buy this issue as a PDF
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Find SysAdmin Jobs
News
-
Kubuntu Focus Announces XE Gen 2 Linux Laptop
Another Kubuntu-based laptop has arrived to be your next ultra-portable powerhouse with a Linux heart.
-
MNT Seeks Financial Backing for New Seven-Inch Linux Laptop
MNT Pocket Reform is a tiny laptop that is modular, upgradable, recyclable, reusable, and ships with Debian Linux.
-
Ubuntu Flatpak Remix Adds Flatpak Support Preinstalled
If you're looking for a version of Ubuntu that includes Flatpak support out of the box, there's one clear option.
-
Gnome 44 Release Candidate Now Available
The Gnome 44 release candidate has officially arrived and adds a few changes into the mix.
-
Flathub Vying to Become the Standard Linux App Store
If the Flathub team has any say in the matter, their product will become the default tool for installing Linux apps in 2023.
-
Debian 12 to Ship with KDE Plasma 5.27
The Debian development team has shifted to the latest version of KDE for their testing branch.
-
Planet Computers Launches ARM-based Linux Desktop PCs
The firm that originally released a line of mobile keyboards has taken a different direction and has developed a new line of out-of-the-box mini Linux desktop computers.
-
Ubuntu No Longer Shipping with Flatpak
In a move that probably won’t come as a shock to many, Ubuntu and all of its official spins will no longer ship with Flatpak installed.
-
openSUSE Leap 15.5 Beta Now Available
The final version of the Leap 15 series of openSUSE is available for beta testing and offers only new software versions.
-
Linux Kernel 6.2 Released with New Hardware Support
Find out what's new in the most recent release from Linus Torvalds and the Linux kernel team.