When a user opens an interactive program, the shell unfolds more magic than you might imagine, especially if you happen to be talking about scripts. Bash is so omnipresent that people tend to forget it is also code that follows certain rules.

Bash has very little to do with accepting key presses. This job is handled by the terminal driver, which feeds a pseudo-terminal either locally at the console or via a detour through SSH or an X client. The pseudo-terminal passes the input on to the interactive shell, which sends a prompt to the terminal beforehand.

The shell waits until it receives an EOL and then interprets the string up to that point according to Bash syntax. If the string is an internal command – such as a while loop, an if condition, or an assignment that uses = – the shell executable can take the necessary action directly. This also applies to the many shell built-ins, such as ulimit or history, or any shell functions you defined yourself.

If none of these cases applies, Bash assumes the user wants to launch an external program, but it first needs to find the program. To do so, it iterates against the content of the PATH environmental variable, which it first separates by the delimiting colons.

Bash opens each element as a path and searches for a file with the execute flag set and with the name of the command input. Recent versions of Bash use a cache for this to avoid the need to search the physical filesystem for the complete path, but this doesn’t really change the approach just described.

External Binaries

If the command interpreter finds something, it initially leaves the rest of the job to the kernel by enabling the execve() call in a new process. It passes in the full path, the command name input, and – if command-line arguments exist – the arguments to the process. The kernel opens the file and checks the first 2 bytes. If they reveal that the file is a genuine binary (e.g., in ELF format), the kernel launches it directly. In the meantime, the shell waits for the program to terminate and then proceeds to process the next command.

#! All Change!

However, if the first two bytes are #!, the control flow takes a convoluted path back to linux/fs/binfmt_script.c in the kernel; Listing 1 shows some simplified code. This triggers a complicated mechanism:lines 10 through 19 delimit the #! line with an end marker and remove any white spaces that occur before this. Line 20 eliminates any blanks that directly follow the Shebang, because Unix creator Dennis Ritchie explicitly permitted this behavior in an email back in 1980.

Listing 1: Kernel Parses Shebang

01 static int
02 load_script(struct linux_binprm *bprm, struct pt_regs *regs) {
03   const char *i_arg, *i_name;
04   char *cp;
05   struct file *file;
06 
07   if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!'))
08   return -ENOEXEC;
09 
10   if ((cp = strchr(bprm->buf, '\n')) == NULL)
11   cp = bprm->buf+BINPRM_BUF_SIZE-1;
12   *cp = '\0';
13   while (cp > bprm->buf) {
14     cp--;
15     if ((*cp == ' ') || (*cp == '\t'))
16     *cp = '\0';
17     else
18     break;
19   }
20   for (cp = bprm->buf+2; (*cp == ' ') || (*cp == '\t'); cp++);
21   if (*cp == '\0')
22   return -ENOEXEC; /* No interpreter name found */
23   i_name = cp;
24   i_arg = NULL;
25   for ( ; *cp && (*cp != ' ') && (*cp != '\t'); cp++);
26   while ((*cp == ' ') || (*cp == '\t'))
27   *cp++ = '\0';
28   if (*cp)
29   i_arg = cp;
30   [...]
31 }

If trimming in line 21 fails to create a meaningful string, there is no interpreter name and the function reports an error in line 22. If this is not the case, the interpreter name is now available in i_name; this is typically /bin/sh. Lines 24 through 29 then take precisely one argument from the remaining line – if it exists – and store it in i_arg. The kernel ignores the rest of the first line.

Finally, the function calls itself recursively within the kernel using the command interpreter it extracted and reappends the original command name and the arguments. For example, if /tmp/runme contains an initial line of #!/foo/bar --myarg, PATH=/tmp is true, and if the user types runme alpha beta, the kernel actually calls execve("/foo/bar", "bar", "--myarg", "/tmp/runme", "alpha", "beta"). But if the kernel fails to determine an interpreter by following this procedure, Bash takes control of the execution, assumes that the file contains valid shell code, opens the file, and interprets its content.

Weird

You might be wondering why the kernel takes this roundabout approach. Ritchie’s response to this was that it allows shell scripts to be launched by exec() calls, that the process display and accounting show more intuitive names, and – this might surprise you – that you can assign set UID flags to shell scripts, too.

Unfortunately, this idea caused administrators no end of security worries. If a Unix user links to a set UID script that starts with #!/bin/sh, and cunningly names the link -i, the resulting call is /bin/sh -i, which gives the user a very convenient, interactive root shell. This explains why Linux dumps the privileges that result from additional flags in the extra Shebang loop.

At the same time, there is also an attack vector for a race condition in which the file changes between parsing and execution. All told, this mechanism – no matter how well-meant it was on Ritchie’s part – has mainly caused confusion. In a survey, Sven Mascheck investigated this aspect in 48 Unix derivatives, finding very little common ground, but a number of vulnerabilities.