Slippery Shell

Developer Andy Chu is currently working on two construction sites at the same time: the Oil shell (OSH) and the Oil language. His work on OSH is already quite advanced, but, as of version 0.6.pre20, it only supports a subset of the Bash constructs. The Oil language, on the other hand, is still a work in progress.

OSH

A shell is a small program that uses text commands to control the system. Many shells also let you write scripts to automate processes. One of the best known and most common shells is the Bourne-Again Shell (Bash) [1]. The Oil shell is a Unix shell that is compatible with Bash.

According to Chu, OSH is already capable of processing the abuild shell script, which is over 2,500 lines long and builds the packages for the Alpine Linux distribution. OSH can also set up an Ubuntu base system via debootstrap and chroot into it.

Unlike other shells, OSH shell scripts not only evaluate command by command, but fully load some command sequences, including mathematical expressions; this makes it possible to identify bugs at an earlier stage. Furthermore, OSH offers automatic completion at the command line (Figure 1).

Figure 1: If you press Tab at this point in OSH, the shell automatically completes $HOSTNAME.

Test Drive

To try out OSH, download the latest release from the Oil project's homepage [2] and unpack the archive. To build the shell, you need a C compiler, make, and the developer package for the Readline library. If these components are missing on your system, you can install them on Ubuntu using the command shown in line 1 of Listing 1. Then change to the Oil shell's source directory, compile the program, and set it up on the system (lines 3 to 5). You can use the Oil shell much like Bash and apply it to shell scripts:

01 $ sudo apt install build-essential libreadline-dev
02 [...]
03 $ ./configure
04 $ make
05 $ sudo ./install

$ osh -c 'echo hello world!'

The call to osh is only a symbolic link to the actual program, which goes by the name of oil.vm. If you start it via osh, the shell will also log on under this name.

Chu wrote OSH in Python. He chose this language, because it helped him obtain results quickly. The disadvantage is a significantly lower speed compared to other shells. OSH runs in a virtual machine (VM), the OVM. This is a modified version of the CPython VM, Python's official reference implementation.

The Python 2 VM is currently used, but its support will expire in January 2020. The VM's code is now generated by the OPy bytecode compiler. The source code for the current OSH development version is available from GitHub [3].

The Oil Language

Parallel to OSH, Chu is developing the Oil language (Oil for short), a completely new scripting language, which is intended to eliminate some of the inconveniences of previous Unix shells and thus make work easier for shell script developers. For example, the expression x=1 leads to the same result as x = 1. Furthermore, Oil's goal is to be easier to learn, write, and debug; do more than the Bash can do; and have a more consistent grammar.

According to Chu, Oil's specification and structure is largely fixed, but the documentation is only rudimentary. A shell of this kind would not execute every command directly one after the other, but first load in a script completely and store its structure in the main memory. Thanks to this static parsing, the shell is able to detect typos and syntax errors at an early stage.

The shell also stores the script's structure in RAM in a special data structure known as a syntax tree. OSH constructs this syntax tree in such a smart way that you can generate the original script from it. With a lossless syntax tree like this, you should be able to detect errors more easily.

Oil offers shell-like functions that are defined with the keyword proc, as well as additional conventional functions introduced by the func keyword. The latter are similar to the Python and JavaScript functions and can accept and return more complex data structures. Chu wants to improve the shell syntax for the manipulation of strings.

Oil encloses code blocks in curly brackets, so the following statement

if ... then ... fi

converts to a shorter counterpart

if ... { ... }

Oil also supports arrays familiar from other programming languages that use square brackets for initialization:

Hello = ['Hello' 'World']

Bash scripts should be easy to translate into Oil. Chu is already working on a suitable translator, which will convert OSH scripts into corresponding Oil code. At present, however, this only exists as a proof of concept prototype.

An example of the results from the OSH-to-Oil converter can be found in the Oil project's blog [4]. Listing 2 shows an original Bash script before conversion, and Listing 3 shows the Oil language variant generated from it. The parser is also written in Python; its source code can be found on GitHub in the OSH source code under tools/ [5].

make_hdb()
{
  # Some distros don't put /sbin:/usr/sbin in the $PATH for non-root users.
  if [ -z "$(which  mke2fs)" ] || [ -z "$(which tune2fs)" ]
  then
    export PATH=/sbin:/usr/sbin:$PATH
  fi
  truncate -s ${HDBMEGS}m "$HDB" &&
  mke2fs -q -b 1024 -F "$HDB" -i 4096 &&
  tune2fs -j -c 0 -i 0 "$HDB"

proc make_hdb
{
  # Some distros don't put /sbin:/usr/sbin in the $PATH for non-root users.
  if test -z $[which  mke2fs] || test -z $[which tune2fs]
  {
    export PATH = "/sbin:/usr/sbin:$PATH"
  }
  truncate -s $(HDBMEGS)m $HDB &&
  mke2fs -q -b 1024 -F $HDB -i 4096 &&
  tune2fs -j -c 0 -i 0 $HDB
  test $Status -ne 0 && exit 1