Talking to Users

Here documents (heredocs) are one way to make a shell script talk to users without a third-party program. They look like this:

cat << EMAILADDRESSES
$MANAGER
jane@linux.com
tom@linux.com
EMAILADDRESSES

As you can see, a heredoc is just a series of lines of plain text, delimited by two occurrences of the same string (EMAILADDRESSES in the above example). The closing occurrence of that string must be the only text on its line. If the heredoc body contains calls of variables, like $MANAGER, those calls are replaced by the current values of the same variables. In the format above, the heredoc content is just printed to the standard output with the cat command. However, it may also be saved into a variable for later use:

MYADDRESSBOOK=$(cat <<'EMAILADDRESSES'
$MANAGER
jane@linux.com
tom@linux.com
EMAILADDRESSES
)

Heredocs are a very convenient system to define and embed generic templates inside a script. Many heredocs are created to dump data into some file or database. In the context of this article, however, their main use is to interact with others. The reason is that heredoc templates may be completely static, but they also can be generated on the fly (and often are) for many different purposes.

A heredoc's content may be shown to the user to explain what the script is doing or to summarize all the provided data and ask for confirmation before using the data.

Heredocs can also allow a script to send multiple input commands to the standard input of programs that should otherwise be executed manually, typing text one line at a time. In that case, however, you should really be careful. Conversations are well and good, but only if both parties can talk and listen at the same speed! You really don't want to throw two or more commands in one shot at a program unless you are absolutely sure that this will not cause any of those commands to be missed or to fail, because the program depends on the complete, successful execution of all previous commands!

Graphic Feedback

Imagine a process that takes some time (possibly a long time) to complete. It is good practice to let the user know how far that task is from completion; you can do this with a simple text-based graphic or animation. The simplest solution, which works in any shell without installing anything else, is a progress bar drawn with echo commands:

echo -ne '##       (33%)\r'
sleep 1
echo -ne '####     (66%)\r'
sleep 1
echo -ne '####### (100%)\r'
echo -ne '\n'

Each command prints a number of hash characters that is proportional to the percentage of the task already completed. Then the carriage return (\r) at the end of each string places the cursor back at the beginning of the line. This makes the next echo overwrite the previous one. Note that the sleep instructions in the above example are just placeholders for the real code that you want to run between each call. This method's main limitation is that it only works if you know what percentage of the total running time each phase takes.

In general, to draw more precise progress bars, in or outside a terminal, you must know the entire task's size. This may be expressed with sufficient accuracy by metrics like the total number of files to compress, text lines to parse, or digital photographs to index. Once you have that number, you can use tools like pv or dialog, which are available in the standard repositories of all major Linux distributions, to draw progress bars.

pv must be placed in the middle of a pipeline to observe the flow of data being processed. Then, it can print to the terminal both an ASCII progress bar and information such as the elapsed time, the percentage of work already completed, and other data. Figure 3 shows the result of measuring the time taken to make a tar archive of a entire directory of size $DIRSIZE:

DIRSIZE=`du -sb . | awk '{print $1}'`; tar c . | pv -s $DIRSIZE > ~/all-my-articles.tar

Figure 3: The pv program creates the most complete progress bars you can display in a Linux console.

You can draw precise progress bars (or more accurately gauges) with the dialog tool, which is the console, text-only version of Zenity. It is the same tool that is used in the text-based installation procedures of many Linux distributions. In addition to drawing progress bars, it can also collect user input with text boxes, radio buttons, and other systems.

Using dialog to draw the gauge is relatively straightforward. Listing 6 shows two types of progress bars that can be created with dialog. Lines 4 and 17 show how to tell it what to draw (--gauge), along with the corresponding caption and window size (20*75 or 10*75). Even the code that actually compresses the archives and echoes (lines 9 and 16) the percentage of work already done is not complex, if you have read the fifth installment of this series "Shell Math" [7]. Listing 6 is important, because it shows two different ways to pass data to dialog (or any other command for that matter) from a whole bunch of shell commands. The first call (line 4) uses process substitution, a general Bash technique that allows you to bundle into one stream the output of multiple commands [8] (Figure 4). The second call to dialog (line 17) receives data from the for loop through a standard shell pipe (Figure 5).

Figure 4: Depending on your preferences, you can use dialog to create a process substitution gauge …

Figure 5: … or a pipe gauge.

Instead of progress bars, you can also use a rotating spinner (Listing 7), which I found online [9], to reassure users that the script is alive and running. Line 5 saves in $n the number of characters of the $sp string. The while loop starting in line 8 runs forever, because its condition is constantly true. Every time it runs, it prints one of the characters in the $sp string, then sleeps for 0.1 seconds. The character that is printed is the one in the i th position in the $sp string. But in line 9, $i is incremented every time printf is used to select a character. This is why a different character of the $sp string is printed every time, giving the illusion of a rotating spinner.

Share Your Scripts!

Throughout this series, I have shown how the average Linux user can use Bash scripts to automate many different time-consuming tasks, in ways that may either be impossible or much more complex with other tools. Take advantage of what you have learned here. Above all else, please share with us your best shell tricks that you've discovered in your Linux scripting adventures!

The Author

Marco Fioretti (http://mfioretti.com) is a freelance author, trainer, and researcher based in Rome, Italy. He has been working with free/open source software since 1995 and on open digital standards since 2005. Marco also is a Board Member of the Free Knowledge Institute (http://freeknowledge.eu).