Number and String Operators

As mentioned earlier, there are three main types of operators, each for a class of objects: numbers, text strings, and files.

As far as numbers are concerned, the next installment in this series will be specifically devoted to doing math with the Bash shell. Therefore, I am only going to briefly mentions the main numerical operators -gt, -ge, -lt, and -eq (greater, greater or equal, less, and less or equal, respectively). These operators let you write "keep doing something, until $COUNTER becomes greater than 100":

while ( $COUNTER -le 100)
    do something

The operators that compare strings are equally intuitive, as you can see in this small list:

$A  = $B True if the $A and $B strings are equal
$A != $B True if they are different
-n $A    True if length of $A is greater than 0
-z $A    True if $A if empty or uninitialized

File Test Operators

Real gurus know that "in Unix, everything is a file." This means that something like /home/marco/somefile may not be what we normally mean by the word "file," but instead could be a physical device (like a serial or parallel port) or even pipes or sockets (i.e., objects that connect running programs so they can directly exchange data). You can reveal the exact nature of what looks like a normal file with these operators:

-e $F : $F exists
-p $F : $F exists and is a named pipe
-S $F : $F exists and is a socket
-f $F : $F exists and is a regular file
-h $F : $F exists and is a symbolik link (equivalent form: -L
-d $F : $F exists and is a directory

Other operators test file permisions and ownerships:

-r $F : $F exists and is readable
 (similarly, -w = writable, -x = executable)
-O $F : $F exists and is owned by the user running the test
-G $F : $F exists and owner by the user's group

The following two operators, instead, compare file time stamps

$A -nt $B
$A -ot $B

to tell the script whether file $A is newer (-nt) or older (-ot) than file $B. See the Linux Documentation Project for the entire list of file test operators [4].

Real World Example

A small company, which I will not name, spent a few years reshuffling everything (business plan, product catalog, staff, and junior consultants) every quarter. That chaos gave people (or perhaps forced on them) the "freedom" to store and (re)organize all the files and software they used in whatever way they wanted. To make a long story short, the dubious honor of sorting that mess eventually fell on me.

In practice, I had to locate, inside hundred thousands of files of all sorts, the files that were broken links, too big, potential security risks due to wrong permissions or other reasons, or symptoms of other problems. Of course, to solve that problem, I used the power of the Bash test operators. The result was a Bash file analyzer (see Listing 1 for a simplified version).

01 #! /bin/bash
02
03 TARGET=$1
04 MAXSIZE=$2
05
06 cd $TARGET
07
08 IFS=$(echo -en "\n\b")
09
10 for FF in `find . | sort`
11 do
12 ##############################################
13   if [ -d "$FF" ]
14   then
15   continue
16   fi
17
18   ##############################################
19
20   if [[ $FF =~ [[:space:]] ]]
21   then
22   printf "%-25.25s : %s\n" "SPACES in file name (1)" $FF
23   fi
24
25   if [[ "$FF" = *" "* ]]
26   then
27   printf "%-25.25s : %s\n" "SPACES in file name (1)" $FF
28   continue
29   fi
30   ##############################################
31
32   if [ -x "$FF" -a ! -O "$FF" ]
33   then
34     printf "%-25.25s : %s\n" "NON-owned executable" $FF
35   continue
36   fi
37   ##############################################
38
39   EXT=`echo $FF | awk -F . '{print $NF}'`
40   if [ -x "$FF" -a "$EXT" = "pdf" ]
41   then
42   printf "%-25.25s : %s\n" "EXECUTABLE PDF" $FF
43   continue
44   fi
45   ##############################################
46
47   if [[ "$FF" == */drafts/* ]]
48   then
49   printf "%-25.25s : %s\n" "DRAFT, remove?" $FF
50   continue
51   fi
52   ##############################################
53
54   if [ -L "$FF" ]
55   then
56   if [ -e "$FF" ]
57     then LINKSTATUS='LINK'
58     else LINKSTATUS='BROKEN LINK'
59     fi
60     printf "%-25.25s : %s\n" "$LINKSTATUS" $FF
61   continue
62   fi
63   ##############################################
64
65   SIZE=`ls -s $FF | sed -e 's/ .*//'`
66   if [ "$SIZE" -eq "0" ]
67   then
68     printf "%-25.25s : %s\n" "EMPTY FILE" $FF
69   continue
70   elif [ "$SIZE" -ge "$MAXSIZE" ]
71   then
72     printf "FILE TOO BIG: %-10.10s : %s\n" $SIZE $FF
73   fi
74 done

The file analyzer takes as input the directory that it should scan and a file size expressed in bytes. It then looks at all the files in that directory, reporting all those that are bigger than that size or violate any other rules embedded in the code, as shown in Listing 2.

BROKEN LINK           : /home/testing/link-to-non-existing-file
DRAFT, remove?        : /home/testing/drafts/biography.md
EMPTY FILE            : /home/testing/just-an-empty-file.txt
EXECUTABLE PDF        : /home/testing/pdfs/executable.pdf
FILE TOO BIG          : /home/testing/archives/a-really-big-file
NON-owned executable  : /home/testing/someweirdfile
SPACES in file name : /home/testing/filename with spaces

After moving to the target directory (Listing 1, line 6), the script finds and sorts all the files and folders it contains, in order to examine them one at a time (line 10). The command in line 8, right before the loop, sets the Bash Internal Field Separator (IFS) to newline (\n), backspace (\b), and nothing else. Without it, files or folder names containing spaces would be split by those spaces, and the computer would therefore load into $FF the names of files or folders that do not exist, causing a long stream of useless warnings.

The body of the loop consists of several, independent file tests, each looking for a specific condition or combination of conditions. Whenever one of those tests matches, the script reports what it found and then moves, thanks to the continue keyword, to the next file in the pipe. The way a for loop can work on a dynamic list of files, as well as the behavior of the continue keyword, are described in detail in the previous installment of this series [1]. In this article, I will just focus on the test operators.

I am only interested in files, so the first thing I do (lines 13 to 16) is to just jump to the next item in the list whenever I find a directory. At first glance, it may seem that a better way to skip directories would be to pass the -f ("files only") option to the find command in line 10. That option, however, would make the loop also skip links and other non-regular files, which I do need to find instead. This is why I make the script "find" everything and then ignore the directories.

The two if blocks in lines 20 to 29 do the same thing twice, just to show you two different methods (regular expressions and plain comparison): They report files with spaces in their names (or in the names of the folders containing them). In the original script, this check existed because such spaces were not allowed due to some legacy custom software unable to handle them properly. Here, I have left it, because knowing several ways to check if a string contains spaces or other special characters is always useful.

The next check (lines 32 to 36) reports all files that are executable (-x) but do not (!) belong to the user that is running the script (-O). Then, in lines 39 to 44, I first save into $EXT the extension of the current file and issue a warning if that file is an executable PDF. (In general, every file should only have the minimal set of permissions it actually needs to be used.)

Another thing I had to do was to find files that may no longer be needed in order to free disk space. This is the task shown in line 47, which detects all files inside subfolders named drafts.

Links to other files are found with the -L operator in line 54. If they point to existing files (line 56), they are just listed. Otherwise, a warning is issued.

The pipe in line 65 saves into $SIZE the size in bytes of the current file (to understand how it works, execute that pipe on any file at your command prompt, one piece at a time). The if statements that follow print proper warnings if that number is either zero or above the threshold passed as the second argument to the script.

The script in Listing 1 works, but there are many things it does not do. The first is to actually check the format of a file, instead of blindly trusting a filename extension that may be wrong or missing. The way to fix this shortcoming is to use the file command; this is left as exercise for the reader.