Playing with Time

Time-related calculations do not involve math functions or handling decimal numbers. However, they may still seem a daunting task for Bash unless you know and creatively use one or two quick and dirty tricks. In many cases, this can be handled by calculating the difference between moments in time. To do this, you use the date command options -d and %s. As an example, Listing 1 calculates the duration, in days, of the Space Shuttle program, from its first flight (April 12, 1981) to its last (July 8, 2011).

01 FIRSTFLIGHT=`date -d "1981-04-12 10:05" +'%s'`
02 LASTFLIGHT=`date -d "2011-07-08 10:05" +'%s'`
03 DURATION=$(( LASTFLIGHT - FIRSTFLIGHT ))
04 DURATION=$(( DURATION/(3600*24) ))
05 echo "Flights of the Space Shuttle Program went on for $DURATION days"

The -d switch in lines 1 and 2 tells date to print out the date that follows, instead of the current one. The + sign defines the format to use, and its %s option returns that date, expressed as the number of seconds since January 1, 1970. Now, all that is left to do is calculate the difference between $LASTFLIGHT and $FIRSTFLIGHT, which for the reason I just explained are both integer numbers, and convert it from seconds to days (line 4). In case you can't wait to solve this yourself, the answer is 11,044 days.

It is easy to see how the trick in Listing 1 can be generalized to calculate any difference between two moments in time, including future ones. As easy as it is, however, this trick has one weakness that may be irrelevant for some and cause problems for others if ignored.

The double parentheses construct may not round correctly the result when the subtraction result is not an integer multiple of 24. This is exactly what happens when the time interval considered includes the day when daylight savings time ends or begins (i.e., a day with 23 or 25 hours). The solution to this problem is simpler than it may seem: It consists of replacing the double parentheses in the example above with the bc command. For a practical example with a very detailed explanation, see reference [8].

Beyond Vanilla Math

Before moving on, I want to introduce, for completeness, three niche types of calculations, each of which could fill its own tutorial. When you encounter these types of calculations, you might incorrectly assume that they cannot be handled with shell scripts.

The first is bit-level arithmetic, which means acting directly on the bits that compose each number (or any string really). It has two applications:

• Doing arithmetic basically in the same, very low-level way it happens in machine language, mostly for didactic purposes.
• Using numbers as very compact status registers, where each bit indicates, for example, if one element of an array is in one of two states.

See the examples online [9] for more information:

15 >> 3 = 1 # '1111' >> 3 = '0001'
15 & 3  = 3 # '1111' & '0011' = '0011'

While it might seem cryptic, the first operation uses the right shift operator (>>), which shifts to the right all of its left term's bits of a number of positions equal to the right term, and replaces the empty spaces with zeroes. Since the bit-level, binary representation of 15 is 1111 (8 + 4 + 2 + 1), shifting those four bits three times to the right yields 0001, which is exactly the number one in binary format. The second operation is a bitwise AND: It performs the Boolean AND operation on each pair of bits of the two numbers you pass to it. Since AND returns 1 only if both terms are equal to one, the result has non-null bits only in the two rightmost positions. This is more visible if you put the binary numbers, and the result, one below the other:

'1111'  = 15
'0011'  =  3
   ^^
'0011'  =  still 3!

Another niche type that some bold Bash users dare to enter from time to time is geographic calculations. This branch of mathematics answers questions like "what is the distance between two points on Earth whose latitude and longitude are known?" And "what is the bearing (i.e., the direction to follow, with respect to geographic North) to go from one to the other?"

Short answer: It is possible to solve these problems with a Bash script (with external help from some simple programs). If you want to know how, see [10], [11], and [12]. Be warned: These methods, while adequate for simple applications, lack the same precision customary of GPS navigators!

The final niche I am only going to mention here is advanced statistical calculations. The right open source command-line tool for this job is R [13], which can be used as a standalone tool or called from inside shell scripts.

Calling the Right Tools

Much of what Bash can do in the "math" realm is outside its direct support for arithmetic expressions. Several Bash built-in commands, as well as some little command-line programs, have lots of number-related applications. A little known example of built-in commands, which I myself only discovered after years of successfully using Bash for fun and profit, is factor. Unsurprisingly, this command prints the prime factors of each integer number it receives from the command line or from standard input:

#> factor 35 63
35: 5 7
63: 3 3 7

Another very handy command is seq, which prints sequences of numbers in a given range and in constant steps. Writing seq 80 in a script would generate all the integer numbers from one to 80. Writing seq 37 2 80, instead, would return a sequence like 37, 39, 41, and so on all the way up to 79: The first parameter is the starting value, and the second the increment to use. All parameters are interpreted as floating-point values, and you can use all the formats supported by the Bash printf command for the output:

for COUNTER in `seq -f '%5.2f' 65.3 1.5 71.7`

This would make $COUNTER cycle over the values 65.30, 66.80, 68.30, 69.80, and 71.30. Piping the output of seq to sort -R (for Random), would return the same numbers, but in random order.

Two other programs you need to know to efficiently collect and filter numbers for further processing are grep and cut. Knowing about these programs is a mandatory requirement for calculations in Bash scripts, whenever the numbers to process are surrounded by larger amounts of other numbers or by raw text in general.

Typical examples are listings of file sizes and timestamps when scanning hard drives, server logs, database backup files, or even traditional spreadsheets in Comma Separated Values (CSV) plain text formats. In all these cases, you can imagine the data as one (potentially endless) table in which only certain cells (that is, certain intersections of rows and columns) contain useful numbers. You can then use grep and cut to slice all and only those cells, using grep to cut rows, and cut to extract columns (or vice-versa, of course). Another more useful tool that can be used for the same types of jobs is the awk utility. Let's look at a practical example.