Analyzing file metadata in the shell

Testing Files

Usually, the test command is not used in full, but rather as a notation using square brackets and matching options:

if [ $? -eq 0 ];
# is the same as
if test $? -eq 0;

test is used to evaluate the type and timestamps of objects in the directory tree. It returns   if the tested condition is true. See Table 3 for test's options.

Table 3

Test Options

Test

True, if …

-e <Object>

Object exists

True, if <Object> exists and …

-b <Object>

It is a block device file

-c <Object>

It is a drawing device file

-d <Object>

It is a directory

-f <Object>

It is a standard file

-g <Object>

The group ID bit is set

-G <Object>

The group entries for the process and file match

-h <Object>

It is a symbolic link

-k <Object>

The sticky bit is set

-L <Object>

It is a symbolic link

-O <Object>

The query process points to the same owner

-p <Object>

It is a FIFO

-r <Object>

It is readable

-s <Object>

Its size is not 0

-S <Object>

It is a socket

-u <Object>

The UID bit is set

-w <Object>

It is writable

-x <Object>

It is executable

True, if <Object1> exists and …

<Object1> -ef <Object2>

<Object2> points to the same object

<Object1> -nt <Object2>

Is newer than <Object2>

<Object1> -ot <Object2>

Is older than <Object2>

The test command is useful for making a distinction in an if construct. The range of applications is extensive. In a script for saving data, for example, it would be possible to check whether a file named BACKUP.INFO exists. If so, the script creates a copy of all files that are newer than this file's timestamp. Otherwise, the script creates a full backup.

Listing 3 shows the code for a script that creates a directory named BACKUPTEST if it doesn't already exist, quickly performing a common task (Figure 12).

Listing 3

tester.sh

#! /bin/bash
uvznew () {
  # FILE?
  if [ -f BACKUPTEST ]; then
    read -p "File with same name exists! Rename (y)? " we
    if [ "$we" = "y" ]; then
      mv BACKUPTEST BACKUPTEST.file
    else
      echo "Either delete or move the BACKUPTEST file!"
      echo "END OF SCRIPT"
    fi
  fi
  if [ -d BACKUPTEST ]; then
    cd BACKUPTEST
    echo -n "The current directory is $PWD"
    echo " "
  else
    echo "Creating BACKUPTEST"
    mkdir BACKUPTEST
  fi
  return 0
}
uvznew
Figure 12: With automation, the tester.sh script in Listing 3 handles tasks more quickly and reliably.

Timestamps and Rights

The find tool not only searches for file and directory names, but it also includes timestamps, access rights, and the file size as a filter if required. See find's man page to learn about the full scope of this command. For the most important find options, see Table 4.

Table 4

Find Options

Action

Option

Note

-type <type>

Search by type

-size +/-<size>

Search by size

- = maximum size; + = minimum size; nothing = same size

-perm <file permissions>

Search for file permissions

-newer <files>

Search for files newer than <file>

-mtime -/+<N>

Search for files not modified for <N>days

+ = at least; - = within

-atime -/+<N>

Search for files not accessed for <N> days

+ = at least, - = within

-execdir <command> "{}" +

Execute <command> for found file

Safer method

If necessary, you can forward find's output to a pipe or process it using xargs, which passes the result to other commands, like tar. As an example, Listing 4 creates subdirectories for files as a function of their modification date and moves them there. Figure 13 shows the directory's contents before running the script. Figure 14 shows the script in action, and Figure 15 shows the output.

Listing 4

sortme.sh

01 #!/bin/bash
02 # $1 = directory to process
03 if [ -z $1 ]; then
04   echo "No input"
05   exit
06 fi
07
08 cd $1
09
10 for i in $(stat -c %y:%n * | sort -r | tr \  \:); do
11   # Populate variables subdir (subdirectory)
12   # and fn (filename)
13   subdir=$(echo $i | cut -d \: -f1)
14   fn=$(echo $i | cut -d \: -f6)
15   # Do not process if directory
16   # restart loop
17   if [ -d $fn ]; then
18     echo "$fn: Skipping directory"
19     continue
20   fi
21   # Create subdir if needed, and
22   # move file to it
23   if [ -d $subdir ]; then
24     mv -v $fn $subdir
25   else
26     mkdir $subdir
27     mv -v $fn $subdir
28   fi
29 done
Figure 13: The directory content before running sortme.sh.
Figure 14: sortme.sh evaluates the metadata and sorts files into the appropriate directory structure.
Figure 15: Files end up in the corresponding subdirectories in the new directory structure.

The for loop receives the data courtesy of stat. Since spaces are used as separators, it replaces them with colons. The output is sorted by date, starting with the newest files.

In the for loop, the subdir variable contains the subdirectory to be created and fn takes the file name. The script evaluates whether or not fn is a directory and, in this instance, aborts processing. The loop then begins with a new pass. This prevents the script from processing a directory.

If, on the other hand, fn is a file, the routine then checks again by means of a test whether the subdirectory already exists. If this is not the case, it creates the directory and moves the file to it. If the directory already exists, it simply does the latter. Figure 15 shows a visual overview of tree.

Conclusions

With the appropriate shell commands, you can identify, evaluate, and change various file and directory metadata. As a result, many operations can be simplified using scripts, which avoid errors and save valuable time.

The Author

Harald Zisler has focused on FreeBSD and Linux since the early 1990s. He is the author of various articles and books on technology and IT topics. The fifth edition of his book Computer-Netzwerke (Computer Networks) was recently published by the Rheinwerk Verlag publishing company. He also works as an instructor, teaching Linux and database topics in small groups.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Perl: noworries

    We'll show you how you can avoid the tragedy of lost files with a transparent, Perl-based version control system.

  • Command Line: ls

    The ls command lists files at the command line and gives you many useful details about file properties. With or without colors, this command keeps you on top of your files.

  • Workspace – Photocrumbs

    Instead of relying on a third-party service for instant photo sharing, you can build your own solution using existing software and a pinch of PHP and Python scripting.

  • Meson Build System

    Developers fed up with cryptic Makefiles should take a look at the new Meson build system, which is simple to operate, offers scripting capabilities, integrates external test tools, and supports Linux, Windows, and Mac OS X.

  • bd, autojump, and Fasd

    Bd, autojump, and Fasd improve the workflow for command-line aficionados thanks to quick navigation in the filesystem.

comments powered by Disqus
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters

Support Our Work

Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.

Learn More

News