Use fzf and fzy to add fuzzy search tools to the shell

Near Miss

Article from Issue 260/2022
Author(s):

Fuzzy finders retrieve useful results from data streams even if there are no exact matches.

Today, fuzzy searches are an integral part of everyday IT life. They correct typos, detect similarities, and offer a way to find what you need with reasonable overhead, even in unstructured data. The basic principle is based on the Levenshtein distance [1], word distances defined in the 1960s (see the "Levenshtein" box). Developers have extended and optimized this algorithm more or less from the start.

Levenshtein

The Levenshtein distance describes the minimum number of changes (replace, delete, insert) needed to transform one string into another. The weighted number of these actions determines how similar the two strings are. Weighting means that, for example, two swapped letters (like in a typo) are less important than, say, inserting additional characters. But the approach also involves a number of problems. Weighting depends on the concrete task (spell check, search, etc.) and the language.

Having said this, many standard tools ignore fuzzy searches or limit themselves to highly simplified variants up to this day. For example, the grep tool from the coreutils package processes arbitrarily complex regular expressions (patterns), but it does not support fuzzy searching. Fuzzy searching is supported by agrep [2] and ugrep [3] at the command line, and there are several other, less well-known tools. Agrep impresses here with a best-match option.

Fzf [4] and fzy [5] (see the "Little Brother" box) enrich the shell construction kit, adding two new, powerful tools. They act as interactive filters, i.e., by default, they source their data from the standard input and return the results to the standard output. Interactive in this context means that they provide interfaces for entering the search patterns, which allow the search patterns to be adapted and refined at runtime.

Little Brother

Fzf is already in use as a standard tool on many systems today. According to its developer, the enhanced fzy tool offers an improved algorithm and a faster search speed. In addition, fzy limits itself to the bare essentials in terms of options, and this simplifies the application. At first glance, fzy really only looks like a variant of fzf, but this is deceptive. Although the output is similar, fzy can at most be considered fzf's little brother. It is also piped, but there is also a very simplified Finder, which gives users a kind of menu with a configurable prompt. But this already exhausts the list of similarities. This becomes particularly clear if you look at fzy's options. By default, its Finder uses only 10 lines of the terminal to display the list of results. This can be changed with the -l option. Fzy can only work with positive patterns; it does not support exclusions. In interactive mode, only a handful of keyboard shortcuts are available. They are listed by the tool's fairly concise man page.

fzf

Fzf stands for Fuzzy Finder. The tool acts as a general-purpose filter, which also acts as a menu generator in interactive mode. Calling fzf launches the tool in the shell. This normally just displays all the files in the current directory. Fzf uses a built-in find command to do this, which is the default if the program is not given any data via the standard input. This feature helps to keep the commands short. You can manage this behavior via the FZF_DEFAULT_COMMAND environment variable, specifying a different command if needed. You can store the options that you want to pass in to fzf as defaults in the FZF_DEFAULT_OPTS environment variable.

Most of the time, the program is used in conjunction with a pipe. This means that it receives its input data via the standard input (Listing 1). This takes you to interactive mode, which consists of two parts. The visible interface, the Finder, supports interactions: You can move around with the arrow keys or by pressing Ctrl+K and Ctrl+J. The terminal displays the current line in bold or inverse type depending on the settings. Pressing the Enter key accepts the selected line and sends it to the standard output or to the terminal.

Listing 1

locate and fzf via Pipe

$ locate / | fzf
[...]
.../Images-sda5/_1130192.png.out.pp3
.../Images-sda5/_1130192-1.png.out.pp3
.../Images-sda5/_1110520.jpg.out.pp3
.../Images-sda5/_1090399.dng.xmp
.../Images-sda5/_1090399.dng.pp3
.../Images-sda5/_1040511a.pts
.../Images-sda5/_1040511.pts
.../Images-sda5/_1040511-6.pts
.../Images-sda5/DANCE!-2020-2021
[...]

In addition, a prompt (>) is displayed at the bottom edge of the Finder, prompting you for input. If you already passed a search pattern into fzf when you called it (as an argument of the -q option), the search pattern can be edited in this line. The second part works in the background, managing the index and the fuzzy search. In the second-to-last line in Figure 1, the tool shows you what is happening in this regard. A constantly changing symbol shows that it is creating an index from the input, which it will then search through for the pattern. The current size of this index is also shown at the bottom.

Figure 1: fzf --preview 'less {}' tells the program to display a preview via the less pager; however, this is primarily intended for text files.

Search Pattern

The Finder is an essential feature of fzf. However, the tools's capabilities are not limited to editing existing search patterns. It also evaluates the patterns incrementally. Each additional character entered refines the search. In Extended Search mode, which is enabled by default, you can also enter additional patterns separated by spaces. The program assigns special meanings to some of the special characters used in these patterns (see Table 1). You can combine several search patterns as described in Listing 2.

Listing 2

Combining Search Patterns

Pattern1$ Pattern2 ^Pattern3 !Pattern4

Table 1

Searches for Power Users

Character

Example

Result

^

^linux

Lines that start with linux

$

linux$

Lines that end with linux

!

!linux

Lines that do not contain linux

'

'linux

Lines that contain precisely linux (not a fuzzy search)

sbtrkt

Token for fuzzy search (default)

You can use the arrow keys to navigate within the search pattern in the Finder. The highlighted line shows the selection. Alternatively, some default key mappings are available (see Table 2). All actions involving selecting or deselecting of multiple lines require you to launch the tool in multiple-selection mode with the -m option. On top of this, the Finder also supports mouse control. Scrolling with the mouse wheel, clicking, and double-clicking lets you enable and select. If you hold down the Shift key, a mouse click selects multiple items. A double-click ends the input.

Table 2

Controls

Key

Explanation

Ctrl+K

Move up

Ctrl+J

Move down

Ctrl+C

Cancel

Alt+F

Jump one word to the right in the search pattern

Alt+B

Jump one word to the left in the search pattern

Ctrl+A

Jump to the start of the search pattern

Ctrl+E

Jump to the end of the search pattern

Tab

Select multiple consecutive lines

Shift+Tab

Deselect the following selected lines successively

Ctrl+I

Select or deselect arbitrary lines

Otherwise the program works with key bindings adapted to Emacs by default. Additional or deviating key bindings can be set with the --bind option. The -e command-line option disables the fuzzy search, returning only exact hits. Other important fzf options are summarized in Table 3.

Table 3

Important Options

Option

Explanations

-e/+x

Display only exact matches, disable fuzzy search

+i/-i

Case (in)sensitive

--algo=v1/v2

Select algorithm: v1 is faster; v2 has better matches (default)

--disabled

Disable search, only provide interface

+s/--no-sort

Do not sort output

--tac

Reverse order of input

User Interface Options

+m/-m

Enable/disable multiple selection; default, return only one match

--cycle

Continue search before the first/after the last entry

--height=PERCENT

Height of Finder as a percentage of the terminal size

-q QUERY

Start software with QUERY

-1

Do not start Finder for a single match

-0

Do not start Finder if there is no match

--preview COMMAND

Use COMMAND for the output

--preview-window

Define form, size, and type of preview

You can define how the program determines and evaluates matches with a number of parameters. The input lines are evaluated on the basis of the criteria from Table 4. index can only occur at the end. Fzf uses this option by default if you do not specify anything. Fzf evaluates the inputs using these criteria, which you explicitly select as arguments for the --tiebreak=CRITERION option. By default, it uses length or length,index. Stipulating end tells the software to evaluate the lines in reverse order.

Table 4

Modifiers

Option

Explanation

length

Prefer shorter lines with matches

begin

Prefer matches at the start of the line

end

Prefer matches at the end of the line

index

Prefer lines nearer to the start of the input stream

One typical fzf application is a fault-tolerant variant of the locate command. The latter searches for files on the local system by reference to the filenames in a previously created database. This search is non-fuzzy; it does not return results in case of typos or abbreviations. In the example in Listing 3, locate / creates a list of all entries and returns it to fzf via a pipe. This stipulates all the arguments passed in as patterns for the command. If you pass in L for no arguments, the entire list appears, and you can enter patterns directly in fzf's interactive mode.

Listing 3

Fuzzy Search with locate

L () { locate / | fzf -q "${*}" ; }

If you have a very large database, you will probably want to exclude many matches to improve the clarity of the results. You can do this by passing in arguments to the function. In Listing 4, the pattern searches for something but excludes butnotthat. If the search still returns too many matches, such as in randomly-named browser caches, more additional patterns in interactive mode will lead to better results. At the command line, you must quote the exclamation mark in front of the excluding pattern with a backslash (/!) or use quotes around longer sequences. Otherwise, the shell will interpret the character as a history command and remove it. In interactive mode, \! has a different meaning.

Listing 4

Excluding Matches

> L something \!butnotthat
[...]
.../Music/The Temptations -  Papa was a rolling stone.mp3.mp3
.../Images/png/2-watermark.png
.../Music/Frumpy - How the Gypsy was Born.mp3
.../Music/2/Frickle - Some People, Follow Me.mp3
.../Music/Byron Metcalf - Heart Warriors+.mp3
.../Images/aerial image no channel water border.png
[...]

Fzf supports a sophisticated preview mode for the matches. The argument in the --preview option is the command line for generating the preview. This is shown for the file currently selected with the cursor in a separate window that Fzf generates on the right (Figure 1). You can only use text output in a terminal, which means that, for many files, the metadata or the previously converted or extracted text contents appear as a preview instead of the binary data.

less is really useful as a universal previewer (Listing 5). After all, the pager is particularly well suited for displaying different file types with lesspipe [6]. You just need to specify --preview 'less {}' as the command. This is all it takes to display the contents of text files, for example, but also to archive content or the metadata of multimedia files, assuming that lesspipe is installed correctly.

Listing 5

Match Preview

$ fzf --preview 'PreviewCommand'

You can define both the preview window's position and size using --preview-window POSITION. The main arguments for this option are the position (top, bottom, left, right), the size (as a percentage of the terminal width), and the mode (wrap, cycle, hidden). You need to enter this information in a colon-separated format without any spaces (Listing 6).

Listing 6

Preview Position

$ fzf --preview-window=top:55%:wrap:cycle

In addition to the preview, there is another way to generate a view of the content via actions. In the program, Actions means a whole group of functions that you can enable in the interactive Finder. You can also trigger Actions through events. The only events fzf can currently handle are change, if the search query changes, and backward-eof if the search query is empty and you still try to delete it using the backspace key. The change event is often used in combination with first (jump to first hit); abort is the default for backward-eof.

Key Bindings

One of fzf's special features is the ability to call other programs within the running program. This feature, which is controlled by the execute action, offers a wide range of possibilities. For example, you can open selected files to view or edit them. Fzf expects a command-line argument for the execute action. It then executes the command line in a subshell. The {} string stands for the selected files.

The --bind option is used to bind actions to a key or keyboard shortcut. You can use more than one action for a key binding. This is demonstrated by an example from the man page (Listing 7): The two forms are equivalent.

Listing 7

Defining Key Bindings

fzf --multi --bind 'ctrl-a:select-all+accept'
fzf --multi --bind 'ctrl-a:select-all' --bind 'ctrl-a:+accept'

Creating your own definitions makes it possible to adapt the way fzf works to match other programs or even to add new functions. To group the --bind arguments, use single or double quotes if there are any spaces.

The current version of fzf supports only a subset of keys for bindings (see Table 5). The program ignores other combinations, such as Shift+Enter, which outputs a warning such as un-Supported key: shift-enter. But the software does accept mouse clicks in combination with the up arrow, down arrow, and double-clicks. Many actions simply delete characters or jump to certain positions in the line. Table 6 summarizes other, more specialized functions. Most of them are not normally mapped to keys, so they are not available by default.

Table 6

Actions

Action

Explanation

accept

Apply the current entry

accept-non-empty

If an entry is selected, apply it

change-prompt

Change the Finder prompt

clear-selection

Deselect the previously selected entry

preview

Display preview

refresh-preview

Refresh preview

select-all

Select all

deselect-all

Deselect all

toggle-all

Toggle all matches

toggle-search

Find toggles

toggle-preview

Show (hide) preview window

toggle-sort

Toggle sorting

disable-search

Disable search

reload

Reload and rebuild index

replace-query

Replace search with current selection

unbind

Use --bind to clear the keyboard binding

Table 5

Supported Keys

Group

Keys

Single keys

Enter, Space, Tab, Esc, Del, Up Arrow, Down Arrow, Right Arrow, Left Arrow, Pos1, End, Ins, F1 to F12, A to Z, \, ]

Ctrl combinations

Space, Shift+7, A to Z

Alt combinations

Space, Up Arrow, Down Arrow, Right Arrow, Left Arrow, Backspace, A to Z

Shift combinations

Up Arrow, Down Arrow, Right Arrow, Left Arrow, ^, Space, Tab

Three-key combinations

Ctrl+Alt+A to Ctrl+Alt+Z, Alt+Shift+Up Arrow, Alt+Shift+Down Arrow, Alt+Shift+Right Arrow, Alt+Shift+Left Arrow

There are two ways to execute external commands. If you use the execute action, fzf executes the command line, waits until the end of the command, and then displays its output. In many cases, this is precisely the desired behavior. To prevent fzf from waiting – and thus not accepting further input while doing so – you can append an ampersand to run the executed commands in the background. If you don't need the output of the executed commands, use the execute-silent action. The program will then run the commands but discard the output.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Fuzzy Finder

    Simplify your searches and get better results with fzf, a modern search tool based on fuzzy logic.

  • Tracked Down

    Searching for text in files or data streams is a common and important function. Ugrep tackles this task quickly, efficiently, and even interactively if needed.

  • McFly

    When it comes to working at the command line, using Bash history effectively can save you time. McFly extends the Bash history's features and helps you find past commands more quickly.

  • Worker

    Worker, a file manager with more than 20 years of development, has evolved into a tested, powerful, and functional tool.

  • KTools: Beagle Helpers

    If you have given up hope of ever finding your way around the mess of data on your desktop, a desktop search engine can help you penetrate the gloom. Kerry and KBeaglebar bring the Beagle engine to the KDE desktop.

comments powered by Disqus
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters

Support Our Work

Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.

Learn More

News