Intelligently Arranged
McFly upgrades Bash with artificial intelligence
When it comes to working at the command line, using Bash history effectively can save you time. McFly extends the Bash history's features and helps you find past commands more quickly.
The most popular Linux shell by far is the GNU Bourne-Again Shell (Bash for short). One of its many outstanding features is its integrated command history, where Bash saves previously run command lines so that you can recall them later on without a lot of typing.
Listing 1 shows the output you'll see if you enter the history
command at the terminal prompt. Each line begins with a line number in ascending order. The last command line to have been entered is at the end of the history list. You can use the arrow keys to scroll backward or forward one step in the history in the terminal. Pressing the Enter key copies the displayed line and runs it again.
Listing 1
Running the history Command
§§nonumer 01 $ history 02 [...] 03 446 nano .config/GIMP/2.10/scripts/image-subdivide.scm 04 447 nano .config/GIMP/2.10/scripts/slice-and-join.scm 05 [...] 06 901 yay -Syu 07 902 trizen -Syu 08 903 yay -Syu 09 [...] 10 1044 ++ 11 1045 ++ /tmp/ 12 1046 mv /tmp/*png . -v 13 1047 ++ 14 1048 mv /tmp/*png . -v 15 1049 ++ 16 1050 mv /tmp/linify.png . 17 $ history | grep slice 18 447 nano .config/GIMP/2.10/scripts/slice-and-join.scm 19 1051 history | grep slice
Bash supports two approaches to searching for specific strings directly in the history: Ctrl+R searches backwards from the current cursor position; Ctrl+S searches forwards from the current cursor position. Ctrl+S only makes sense if you have already advanced further back in the history and are no longer at the end. In addition, entering Ctrl+S only works in an appropriately configured terminal; normally, this keyboard shortcut blocks the terminal.
To indicate reverse searching in the history, Bash uses Ctrl+R with a special prompt:
(reverse-i-search)`':
i-search
stands for an incremental search, which refines or enhances the search by entering a new character. The function finds the last line in the history that matches the search criteria; each time Ctrl+R is pressed, the search jumps one match further back into the past.
In practice, this form of search in the history proves to be as simple as it is effective. A few cleverly selected entries are usually sufficient to find the desired line. However, you need to know what you are looking for – and, above all, you need to know a good pattern that will quickly take you to the desired command. The pattern can be anywhere on the command line, so you don't have to type it from the beginning. (See also the "Search Patterns" box.)
Search Patterns
You can combine history
with grep
to find all command lines in the history where a given search pattern occurs (Listing 1, line 17). Using agrep -B
instead of grep
gives you a fuzzy search.
For instance, to retrieve the command from Listing 1 (line 16)
mv /tmp/linify.png .
you would just need to type Ctrl+R I F Y. Command lines such as trizen -Syu
or yay -Syu
can be quickly found with Ctrl+R, Shift+S, or Ctrl+R - Shift+S.
As shown in Listing 1, the history usually contains command lines that are very similar to each other and can be described well with patterns, but cannot be easily distinguished. A remedy for this requires a new approach.
Modern Times
Bash history lacks the ability to search in a context-based way. For example, on many modern systems based on systemd, /var/log/journal/
contains large numbers of logfiles. Deleting outdated logfiles can often free up a large amount of storage space.
To see if, and how many, old logfiles are stored there, change to the directory. The du -sh
command then determines the space used by the existing files. find -mtime 7
lets you search for all files that have not been changed in over one week; you can then delete them using rm [...]
.
You could now write a script or – preferably – a shell function for this task in a few moments, but the problem remains the same: You have to actively develop a solution for every situation. Wouldn't you prefer that the shell guessed what you were going to do based on the directory and offered you a corresponding action?
McFly [1] does this by reading the previous history, analyzing it in a synthetic neural network, and attempting to recognize correlations in the command lines by machine learning. These relationships then serve as the basis for the suggestions offered by the shell's search functions.
In contrast to the neural network's quite complicated structure, McFly's last step is very easy to understand: The previous-history
function normally mapped to Ctrl+R is replaced by the new mcfly search
function.
The following parameters serve as the basis for the neuronal evaluation of the history and the McFly suggestions for the search function, in the specified order:
- The current directory
- The previously entered command lines
- The frequency and order of the call
- Whether a command line was previously found by McFly
- Whether the command ran without errors
After setting up McFly (see the box "Installing McFly"), the shell behavior changes. Ctrl+R calls the tool directly; the titlebar changes to reflect this, and the new key bindings become active (Figure 1).
Installing McFly
McFly [1] is not yet found in popular distribution's package sources. Only Arch Linux offers the program in the Arch User Repository (AUR). On Arch, you can install the program with an AUR helper, for example, using
yay -S mcfly
This will add a whole series of additional packages with a total volume of several hundred megabytes due to the dependencies.
In Ubuntu, and many other distributions, you need to install the Brew program, which was actually developed as a package manager for Mac OS X and later ported to Linux. Depending on the distribution, the package in question goes by the name of brew, linuxbrew, homebrew, homebrew-bundle, or – like in Ubuntu – linuxbrew-wrapper. Using the commands from the first six lines of Listing 2, you import Brew in Ubuntu and configure the program. Now set up the tap
with Brew (line 7), which retrieves several hundred megabytes of additional software. Following this, install by typing the command on line 8, which you then have to initialize (line 9).
Finally, expand your $PATH
to include the executable file's folder. To do this, transfer the code in Listing 3 to the ~/.bashrc
file. After a restart, you will be able to call the mcfly
command from the terminal.
Everything went smoothly on my Ubuntu test system, but some things can go wrong with this procedure: The /home/linuxbrew/
directory and its .linuxbrew/
subfolder may not be created automatically. When running brew install mcfly
, you will then see an error message. In this case, you need to create the directory yourself or create a symbolic link to it. (See also the "Bug Report" box.)
The McFly developer has also posted ready-to-go 32- and 64-bit binaries of the program on GitHub [2].
Bug Report
If McFly crashes on your system with a cryptic message similar to:
Unable to add cmd_tpl to commands: SqliteFailure([...]])', src/libcore/result.rs:1009:5
you are struggling with a bug that has affected many other users [3]. Commands with special characters will disrupt the program. In the test this happened when I accidentally entered the command:
^@^@^[...]^@^@sudo
Until the developers fix this problem, you'll have to make do with simply deleting the line from ~/.bash_history
with an editor.
Listing 2
Configuring and Installing Brew
§§nonumer 01 $ sudo apt install linuxbrew-wrapper 02 $ brew 03 $ echo 'PATH="/home/linuxbrew/.linuxbrew/bin:$PATH"' >> ~/.profile 04 $ echo 'MANPATH="/home/linuxbrew/.linuxbrew/share/man:$MANPATH"' >> ~/.profile 05 $ echo 'INFOPATH="/home/linuxbrew/.linuxbrew/share/info:$INFOPATH"' >> ~/.profile 06 $ PATH="/home/linuxbrew/.linuxbrew/bin:$PATH" 07 $ brew tap cantino/mcfly https://github.com/cantino/mcfly 08 $ brew install mcfly 09 $ source "$(brew --prefix)/opt/mcfly/mcfly.bash"
Listing 3
Expanding $PATH
§§nonumer if [ -f $(brew --prefix)/opt/mcfly/mcfly.bash ]; then . $(brew --prefix)/opt/mcfly/mcfly.bash fi
While initializing, McFly reads the existing history and evaluates it, which takes some time. You can press Esc to exit this mode and return directly to the prompt. Then use the up and down arrows to navigate the recommendations offered by McFly. When you enter a search term, McFly retrieves matching hits from the history.
Press Enter to accept the currently highlighted line from the history and execute the command immediately. The Tab key, on the other hand, transfers the highlighted history line to the input prompt without executing the command immediately. You can then edit the command and press Enter to execute it. Press F2 to delete individual lines from the list of suggestions after a prompt.
Artificial Intelligence
The interesting thing about McFly is its underlying artificial intelligence, the neural network. What used to require a huge amount of power and entire data centers, can now (with some limitations) even be performed by a Raspberry Pi. Today, such small neural networks are useful for all kinds of tasks, from image processing to character and pattern recognition.
These systems' missing logic often proves to be problematic. Neural networks learn coherencies without it being possible to clearly understand how they do it in each case. With McFly, however, you can still see the learning behavior, or at least the rudiments of it. If you call a command from the list of suggestions, McFly moves the command further up the list the next time it is called. However, it is less clear which commands appear (and in which order) initially in the proposal list. The rules used already seem so complex that some effort is needed to predict their impact.
You actually encounter a very similar system every day when searching on the Internet. Here, too, it is often impossible to identify how, where, and why certain results appear in the hit list while others do not. This problem is attributable to the principle of neural networks and is not completely harmless. For this reason, IT researchers repeatedly urge cautious use of the technology [4].
Conclusions
In addition to autocompletion, the history is an important shell feature for interactive use. McFly adds value to the history
command. Fortunately, the developer implemented the tool as an independent program instead of adding it to the already complex Bash or developing a plugin. This minimizes the side effects and allows the tool to be tried out quickly and safely.
As far as McFly's AI performance is concerned, a typical problem of self-learning systems applies here. It is virtually impossible to make reliable statements about the software, since using the software changes the system itself at run time. In my test, McFly delivered very good results; most of the command lines I searched for were found in the first list of suggestions.
If you are just looking for an upgrade of history
, there are simpler alternatives in the open source universe, such as hstr
[5] and cdhist
[6], which do without neural networks and take up far less disk space.
Infos
- McFly: https://github.com/cantino/mcfly
- McFly binaries: https://github.com/cantino/mcfly/releases
- SQLite3 error: https://github.com/cantino/mcfly/issues/56
- AI potential harm: https://en.wikipedia.org/wiki/Artificial_intelligence#Potential_harm
- hstr: https://github.com/dvorka/hstr
- cdhist: https://github.com/bulletmark/cdhist
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Latest Cinnamon Desktop Releases with a Bold New Look
Just in time for the holidays, the developer of the Cinnamon desktop has shipped a new release to help spice up your eggnog with new features and a new look.
-
Armbian 24.11 Released with Expanded Hardware Support
If you've been waiting for Armbian to support OrangePi 5 Max and Radxa ROCK 5B+, the wait is over.
-
SUSE Renames Several Products for Better Name Recognition
SUSE has been a very powerful player in the European market, but it knows it must branch out to gain serious traction. Will a name change do the trick?
-
ESET Discovers New Linux Malware
WolfsBane is an all-in-one malware that has hit the Linux operating system and includes a dropper, a launcher, and a backdoor.
-
New Linux Kernel Patch Allows Forcing a CPU Mitigation
Even when CPU mitigations can consume precious CPU cycles, it might not be a bad idea to allow users to enable them, even if your machine isn't vulnerable.
-
Red Hat Enterprise Linux 9.5 Released
Notify your friends, loved ones, and colleagues that the latest version of RHEL is available with plenty of enhancements.
-
Linux Sees Massive Performance Increase from a Single Line of Code
With one line of code, Intel was able to increase the performance of the Linux kernel by 4,000 percent.
-
Fedora KDE Approved as an Official Spin
If you prefer the Plasma desktop environment and the Fedora distribution, you're in luck because there's now an official spin that is listed on the same level as the Fedora Workstation edition.
-
New Steam Client Ups the Ante for Linux
The latest release from Steam has some pretty cool tricks up its sleeve.
-
Gnome OS Transitioning Toward a General-Purpose Distro
If you're looking for the perfectly vanilla take on the Gnome desktop, Gnome OS might be for you.