Pattern Seekers

A regular expression (regex) [1] is a sequence of characters that describes a search pattern. You can use a regex to save time searching and replacing in texts, such as strings in programming languages, database query results, or normal documents. Regexes also can help you effectively use utilities, such as grep [2], xmlgrep [3], and ugrep [4].

Ultimately, a regex's usefulness depends on the pattern you select. Formulating a regex that delivers precise results is no easy task (see the "DIY Regular Expressions" box). To save time, a regex generator can automate this step for you by taking a text/character string and deriving a suitable regular expression.

DIY Regular Expressions

While formulating a regular expression can take some time, especially for newcomers, it is worth the effort if you want faster results. Regular-Expressions.info [5], Regex DB [6], and Mastering Regular Expressions [7] all clearly explain strategies for creating regexes and provide a huge number of examples to show how useful regexes can be.

For instance, you can use a regular expression to test whether a credit card number entered on a website form is in the correct format. Figure 1 shows a regular expression for numbers on Visa credit cards: the number 4 followed by any three digits, then a separator (nothing, a space, or a hyphen), followed by up to three blocks of any four digits, which can again be followed by a separator. This regular expression does not validate the card, but it reliably filters out incomplete numbers and identifies typing errors.

Figure 1: Regular expressions offer a relatively easy approach to validating the structure of a Visa card number.

The RegEx101 wiki [8] also has a good reputation for testing your regular expressions. Not only does RegEx101 support different regex dialects, but it also shows you which patterns match and provides an explanation. Figure 2 shows a RegEx101 test for dates, where the individual components (day, month, and year) are given as decimal values separated by hyphens.

Figure 2: RegEx101 checking date input.

Regex generators work with varying degrees of precision. Some regex generators offer maximum precision, where the regular expression finds exactly one pattern. Others offer minimum precision, where the regex finds a set of patterns with a similar structure. In this article, I test several regex generators to determine how well they work (see also the "Nonfunctional" box).

Olaf Neumann's Regex Generator

Regex Generator [12], which is implemented in Kotlin and JavaScript, lets you create regular expressions via Olaf Neumann's website. The website offers several input boxes. First, you enter the text string for which you are searching. As an example, I used a typical log entry consisting of a date and a message, both separated by a space.

From your entered text in box 1, the software derives the individual components, which it highlights in different colors (shown in box 2). You then select the relevant component by clicking on the respective color or component. The component is then colored gray. Regex Generator then generates the appropriate regular expression from your selection and displays it in box 3 (Figure 3).

Figure 3: After entering the desired text string and selecting the components in Neumann's online Regex Generator, the appropriate regular expression is returned.

You can then select the programming language where you will use the regular expression. Currently, you can choose between several languages, including PHP, Ruby, and JavaScript (Figure 4).

Figure 4: After selecting the desired programming language, Regex Generator displays a regular expression (shown here as a PHP function).

In terms of precision, Regex Generator provides moderate accuracy. The date matches any data, but the text only matches the error message Login attempt failed in upper and lowercase thanks to the /i parameter at the end of the regular expression. Consequently, Regex Generator is adequate if you just want to search for failed logins for arbitrary dates, for example.

rgxg

ReGular eXpression Generator (rgxg) [13] generates regexes from the command line (Figure 5). You can find rgxg in the Debian, Ubuntu, and Linux Mint repositories and install it using familiar on-board tools. For better output readability, you should use rgxg in a terminal window with a dark background.

Figure 5: Regular expressions generated by the rgxg command-line program.

In practice, rgxg expects a call with different subcommands (see Table 1).

In addition, rgxg offers a whole range of parameters to further refine the generated expressions (see Table 2).

In terms of accuracy, rgxg offers maximum precision thanks to the parameters that let you generate a regular expression that exactly matches the given pattern.

txt2regex

Another command-line option, txt2regex [14] is a shell script with keyboard-based menu control. Like rgxg, you will find txt2regex in the repositories of most recent distributions, such as Debian or Fedora. On macOS, you can use Fink [15] for the install.

Due to the user interface's color scheme, you will want to use txt2regex in a terminal with a dark background. Alternatively, use the --nocolor parameter to switch off the color or --whitebg to switch to a suitable display for a light background. When txt2regex is running, typing an asterisk will toggle the display.

Txt2regex offers a helping hand when generating regular expressions by asking questions about the characters, including how many characters follow each other. Based on the input, txt2regex assembles the expression for various tools, such as awk, find, grep, PHP, and PostgreSQL.

You can access a complete list of dialects by typing a forward slash. You then select the corresponding letter from the list (Figure 6) to generate the regular expressions in the desired dialect (Figure 7).

Figure 6: Txt2regex supports a large number of dialects for regular expressions.

Figure 7: Txt2regex generates patterns for the selected dialects. This is an intermediate step in querying for the desired string.

Txt2regex requires precision. As you use txt2regex, you will learn to specify the pattern sequence precisely. Txt2regex provides an overview of how you need to specify the regular expression in the respective tool for it to recognize the pattern. This is a big help because various implementations and dialects are found in everyday Linux: regular expressions, extended regular expressions, and Perl-Compatible Regular Expressions (PCRE).

Like rgxg, txt2regex formulates regular expressions with maximum precision by using parameters to adapt the expression to your specified pattern.