Custom Shell Scripts
Text Processing with Variables
The most popular interpreters, and corresponding scripting languages for processing text strings, are probably Perl and Python. The current versions of Bash, however, include enough string processing operators to be more than adequate for the simplest and most common tasks. The main ones are:
#> NAME=Marco #> echo ${NAME%co} Mar #> echo ${NAME/co/36} Mar36 #> echo ${NAME^^c} MarCo #> NAME=MARCO echo ${NAME,,*} marco
The statements above show how the shell can perform removal, substitution, and case modification, respectively, on parts of a variable.
The %
operator removes the shortest matching pattern (co
in the example) from a variable, starting from its end. The slashes delimit which part of the string should be replaced, and which other string to use. The caret (^
) and comma (,
) operators convert any occurrence of the matched characters to uppercase (c
) or lowercase, respectively (the asterisk is a shortcut for "any character"). All these operators have other formats and variants: To know how they work, check the Bash man page [3].
It is also possible to extract and replace specific parts of a string, whatever their value. I will show how to do it, together with another very useful feature of the shell, with a final, simple script (Listing 2).
Listing 2
Replacing Specific Parts of a String
01 #! /bin/bash 02 03 FIRSTNAME=$1 04 SURNAME=$2 05 PHONE=$3 06 07 cat<<INITIAL_DATA 08 09 Initial Data: 10 First Name : $FIRSTNAME 11 Surname : $SURNAME 12 Phone Number : $PHONE 13 14 INITIAL_DATA 15 16 FIRSTNAME="${FIRSTNAME,,}" 17 FIRSTNAME="${FIRSTNAME^}" 18 19 20 SURNAME="${SURNAME,,}" 21 SURNAME="${SURNAME^}" 22 23 PHONE_1="${PHONE:0:3}" 24 PHONE_2="${PHONE:4:3}" 25 PHONE_3="${PHONE:6}" 26 27 cat<<FORMATTED_DATA 28 29 Formatted Data: 30 First Name : $FIRSTNAME 31 Surname : $SURNAME 32 Phone Number : ($PHONE_1) $PHONE_2-$PHONE_3 33 34 FORMATTED_DATA 35 exit
This code is a really bare-bones formatter of names and phone numbers: No matter how you write them, the script returns name and surname with the first letter capitalized and the phone number in a standard format.
Listing 3 shows what you get when you run the script from Listing 2.
Listing 3
Output from
#> phone-formatter LEia ORGaNA 5553236856 Initial Data: First Name : LEia Surname : ORGaNA Phone Number : 5553236856 Formatted Data: First Name : Leia Surname : Organa Phone Number : (555) 236-6856
As is, the script is not complete, because it only formats one entry at a time. In a future installment, I will show how to work on multiple records. Even in this form, however, the code is useful as a first approach to text formatting and parsing, because the same tricks may be adapted, for example, to process generic spreadsheets in CVS format.
In Listing 2, lines 3 to 5 show how you can pass different parameters to a script every time you run it. The variables $1
, $2
, and $3
contain the parameters that were passed in from the command line. You may use them directly, but for readability, it is much better to copy them into $FIRSTNAME
, $SURNAME
, and $PHONE
.
Line 7 marks the beginning of a so-called "here document," which ends on line 14; cat
, a Linux command, simply prints out whatever you pass to it.
A here document is a really cool way to use variables and generate shell output. You can think of a here document as a pseudo-file or, more appropriately, template that is embedded into a shell script. The template ends with a line containing the same string used to mark its beginning (INITIAL_DATA
).
Inside the template, you can put as many variables as you like, and every time you use it, it will contain the current values of all those variables.
You can use here documents to custom generate any kind of complex, pre-formatted data on the fly, from database records to sequences of commands to pass to any other program.
Lines 16 to 21 just apply the case modification tricks already presented, and lines 23 to 25 show how to extract substrings. The final part of the script uses another here document to display the reformatted input values.
Conclusion
In this installment, I have covered the basic components that make shell scripts so flexible and usable. Next month, I'll play with more complex data structures like shell arrays and hashes. In the meantime, I highly recommend that you copy the provided script [5] and experiment both with the string processing commands and the here document layout to determine how much you can customize them. Happy scripting!
Infos
- Bash: http://www.gnu.org/software/bash/
- "Tutorial – Desktop News Feeds" by Marco Fioretti, Linux Magazine, issue 217, December 2018, pp. 90-95: http://www.linuxpromagazine.com/Issues/2018/217/Read-Me
- Seven expansion types: https://tiswww.case.edu/php/chet/bash/bash.html
- Generate random numbers in a given range: https://bytefreaks.net/gnulinux/bash/bash-get-random-number-that-belongs-in-a-range
- Code in this article: ftp://ftp.linux-magazine.com/pub/listings/linux-magazine.com/219/
« Previous 1 2 3
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.
-
ZorinOS 17.1 Released, Includes Improved Windows App Support
If you need or desire to run Windows applications on Linux, there's one distribution intent on making that easier for you and its new release further improves that feature.
-
Linux Market Share Surpasses 4% for the First Time
Look out Windows and macOS, Linux is on the rise and has even topped ChromeOS to become the fourth most widely used OS around the globe.
-
KDE’s Plasma 6 Officially Available
KDE’s Plasma 6.0 "Megarelease" has happened, and it's brimming with new features, polish, and performance.
-
Latest Version of Tails Unleashed
Tails 6.0 is based on Debian 12 and includes GNOME 43.
-
KDE Announces New Slimbook V with Plenty of Power and KDE’s Plasma 6
If you're a fan of KDE Plasma, you'll be thrilled to hear they've announced a new Slimbook with an AMD CPU and the latest version of KDE Plasma desktop.
-
Monthly Sponsorship Includes Early Access to elementary OS 8
If you want to get a glimpse of what's in the pipeline for elementary OS 8, just set up a monthly sponsorship to help fund its continued existence.