Wrangling Structured Text on the Command Line
Mastering gawk
Process text, crunch data, and automate reporting with the GNU implementation of AWK.
Among the many command-line utilities available on a Linux system, few are as quietly powerful as GNU AWK (gawk), the GNU implementation of the AWK language. It has been a staple of Unix-like environments since the 1980s, yet it remains one of the most underappreciated tools in the average user's toolkit. Gawk is a pattern-scanning and text-processing language: It reads input line by line, splits each line into fields, tests whether each line matches a given pattern, and executes an associated action whenever a match is found. That seemingly simple model turns out to be extraordinarily versatile.
Unlike general-purpose scripting languages such as Python or Perl, gawk is purpose-built for structured text. It excels at tasks that would require dozens of lines of code in other languages but can often be expressed in a single gawk one-liner: summing a column of numbers, reformatting a CSV file, filtering log entries by date, or computing per-host statistics from an access log. Because it processes input line by line without loading the entire file into memory, gawk also scales gracefully to very large files.
This article walks through the most useful and frequently encountered use cases for gawk, starting from the fundamentals of its program structure and progressing through field manipulation, pattern matching, built-in variables, functions, and finally a set of practical, real-world recipes. By the end, you will have a solid foundation for using gawk whenever you need to wrangle structured text on the command line.
[...]
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Digital Autonomy
• How the Fediverse Found Parts for Its Platforms
• The Answer Was Already on the Shelf
• Changing the Chip Industry: How Public Investment Has Grown Open Silicon
• United Nations Open Source Portal Goes Live
• EU Open Source Strategy Plays Key Role in Tech Sovereignty Package
• France Says “Au Revoir” to Microsoft
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Epic Games Targets Linux for New Launcher
The Epic Games new launcher is still in preview, but it looks to land on several Linux distributions.
-
Fedora 45 Now Available
The latest iteration of Fedora has arrived with Gnome 51 and KDE Plasma 6.8.
-
Linux 7.4 is "Much Healthier" Thanks to AI
At nearly 42 million lines, Linux 7.4 is the largest merge in the kernel’s history.
-
Working Exploit for a Linux Kernel Flaw Made Public
Even before a bug received its CVE identifier or an upstream fix, the local privilege escalation vulnerability exploit found its way into the wild.
-
r/opensource Bans AI-Assisted and AI-Related Projects
The de facto subreddit for all things open source has not only banned AI-related projects but also users who "don't speak English."
-
Bottlenecks Discovered by Meta Engineers Lead to Scalability Improvements
Thanks to a small group of engineers at Meta, the Linux kernel module kernfs has been improved, which has led to some serious gains.
-
PwnEscape Poses VM Vulnerability
A guest-to-host escape flaw in Linux KVM, named PwnEscape, might be yet another means of attacking Linux.
-
Linux Mint 27 Set for a Christmas Release
The developers behind Linux Mint have announced a Christmas release, and it looks like Santa's sack will be brimming with features.
-
Is Linus Torvalds Vibe Coding?
The creator of Linux has added AI into his process for Linux kernel development, and it's more than just for debugging.
-
openKylin 3.0 Released with Expanded AI Integration
The latest release features a pre-installed AI agent to help perform tasks.
