Managing Systems and Applications with pyinfra
Scripted Setup
© Lead Image © Carlos Castilla Jimenez, 123RF.com
Keeping Linux machines in a known state requires a configuration management system. Discover how pyinfra simplifies this task with Python's full programming power.
When managing a few Linux machines, it's easy to handle them with ad-hoc commands, but this approach becomes inefficient as the number of machines increases. A configuration management tool ensures that all your machines have a consistent setup. Popular tools in this domain are Ansible, Puppet, Chef, SaltStack, and CFEngine. A lesser-known alternative, pyinfra [1], is the subject of this article. Unlike many other configuration management tools, pyinfra doesn't rely on YAML [2] or a domain-specific language. Instead, you configure your systems using Python code.
If you're familiar with Python, you'll find pyinfra easily accessible. Moreover, you can leverage the full Python language, including loops, conditions, functions, and classes to streamline your configuration tasks. You can even import any Python library to meet specific requirements, allowing you to harness the extensive Python ecosystem.
Under the hood, pyinfra translates your Python code into shell commands and executes them on your machines. This means the target machines don't need to have Python or any special dependencies; all that's required is a POSIX-compatible shell. There's no agent running on the target machines. Instead, pyinfra simply logs into machines via SSH to execute its shell commands.
As a downside, pyinfra isn't as widely used as other tools. Ansible, for example, offers numerous built-in functionalities and boasts a vast ecosystem of reusable packages. Pyinfra has more limited built-in functions, and its ecosystem isn't that big, which means there might be times when you need to implement solutions on your own. Fortunately, it's less daunting when you're dealing with Python.
Installing pyinfra
Because pyinfra is a Python package, you can install it using Python's package manager, pip, or using its more modern counterpart uv [3]. With uv, you can install pyinfra globally as a tool:
uv tool install pyinfra
Then confirm that the pyinfra command is recognized:
$ pyinfra --version pyinfra: v3.7
Inventory, Operations, and Facts
Three key concepts when working with pyinfra are an inventory, operations, and facts. An inventory is a collection of hosts, groups, and data. Hosts are the targets where pyinfra runs operations, and they can be assigned to groups. For example, you might define a group for all web servers to configure them uniformly. You can also assign data to individual hosts or to groups, such as a list of packages to install.
Operations come in two types. One type is a straightforward shell command to execute on a host. The other type is a state definition, which is an idempotent operation. This ensures that running it twice doesn't change the desired state if it has already been achieved. A state definition operation first compares the desired state against the current state and only makes changes when they differ. For instance, a state change might involve ensuring a specific package is installed.
Most operations in pyinfra are state definitions. You should consider using these over shell commands when possible, because this approach simplifies reasoning about your systems and enables optimizations during pyinfra execution. For instance, instead of explicitly running an apt get install command on your systems with pyinfra, opt for an apt.packages operation with the desired packages, and pyinfra will only perform installation if necessary.
To determine the current state of a system, pyinfra uses facts. These are read-only during pyinfra's run on your systems. Operations use these facts to generate shell commands that alter the system state. For instance, the apt.packages operation to install packages uses the deb.DebPackage fact to determine whether the packages are already installed.
Running Commands with the pyinfra CLI
Before writing Python code to configure your systems with pyinfra, you can explore the concepts of an inventory, operations, and facts through the command line. This will give you an idea about the tool's capabilities. All pyinfra commands on the command line follow the format pyinfra <inventory> <operations>.
In its simplest form, the inventory is just a hostname, and the operation is an exec command. For instance, if you want to check the kernel version a specific machine is running, just execute
pyinfra podpile.home exec -- uname -r
This command connects to the specified host over SSH, requesting your password or your key's passphrase if necessary, and runs a server.shell operation executing the uname -r command on the host. It shows the command's output, considers it successful, presents a table with the results, and then disconnects. If you add more hosts on the command line, separated by a comma, it will perform the same action on all hosts concurrently (see Figure 1).
You can also use this approach to keep an eye on the systemd journals of multiple systems in parallel:
pyinfra podpile.home,nov.home exec -- journalctl --follow
Because this is an operation that never finishes, pyinfra keeps showing you the command's output until you terminate it with Ctrl+C.
You can also use this to reboot one or more specific servers:
pyinfra podpile.home -- sudo server.reboot
Note the option --sudo, which directs pyinfra to execute the command of the server.reboot operation with sudo. Pyinfra prompts you for the sudo password before rebooting the machine. Don't be alarmed when you see a Python exception with a large stack trace: This occurs because the SSH connection is lost during the reboot.
After this, you can verify whether the server has really rebooted by checking its uptime:
pyinfra podpile.home exec -- uptime
Another use for pyinfra's command line is installing specific packages on your hosts:
pyinfra nov.home --sudo server.packages tmux vim
The server.packages operation is a state definition: It first checks whether the packages are installed and only installs them if they're absent. It uses your system's package manager, such as APT or DNF. Have a look at the pyinfra Operations Index [4] for an exhaustive list of provided operations.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Digital Autonomy
• How the Fediverse Found Parts for Its Platforms
• The Answer Was Already on the Shelf
• Changing the Chip Industry: How Public Investment Has Grown Open Silicon
• United Nations Open Source Portal Goes Live
• EU Open Source Strategy Plays Key Role in Tech Sovereignty Package
• France Says “Au Revoir” to Microsoft
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Epic Games Targets Linux for New Launcher
The Epic Games new launcher is still in preview, but it looks to land on several Linux distributions.
-
Fedora 45 Now Available
The latest iteration of Fedora has arrived with Gnome 51 and KDE Plasma 6.8.
-
Linux 7.4 is "Much Healthier" Thanks to AI
At nearly 42 million lines, Linux 7.4 is the largest merge in the kernel’s history.
-
Working Exploit for a Linux Kernel Flaw Made Public
Even before a bug received its CVE identifier or an upstream fix, the local privilege escalation vulnerability exploit found its way into the wild.
-
r/opensource Bans AI-Assisted and AI-Related Projects
The de facto subreddit for all things open source has not only banned AI-related projects but also users who "don't speak English."
-
Bottlenecks Discovered by Meta Engineers Lead to Scalability Improvements
Thanks to a small group of engineers at Meta, the Linux kernel module kernfs has been improved, which has led to some serious gains.
-
PwnEscape Poses VM Vulnerability
A guest-to-host escape flaw in Linux KVM, named PwnEscape, might be yet another means of attacking Linux.
-
Linux Mint 27 Set for a Christmas Release
The developers behind Linux Mint have announced a Christmas release, and it looks like Santa's sack will be brimming with features.
-
Is Linus Torvalds Vibe Coding?
The creator of Linux has added AI into his process for Linux kernel development, and it's more than just for debugging.
-
openKylin 3.0 Released with Expanded AI Integration
The latest release features a pre-installed AI agent to help perform tasks.
