From the Cockpit

Wouldn't it be wonderful if you could configure and control all your Linux systems from one friendly interface? More than twenty years ago, the answer to this wish was a project called Linuxconf [1], which stopped development in 2005 and is hardly missed. Linuxconf tried to do too much at once, too often in ways that clashed with the default management tools of most distributions. After Linuxconf came Webmin [2], which is still actively developed and useful; however, in my opinion, Webmin has a dated interface, and you need relatively good knowledge of Linux to use it properly.

The quest for a better admin tool led to the start of the Cockpit project [3] a few years ago. Cockpit is a free and open source, web-based interface for managing Linux systems. The official goals of the Cockpit project are to make "Linux servers usable by non-expert admins" and to make "complex Linux features discoverable" [4]. Cockpit is supported by Red Hat, but you can run it on any distribution. This tutorial explains how Cockpit works and how and why it might help you with simplifying and consolidating your Linux management tasks.

Advantages and Limits

The first thing to know about Cockpit is that it is not a configuration management system like Ansible [5] or Puppet [6]. You cannot tell Cockpit "I want all my Linux boxes to look like this" and then take a stroll while it executes your wishes. This limitation is also its strength, because Cockpit is deliberately light and therefore easy to use.

On the surface, Cockpit is clean, intuitive, and smooth, even in a browser with many other tabs open. The documentation says that graphical and interface designers are involved in the project, and it shows. The interface is much nicer than the standard, low-level Linux tools (Figure 1).

Figure 1: The Cockpit login screen to access the local computer on port 9090. Notice the "Reuse my password …" checkbox.

The Cockpit back end communicates with the Linux system it controls via systemd sockets [7]. Systemd sockets are connectors that do not use any memory when there is nothing to do but can activate a Cockpit component whenever it is needed.

By default, Cockpit does not store performance or status data, nor does it keep its own copy of the configuration for the computer it controls, except for the parameters necessary to connect with the computer.

Cockpit includes an embedded terminal that works over secure SSH connections. Another important benefit is the fact that one installation can control other remote machines that also run Cockpit, all from the same browser tab.

Cockpit has many capabilities that could potentially be of use to the average admin, but you might find that it is still useful even if you just need one or two of its functions.

Up Close

The Cockpit back end has three essential components. A component called cockpit-ws acts as something like a web server, talking with browsers through TCP port 9090, which is the port number reserved for web system manager services. Another executable program, called cockpit-session, takes care of authentication, using standard Linux mechanisms such as PAM or GSSAPI [8].

Once you are logged in, a component called cockpit-bridge creates and runs the cockpit session. Below the surface, Cockpit is just like any other ordinary session-based Linux tool (TTYs, X11, SSH, etc.); your login credentials, user privileges, SELinux settings, and other settings are exactly the same as if you had logged into the system from an ordinary prompt.

There is an exception, though. At a normal command prompt, you can always use sudo to run commands that require root privilege. The Cockpit graphical interface (not the embedded terminal), does not support a secondary authentication routine that would let you follow up a privileged command by entering your password. By default, if you click on any button corresponding to an operation that would require sudo, you get the error message (Figure 2). However, if you check the box labeled Reuse my password for privileged tasks in the Cockpit login screen (Figure 1), you allow Cockpit to transparently escalate your privileges when necessary. (Of course, this only works when your account is authorized to execute privileged commands through sudo.) Inside Cockpit, you can configure permission to execute root-level commands by selecting the Users tab and checking the box labeled Server Administrator (Figure 3).

Figure 2: Unless you check the "Reuse my password …" box at login, Cockpit will not allow you to execute sensitive, or potentially dangerous, actions.

Figure 3: Choose the Server Administrator role to configure the important parameters of Linux user accounts.

Cockpit Installation

Conceptually, the Cockpit installation procedure is very simple: first, install the Cockpit software, then configure the firewall to make that software accessible from the Internet. Next, point your browser to port 9090 of the computer, log in with your user account on the computer you are using, and start working. It is simpler than it sounds because all the steps are well supported and documented.

All images in this tutorial come from installation and usage of Cockpit on two Linux systems: a spare home desktop running Ubuntu 16.04 LTS and a CentOS 7.6 VPS that hosts some of my websites. One command was enough to get Cockpit running on the Ubuntu system:

#> sudo apt-get install cockpit

On the CentOS remote server, I had to type a bit more:

#> yum install epel-release
#> yum install cockpit
#> systemctl enable --now cockpit.socket
#> firewall-cmd --permanent --zone=public --add-service=cockpit
#> firewall-cmd --reload

The first command enables the EPEL repository that contains the Cockpit package for CentOS, the second installs it, and the third enables the socket that Cockpit will use to communicate with the system. The last two lines of that sequence change and reload the firewall configuration to make Cockpit accessible from the Internet.

These two installation examples show that the actual procedure depends on how Cockpit was packaged for each distribution; for instance, on CentOS, I had to explicitly enable the Cockpit socket. In practice, installation is very simple on all major Linux distributions because binary packages are available, and any extra commands you may need to type are well documented on the Cockpit website.

If you follow the documented procedures, Cockpit will be accessible from your browser without running any other server – at addresses like https://localhost:9090 if installed on your local computer or https://example.com:9090 on remote servers.

If you want, or need, to put Cockpit at different addresses, be it a subdomain like cockpit.example.com or a subfolder like example.com/cockpit, you'll need to hide Cockpit behind a web server like Apache or nGinx that acts as a reverse proxy, transparently relaying the traffic between Cockpit and the browsers of its users. You will find instructions and configuration files for this reverse proxy option online [9] [10] [11], but honestly, this configuration only seems worth it if the Cockpit installation must serve many different users without making any of them install any extra software. If one person wants to install and use Cockpit on several independent servers, a better solution is the multi-server approach I describe at the end of this article.

The Cockpit configuration file (/etc/cockpit/cockpit.conf) has a simple syntax, and all the options are described in the project documentation. However, I have experienced some discrepancies between the documentation and Cockpit's behavior in the wild (see the box entitled "Theory vs Practice").

What to Do with Cockpit

Cockpit lets you monitor, analyze, and configure Linux systems running Cockpit, as well as any container or virtual machines they host.

You get a quick, dynamic overview of the performance of all the systems in which you log in from the browser, and all are immediately reachable from the menu. Figures 4 and 5 show examples of the hardware performance diagrams and the storage management interface, which summarizes the status of all devices and partitions. All the charts created by Cockpit have a very clean style and, in my experience, their continuous refresh never slows down the browser. Figure 6 shows an equally clean summary of system logs.

Figure 4: The performance charts in the system section of a Cockpit control panel. The peaks in disk I/O and network traffic shown in the two bottom charts are due to the download of an ISO image.

Figure 5: The Cockpit server menu (left) and default storage panel layout, in privileged mode, showing partition and devices statuses, disk accesses (top), and storage logs (bottom).

Figure 6: Cockpit displays a summary of system logs.

In addition to its support for monitoring, Cockpit also lets you create and launch Linux or FreeBSD virtual machines and Docker containers, update software, manage LVM volumes, and define systemd timers to automatically run programs at specified times. You can also configure, stop, and restart system services or network interfaces. All the corresponding panels are basically visual representations of the main options you normally have at the command line, so I will not describe them in detail.

By default, the Cockpit charts only display the present status of a computer. By displaying the present status, the charts can give immediate, albeit qualitative feedback of what happens to your local or remote server if its load suddenly changes (e.g., if you start updating packages or uploading files). It is possible to enable persistent metrics by enabling an optional module, but this operation is distribution-dependent.

The Cockpit embedded terminal is visible in Figure 7. The terminal gives you a secure SSH session into any remote Linux box running Cockpit. Unless you block access, that SSH access will be possible even when you are borrowing someone's non-Linux computer or working from a public Internet kiosk. (You'll need to decide for yourself if an Internet kiosk is secure enough for what you are doing.)

Figure 7: The Cockpit embedded terminal, connecting via SSH to a server running Cockpit.

The most important thing to say about the terminal option is that you can use it when and how you want. The very fact that Cockpit is nothing more than a graphical intermediary between its user and a command prompt means that you can move back and forth between the Cockpit terminal and Cockpit GUI any way you want. Even the error messages you type in the terminal will show up in the Cockpit journal.

Managing Multiple Servers

Another feature of Cockpit that I like a lot is shown in Figure 8: the possibility to use a local copy of the software to manage any remote Linux machine, as long as it also runs Cockpit. In Figure 8, the Cockpit dashboard in the browser is connected to the Cockpit instance listening on port 9090 of the local machine that runs Ubuntu (see detail 1), but this instance can also control a remote CentOS server and display its performances in real time.

Figure 8: One Cockpit dashboard controlling two servers.

All the servers controlled from one Cockpit installation are listed in its dashboard. To add a server, click on the plus button of the dashboard (detail 2 of Figure 8) and add the server address in the pop-up window (Figure 9). Use the following syntax:

servername:portnumber

Figure 9: To add another server to the Cockpit dashboard, enter its name, SSH port number if needed, and a color for its graphs.

The optional port number refers to the number of the SSH port for that server, if it is different from the default value of 22. Please note that this procedure only works if you log into Cockpit with escalated privileges.

This scenario is also the main exception to the design rule that Cockpit does not store server data. With the exception of passwords, which, as far as I can tell, you have to retype every time, the parameters that are needed to connect to servers are stored in one JSON file in the /etc/cockpit/machines.d folder in the following format:

{
  "SERVER_NAME" : {
    "visible" : true,
    "color" : "rgb(103, 211, 0)",
    "user" : "USERNAME",
    "port" : "SSH_PORT_NUMBER",
    "address" : "ADDRESS"
  }
}

The ADDRESS can be either a domain name or a numeric IP address. The SSL certificates and keys needed for secure connections are, instead, stored inside the other Cockpit folder you need to back up regularly, /etc/cockpit/ws-certs.d.

Using Cockpit Securely

Making a server manageable from the Internet increases its exposure to attacks. As far as Cockpit is concerned, you can sensibly reduce that exposure in two simple steps.

One thing I really dislike about Cockpit is that, by default, it bypasses one basic security practice for SSH connections, which is "NO direct SSH logins with the root account!" The idea is that whoever wants to do root work using SSH should first log in with a normal account and then switch to root. In Cockpit, this barrier is bypassed, but the way to restore it is easy, though not well documented [12]. On each server where you want to forbid direct root logins via Cockpit, add the following setting:

auth requisite pam_succeed_if.so uid >= 1000

as the first line in the file /etc/pam.d/cockpit. This tells the Linux daemon for Pluggable Authentication Modules (pam.d) to refuse login requests from all privileged accounts that have numeric identifiers (uid) below 1000 (root's uid is zero).

The other step to reduce the exposure of a Cockpit-managed Linux server is what the Cockpit documentation calls the bastion host model. To begin with, configure the firewall of the host(s) where Cockpit runs to block external connections to the TCP port number 9090. On CentOS, these are the commands that would permanently close that port, and immediately apply the following configuration change without rebooting the system:

#> firewall-cmd --permanent --zone=public --remove-service=cockpit
#> firewall-cmd --reload

The trick here is these actions only block remote connections. Closing port 9090 on a computer does not prevent a browser and a Cockpit instance that both run on that same computer from communicating. Couple that with the fact that two Cockpit instances can directly, securely talk to each other through the same SSH port that should remain open anyway, and the problem is solved: A Cockpit instance running on your local computer will be all you need to control any other Cockpit-enabled server where you have an account. In Figure 10, the browser is connected to the local copy of Cockpit on the local Ubuntu desktop (detail 1). The Cockpit of the remote CentOS server is not reachable in the same way with a browser because its port 9090 was closed (detail 2). However, that same server is reachable from the local Cockpit (detail 3).

Figure 10: One Cockpit instance, controlling a remote server whose Cockpit is unreachable by browsers.

Cockpit Modules

Like many other software packages, Cockpit has a modular architecture, with a set of officially supported core modules and guidelines to develop and install third party modules, or addons. All the functions shown or described in this tutorial come from core modules that are not mandatory but are either included in the Cockpit basic packages for CentOS and Ubuntu or are declared as dependencies. These modules were therefore automatically installed by apt-get or yum. Third-party packages are another story: the documentation might not be up to date, and they might not work without debugging and manual fixes that you may or may not be able to afford. For example, I downloaded the Cockpit module for RSS feeds and placed it into the required Cockpit subfolder (addons), as explained in the documentation, but nothing happened.

Conclusion

Some Cockpit modules and documentation pages could benefit from a bit more attention, but that is no reason to avoid Cockpit. Webmin might offer finer control, and the command line might be more flexible, but Cockpit provides a nice, quick, user-friendly and mobile-friendly way to monitor and configure your physical or virtual Linux systems from the tab of any modern web browser. Cockpit is also useful as an aid for Linux trainers. A Cockpit dashboard is an effective way to monitor and configure all the computers in a Linux class, as well as a good tool for teaching about Linux configuration.

The Author

Marco Fioretti (http://stop.zona-m.net) is a freelance author, trainer, and researcher based in Rome, Italy, who has been working with Free/Open Source software since 1995 and on open digital standards since 2005. Marco is also a Board Member of the Free Knowledge Institute (http://freeknowledge.eu), and he blogs about digital rights at http://stop.zona-m.net.