Set up your own lab environment with KVM, Qemu, and Libvirt
Playground
If you don't have room on your desk for a whole laboratory of servers, simply hitch up a virtual playground on your own workstation.
It is better to test any network configuration you will one day have to depend on. If you're lucky, the application you are testing runs as a standalone tool. In the real world, however, you might not be so lucky. For instance, you might need to test a website before you make it live on the Internet, or perhaps you want to experiment with a network backup system before implementing it on your local network.
In the old days, programmers, testers, and documentation specialists often sat with three or four different computers on their desk as they tested various networking scenarios. In today's world, you can model a whole network on one computer using virtualization. Before you gum up your whole network adding a new music server or remote monitoring system, test the configuration on your laptop and work out the kinks.
This article describes how to set up a test network on a Linux portable computer. In this case, the configuration consists of three virtual systems that collect performance metrics, save the information to a database, and provide an interface for visualizing the data. You could easily adapt the techniques described in this article for other applications: file server, media center, web server, or any other networked configuration you need to test.
This solution depends on the KVM and Qemu virtualization tools described in another article in this issue, along with the libvirt virtualization API [1], which is present or available in the repositories of most popular Linux systems. The virsh
command-line utility and the virt-manager graphical management tool serve as convenient interfaces to the libvirt virtualization environment. The configuration described in this article all took place on a single Lenovo ThinkPad computer (see Table 1). For a scenario like the one described in this article, I recommend a minimum of four usable CPU cores, 6GB RAM, and 200GB of free disk space.
Table 1
Reference System
Component | Equipment/version |
---|---|
Model |
Lenovo ThinkPad 460p |
CPU |
Intel Core i7-6700HQ (2.60GHz) |
RAM |
32GB DDR4 (2133MHz) |
Hard disk |
Samsung SATA SSD (512GB) |
Operating system |
Arch Linux (64 Bit) |
Kernel |
4.10.13 |
Qemu |
2.9.0 |
Libvirt |
3.2.0 |
Careful Planning
Plan your virtual network carefully. You should only set up new systems when you really need them. Ideally, you should also delete these virtual machines (VMs) later through a targeted clean up operation or at least move them onto an external hard disk.
Every virtual system on the playground should serve only one purpose so that tests run isolated from one another. In some cases, virtual systems might need to communicate with each other (such as high-availability applications). Group related applications and systems in their own lab networks and isolate them from other systems.
The virtual lab in this article consists of three VMs (Figure 1). The collectd01 VM collects metrics from the local system and sends them to the influxdb01 VM. The Grafana visualization tool runs in the third virtual system grafana01, which visualizes the metrics. See Table 2 for some details on the virtual systems. The VMs all use Ubuntu 16.04 as the operating system, and the environment does not use a firewall. Unless noted, all commands within the VMs are issued as root; the commands on the host system assume a normal user with sudo rights.
Table 2
Lab Systems
Host name | OS | CPUs | RAM | Disk | IP Address | MAC Address |
---|---|---|---|---|---|---|
collectd01 |
Ubuntu 16.04 |
1 |
1GB |
20GB |
|
|
influxdb01 |
Ubuntu 16.04 |
1 |
2GB |
30GB |
|
|
grafana01 |
Ubuntu 16.04 |
1 |
2GB |
20GB |
|
|
Configuration Work
If you use a typical tool such as virsh or virt-manager for creating the VMs, the configuration data will end up under /etc/libvirt/
, with the VMs under /var/lib/livirt/images/
. However, this approach creates problems later if you wish to transport the virtual lab to another host system. It is better to create a separate folder on an external or internal hard drive with enough space for each new playground.
In the example, I will format an external hard disk with the ext4 filesystem and mount it on my notebook system under /srv/demo/
. Now create the virtual-lab/
folder and some subdirectories in this directory; Figure 2 shows the directory structure.
The subdirectories shown in Figure 2 contain both the storage location of the VM images and adequate space for configuration files. You will need to create a storage pool and a network manually. Starting from the /srv/demo/virtual-lab/
directory, create and open the config/conf/networks/virtual-lab.xml
file. Fill it with the contents of Listing 1.
Listing 1
Configuring Networking
<network> <name>virtual-lab</name> <uuid>13f7adc7-475c-4e17-a0cd-4a80b94a70d7</uuid> <bridge name="virtual-lab" /> <mac address='02:f1:21:33:07:9c'/> <forward mode='route'/> <domain name='virtual-lab.test' localOnly="yes" /> <dns> <forwarder addr="8.8.8.8"/> <host ip='172.100.100.11'> <hostname>collectd01</hostname> </host> <host ip='172.100.100.21'> <hostname>influxdb01</hostname> </host> <host ip='172.100.100.31'> <hostname>grafana01</hostname> </host> </dns> <ip address="172.100.100.1" netmask="255.255.255.0"> <dhcp> <range start="172.100.100.100" end="172.100.100.254" /> </dhcp> </ip> </network>
The next step is to create and open the config/conf/storage/virtual-lab.xml
configuration file and transfer the contents from Listing 2. With the commands shown in Listing 3, activate the new virtual network and storage pool.
Listing 2
Configuring Storage
<pool type='dir'> <name>virtual-lab</name> <uuid>1c2b3c2d-0140-1e49-a571-840f4c88210d</uuid> <capacity unit='bytes'>0</capacity> <allocation unit='bytes'>0</allocation> <available unit='bytes'>0</available> <source> </source> <target> <path>/srv/demo/virtual-lab/storage/vm</path> <permissions> <mode>0755</mode> <owner>-1</owner> <group>-1</group> </permissions> </target> </pool>
Listing 3
Activating Network and Storage
$ sudo virsh net-create /srv/demo/virtual-lab/config/conf/networks/virtual-lab.xml $ sudo virsh pool-create /srv/demo/virtual-lab/config/conf/storage/virtual-lab.xml
Then open the administration tool virt-manager and create three new VMs. To gain access to the Internet during the installation, issue an appropriate iptables command (Listing 4, first line). Replace the wlp3s0
interface with an interface that has access to the Internet on your computer.
Listing 4
Accessing the Network
$ sudo iptables -t nat -A POSTROUTING -o wlp3s0 -j MASQUERADE $ sudo sh -c "echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.d/demo" $ sudo sysctl -p /etc/sysctl.d/demo
This configuration works if your system allows the forwarding of IPv4 packets. To allow IP forwarding, create the /etc/sysctl.d/demo
file with administrative rights, store the appropriate entry, and then activate packet forwarding via sysctl (Listing 4, second and third lines).
Take care when installing the test systems to select Select or create custom storage under Step 4 and click on Manage (Figure 3). Then select the virtual-lab storage pool and click on the plus icon next to Volumes (Figure 4). A new window opens, where you can specify the name for the image file and the capacity (Figure 5).
For the first VM, select collectd01.qcow2 and 20GB. Press Finish, select the new disk, and press Choose volume. Return to the Create a new virtual machine dialog. Now enter the name for the VM (such as collectd01), unfold Network selection, and make sure that Virtual network 'virtual lab': NAT is active.
Boot the VM and the operating system installation by pressing Finish. Set up all three virtual systems in this way, and then shut down all the systems. I was asked for a user account during the installation and entered demo as the username and password.
To guarantee that each of the three new lab VMs will receive the same IP address with each launch, store the MAC address for each virtual system in the network configuration. You can find the MAC addresses by double clicking a VM in virt-manager and jumping to the NIC entry using the information icon (Figure 6).
Now make sure all new VMs are shut down and stop the virtual lab network (Listing 5, first line). Add the DHCP entries to the network configuration for each VM. The configuration should roughly match Listing 6. (Replace the MAC addresses with the addresses for your systems.) Then relaunch the network with the command in the second line of Listing 5.
Listing 5
Stopping and Relaunching
$ sudo virsh net-destroy virtual-lab $ sudo virsh net-create /srv/demo/virtual-lab/config/conf/networks/virtual-lab.xml
Listing 6
Adding DHCP
<network> <name>virtual-lab</name> <uuid>13f7adc7-475c-4e17-a0cd-4a80b94a70d7</uuid> <bridge name="virtual-lab" /> <mac address='02:f1:21:33:07:9c'/> <forward mode='route'/> <domain name='virtual-lab.test' localOnly="yes" /> <dns> <forwarder addr="8.8.8.8"/> <host ip='172.100.100.11'> <hostname>collectd01</hostname> </host> <host ip='172.100.100.21'> <hostname>influxdb01</hostname> </host> <host ip='172.100.100.31'> <hostname>grafana01</hostname> </host> </dns> <ip address="172.100.100.1" netmask="255.255.255.0"> <dhcp> <range start="172.100.100.100" end="172.100.100.254" /> <host mac='52:54:00:b2:66:b6' name='collectd01' ip='172.100.100.11'/> <host mac='52:54:00:ec:58:8e' name='influxdb01' ip='172.100.100.21'/> <host mac='52:54:00:f9:9e:9a' name='grafana01' ip='172.100.100.31'/> </dhcp> </ip> </network>
At the end of the configuration work on the virtual lab, you need to move the VM configuration files. Navigate to the command line in the directory for the VM configuration (Listing 7, first line) and move the configuration files (Lines 2 to 4).
Listing 7
Moving the Config Files
$ cd /srv/demo/virtual-lab/config/conf/machines $ sudo mv /etc/libvirt/qemu/collectd01.xml . $ sudo mv /etc/libvirt/qemu/influxdb01.xml . $ sudo mv /etc/libvirt/qemu/grafana01.xml .
Starting up Virtual Lab
At this point, you have set up the new virtual playground on your system and put all the necessary data in one directory, so you can easily transport it onto an external hard drive and integrate it with other systems.
Launch the three test systems using the virsh create
command (Listing 8).
Listing 8
Launching the VMs
sudo virsh create /srv/demo/virtual-lab/config/conf/machines/collectd01.xml sudo virsh create /srv/demo/virtual-lab/config/conf/machines/influxdb01.xml sudo virsh create /srv/demo/virtual-lab/config/conf/machines/grafana01.xml
Virt-manager displays the three test systems (Figure 7). Now manage the trio using the console (or just via virt-manager) or install the openssh-server
package and connect to an SSH client (example: ssh demo@172.100.100.11
). Perform a few connection tests to check whether the network connections are working. A short test with ping linux-magazine.com
and ping grafana01
, ping influxdb01
, as well as ping collectd01
is sufficient for each VM.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
New Slimbook EVO with Raw AMD Ryzen Power
If you're looking for serious power in a 14" ultrabook that is powered by Linux, Slimbook has just the thing for you.
-
The Gnome Foundation Struggling to Stay Afloat
The foundation behind the Gnome desktop environment is having to go through some serious belt-tightening due to continued financial problems.
-
Thousands of Linux Servers Infected with Stealth Malware Since 2021
Perfctl is capable of remaining undetected, which makes it dangerous and hard to mitigate.
-
Halcyon Creates Anti-Ransomware Protection for Linux
As more Linux systems are targeted by ransomware, Halcyon is stepping up its protection.
-
Valve and Arch Linux Announce Collaboration
Valve and Arch have come together for two projects that will have a serious impact on the Linux distribution.
-
Hacker Successfully Runs Linux on a CPU from the Early ‘70s
From the office of "Look what I can do," Dmitry Grinberg was able to get Linux running on a processor that was created in 1971.
-
OSI and LPI Form Strategic Alliance
With a goal of strengthening Linux and open source communities, this new alliance aims to nurture the growth of more highly skilled professionals.
-
Fedora 41 Beta Available with Some Interesting Additions
If you're a Fedora fan, you'll be excited to hear the beta version of the latest release is now available for testing and includes plenty of updates.
-
AlmaLinux Unveils New Hardware Certification Process
The AlmaLinux Hardware Certification Program run by the Certification Special Interest Group (SIG) aims to ensure seamless compatibility between AlmaLinux and a wide range of hardware configurations.
-
Wind River Introduces eLxr Pro Linux Solution
eLxr Pro offers an end-to-end Linux solution backed by expert commercial support.