Run Samba in clustered mode with Ceph
Step 1: Get Ceph Ready for Action
The first step toward setting up Samba on a cluster is to determine the data storage solution; in this example, the solution is CephFS on a Ceph cluster (for alternatives, see the box entitled "Plan B for Ceph and Samba"). It is very good idea to use the current version of Ceph, which was Jewel 10.2.0 when I wrote this article, because the developers only guarantee stability as of this version of CephFS. In this example, I used Ubuntu 16.04; the Ceph project already has pre-built packages with the required Ceph version [1].
Plan B for Ceph and Samba
If you are looking for an alternative to the configuration described in this article, you can combine Ceph and Samba in other ways. For instance, Red Hat's developers have written a VFS module for Samba that is coupled directly to the RADOS library and can talk directly to the object store.
The advantages of this VFS solution are obvious: On one hand, the need for a local CephFS mount on the Samba server is removed. As a result, the deadlock problem mentioned previously in the context of releasing storage elegantly disappears. Additionally, it is no longer necessary to run metadata servers because you are not using CephFS.
The VFS module for Samba is included in all versions of Samba from 4.2 (Figure 1). If you find a more recent package in your distribution, you can the opt for the vfs_ceph
module. Look online for details of the drivers and the necessary configuration [2].
You can use ceph-deploy
to set up the Ceph configuration. This tool prepares the three computers for Ceph, installs Ceph, and also lets the admin activate the disks in Ceph. The first step, then, is to install ceph-deploy
. Although a package with the tool is included in Ubuntu, I recommend using the ceph-deploy
version from the Red Hat repository (Figure 2). You need to enable the repository up front by typing
wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add -
for the GPG key and then
sudo apt-add-repository 'deb http://download.ceph.com/debian-jewel/xenial main'
for the packages themselves. Then, install the tool on your system with:
apt-get update && apt-get install ceph-deploy
You only need to run these steps on one of the three systems.
Preparing the Servers for Ceph
For Ceph and ceph-deploy
to work, you need a working SSH connection. On each host, you should also have a user named ceph who runs ceph-deploy
. The user should also be able to use sudo
without a password to run the commands on the systems as root. The following commands will do the trick:
echo "ceph ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/ceph sudo chmod 0440 /etc/sudoers.d/ceph
The ceph-deploy
command uses SSH to execute commands within the cluster. To avoid the need for many passwords, the ceph user should be able to connect freely between the servers using SSH without a password.
The ssh-copy-id
command copies the public part of an SSH key to a remote host. The default username should also be set to ceph in the ~/.ssh/config
file on the other servers in the Ceph cluster, which will help you avoid ceph-deploy
trying to log in as root.
The three computers and their IP addresses on all three hosts must be available in /etc/hosts
. It makes sense to make one of the three nodes the master and use it for the ceph-deploy
calls, although this has no effect on the functionality of Ceph.
Rolling Out the Ceph Cluster
Once ceph-deploy
is ready to run, the other steps of the Ceph installation are almost automatic. The install
command in ceph-deploy
makes sure the necessary packages for Ceph exist on all specified hosts. The command
ceph-deploy install --release jewel alice bob charlie
run on the imaginary master host automatically installs Ceph on all three computers.
The cluster setup follows. Just as a reminder: Ceph consists of three components: Object Storage Devices (OSD), Monitoring Servers (MON), and Metadata Servers (MDS). The OSDs are the hard drives that the cluster requires. The MDSs extend the Ceph object store to include POSIX capabilities for use as a filesystem. The MONs are the most important components of Ceph: They monitor the quorum and ensure cluster operation.
When rolling out Ceph, the MONs are the first components you need to turn to. For installation in a new cluster, ceph-deploy
has the new
command. On the master server,
ceph-deploy new alice bob charlie
automatically rolls out a new Ceph cluster on the three hosts; At the end of the process, ceph-mon
is running on each of the hosts. Once ceph-deploy
has completed its work, the ceph -w
command on the first of the three hosts will identify a working cluster.
The next step is to set up the disks: On Alice, Bob, and Charlie, /dev/sdb
is intended for Ceph. The command
ceph-deploy create alice:sdb
(similar for Bob and Charlie) adds the disks to the cluster. If you want to outsource the OSD's internal journal to a fast solid-state drive (SSD), you can define the target partition by tagging :<partition>
to the end of the command, as in
ceph-deploy create alice:sdb:/dev/sdf1
where /dev/sdf1
is a partition on a SSD.
Because CephFS will be used in the cluster, you'll need metadata servers. You install the metadata servers with the ceph-deploy
command, as well:
ceph-deploy mds create alice bob charlie
Another call to ceph -w
shows a new situation: Now you have a Ceph cluster with three MONs, three OSDs, and three MDSs, and it is ready for operation (Figure 3).
Incidentally, if you want to operate this cluster later on, you will need physical disks. The usual Ceph tips for redundancy and performance apply: You should have three individual servers, and if you want to out-source the OSD journals to fast SSDs, you will want to put more than four OSD journals on a single SSD. The throughput is a practical measure of a Ceph cluster's quality – a fast connection of each Ceph node certainly won't hurt.
« Previous 1 2 3 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Fedora 41 Released with New Features
If you're a Fedora fan or just looking for a Linux distribution to help you migrate from Windows, Fedora 41 might be just the ticket.
-
AlmaLinux OS Kitten 10 Gives Power Users a Sneak Preview
If you're looking to kick the tires of AlmaLinux's upstream version, the developers have a purrfect solution.
-
Gnome 47.1 Released with a Few Fixes
The latest release of the Gnome desktop is all about fixing a few nagging issues and not about bringing new features into the mix.
-
System76 Unveils an Ampere-Powered Thelio Desktop
If you're looking for a new desktop system for developing autonomous driving and software-defined vehicle solutions. System76 has you covered.
-
VirtualBox 7.1.4 Includes Initial Support for Linux kernel 6.12
The latest version of VirtualBox has arrived and it not only adds initial support for kernel 6.12 but another feature that will make using the virtual machine tool much easier.
-
New Slimbook EVO with Raw AMD Ryzen Power
If you're looking for serious power in a 14" ultrabook that is powered by Linux, Slimbook has just the thing for you.
-
The Gnome Foundation Struggling to Stay Afloat
The foundation behind the Gnome desktop environment is having to go through some serious belt-tightening due to continued financial problems.
-
Thousands of Linux Servers Infected with Stealth Malware Since 2021
Perfctl is capable of remaining undetected, which makes it dangerous and hard to mitigate.
-
Halcyon Creates Anti-Ransomware Protection for Linux
As more Linux systems are targeted by ransomware, Halcyon is stepping up its protection.
-
Valve and Arch Linux Announce Collaboration
Valve and Arch have come together for two projects that will have a serious impact on the Linux distribution.