Creating virtual clusters with Rocks

Administering Virtual Compute Nodes

After a virtual compute node is created, you can query the VM state from the front-end node to make sure the virtual machine boots successfully. To query the state of the compute nodes, execute the command rocks list host compute-0-0-1; output similar to Listing 1 will provide information about the currently installed virtual machines and their states on all VM containers. (The status output is blank because processes have not been scheduled to run on the newly created VM.) Also, you can check the state of the VMs on the VM container itself with Xen xm commands (Listing 2).

Listing 1

Checking Vitual Machine Status

01 [root@cluster ~]# rocks list host vm
02
03 VM-HOST        SLICE  MEM   CPUS   MAC                         HOST                                 STATUS
04 compute-0-0-1: 1             512      1          00:16:3e:00:00:02     vm-container-0-0               active
05 compute-0-0-2: 2             512      1          00:16:3e:00:00:03     vm-container-0-0               active
06 compute-0-0-3: 3             512      1          00:16:3e:00:00:04     vm-container-0-0               active

Listing 2

The xm Command

01 [root@vm-container-0-0 ~]  xm list
02 Name                                                           ID     Mem(MiB)    VCPUs    State     Time(s)
03 Domain-0                                                     0       1024               1              r------    4035.5
04 Compute-0-0-1                                             2       512                 1              -b-----    23174.3
05 Compute-0-0-2                                             3       512                 1              r------     3228.3
06 Compute-0-0-3                                             4       512                 1              -b-----     2114.3

The rocks command is the primary administration command for the entire Rocks management system. The basic command-line structure is rocks <command> <arguments>. A full list of Rocks commands is available at the rocksclusters.org website or by typing rocks list help at the command line. The rocks command allows you to start and stop compute nodes, change configurations, and query configuration entries.

Rocks includes distributed administration utilities for executing commands against an entire cluster or group of cluster nodes. The default command for distributed command execution is tentakel, which is described as a program for executing the same command on many hosts in parallel. Tentakel is simple to use, and Rocks automatically adds all cluster nodes to /etc/tentakel.conf in various groupings. To execute a command against all nodes in /etc/tentakel.conf, type tentakel <command>. To execute a command against a group of nodes, execute tentakel -g <group_name> <command>.

Clustered Applications

To utilize all of the cluster nodes in a single application, it must be designed for the cluster. OpenMPI is a software library that is commonly used to design distributed applications for running on clusters. Sun Grid Engine and Torque are queuing systems for distributing jobs among the elements of the cluster. Although the problem of designing and implementing applications to use all cluster resources is an extensive topic, you will find resources online about the use of Sun Grid Engine, Torque, and Condor for these purposes.

Conclusion

Rocks' ease of use and support of the most common distributed applications makes it a favorite among scientific supercomputing facilities working on unlocking the mysteries of the universe.

The Author

Matthew Sacks is a Systems Administrator and writer from Los Angeles, CA. Check out his blog at http://matthewsacks.com/techblog/. Special thanks go to the Rocks teams at NSF and to the UCSD Supercomputing Center.

Read full article as PDF:

038-041_rockclusters.pdf  (543.34 kB)

Related content

  • Rocks Releases Mamba

    The latest version of Rocks cluster distribution – an open source toolkit for real and virtual clusters – has been released.

  • Proxmox VE

    The Proxmox Virtual Environment has developed from an insider’s tip to a free VMware ESXi/ vSphere clone. We show you how to get started setting up a PVE high-availability cluster.

  • OpenSSI

    The OpenSSI framework rearranges processes for easy and transparent clustering.

  • PelicanHPC

    Crunch big numbers with your very own high-performance computing cluster.

  • StarCluster

    Cloud computing has become a viable option for highperformance computing. In this article, we discuss the use case for cloud-based HPC, introduce the StarCluster toolkit, and show how to build a custom machine image for compute nodes.

comments powered by Disqus

Direct Download

Read full article as PDF:

038-041_rockclusters.pdf  (543.34 kB)

News

njobs Europe
What:
Where:
Country:
Njobs Netherlands Njobs Deutschland Njobs United Kingdom Njobs Italia Njobs France Njobs Espana Njobs Poland
Njobs Austria Njobs Denmark Njobs Belgium Njobs Czech Republic Njobs Mexico Njobs India Njobs Colombia