HPC in the cloud
Cloud computing is most definitely here, but does it have a role in HPC? We discuss changes in HPC that could be solved effectively by cloud computing.
I'm not a big Bob Dylan fan, but when it comes to HPC, "The Times They Are a-Changin'." Some of this change is going to the cloud, not because it would be really cool, but rather because HPC has existing workloads that fit well into cloud computing and can save money over traditional solutions. Perhaps more importantly, I think HPC has evolved to include non-traditional workloads and is adapting to meet those workloads – in many cases using cloud computing to do so. I will explain by giving two examples.
1. Massively Concurrent Runs
At one HPC center that I'm familiar with, users periodically submit 25,000 to 30,000 jobs as part of a parameter sweep; that is, they run the same application with 25,000 to 30,000 different data sets. Many times the application is a Matlab script, a Python script, an R script, a Perl script, or something particularly serial (i.e., it runs well on a single core). The same script is run but with thousands of different input files, resulting in the need to run thousands of jobs at the same time. Many times these applications run fairly quickly – perhaps in a couple of minutes – and many times they do not produce a great deal of data.
A closely related set of researchers study operating systems and security, during which they run different simulations with different inputs. For example, they might run 20,000 instances of an OS, primarily a kernel, and explore exploits against that OS. As with the previous set of researchers, the goal is to run a huge number of simulations as quickly as possible to find new ideas about how to protect an OS and kernel. The run times are not very long, but they must run the tests against a single OS. Consequently, they run thousands of jobs at the same time and then look through the results and continue with their research.
Read full article as PDF:
Version 16 of the popular Linux desktop reveals new tools, edge-snapping, and performance improvements.
Symantec says Linux-Darlioz burrows in through PHP.
Dell renews its quest for the ultimate developer machine.
Innovative back door looks like normal SSH traffic.
One of CeBITs most successful forums opens the new year with a new name. The popular Open Source Forum continues in 2014 under the name Special Conference: Open Source. This year, the forum will be bigger and offer a wider range of possibilities for sponsors.
New release offers better graphics drivers and expands filesystem support.
New mail protocol will shut out the NSA and prevent snooping on metadata.
A new web application helps users visualize distributed denial-of-service attacks.
Ubuntu 13.10 takes a step toward convergence, with lots of mobility, but Mir only partly here.
Galileo board is targeted to embedded developers and educational institutions.