Storm is brewing – HPC Cloud

At the 2009 Supercomputing conference in Portland last year, Platform Computing showed off our our first generation cloud computing management tool, Platform ISF. At that time, the “cloud” buzzword was still fairly new to the HPC community, and had several stern critics in the HPC space. To many HPC folk, “cloud” meant virtualization, and virtualization meant low performance. Very few other vendors at the conference last year even used the C-word, and when they did it was to describe other types (e.g. enterprise computing, dynamic datacenter provisioning, etc) of computing. So for a while, many believed as we did, that virtualization takes the “H” out of “HPC.”

In contrast to last year’s conference, this year both software vendors and infrastructure vendors were present talking about making cloud adoption easy, with every hardware vendor trying to persuade potential customers that building a private cloud using their hardware was a smart choice – especially when the vendor offers their own IaaS model for workload overflow (Platform calls that “Cloud Bursting”).

Also in contrast to 2009, this year has shown hypervisors and processors alike have matured to better support near hardware performance with virtualization. Indeed the performance chasm for some applications has narrowed to a crack (For more on this see our Platform whitepaper). Also, perceptions of the cloud have started to change in the HPC community. For the correct jobs, virtualization doesn’t have to mean there’s an unacceptable performance burden, and the advantages it brings to management, not to mention flexibility, are hard to ignore.

This year at SC’2010 we gave almost the same demonstration with more polish. The difference was the reaction had turned from disdain and skepticism to curiosity and interest. Yes, there are still several issues that need to be sorted before cloud computing is simple for HPC (licensing, data movement, and data security are the biggies). Nevertheless, HPC users are finally beginning to think about the cloud and performance is becoming less and less of an issue. Amazon, for instance, let their HPC performance data walk and talk for itself at the show. A cluster using HPC on EC2 placed at 230th in the TOP500(see So there’s no debating it--you can do HPC in the external public cloud – at least if you’re running Linpack.

Even if your application may be difficult to adapt to the cloud, the barriers are falling one by one. So taking the longer term view, in the next 5 years, HPC in the cloud doesn’t seem only feasible, it seems--as we a Platform Computing believe--inevitable.


Post a Comment