Cloud Computing is Getting Less Cloudy

Well, at least for me. I just attend the 2nd Annual Cloud Computing World Forum in London this week and after dozens of presentations, many conversations at the stand, and sifting through all the conference material, I believe my understanding of cloud is ‘less cloudy.’ Why? Because cloud computing is not just one thing that you can point to; it’s many use cases and I think I heard almost all of them here, from cloud bursting to SaaS services. But one thing that became clear is that cloud can be almost anything so long as ‘shared resources’ form its underpinnings.

Other than shared resources, there was another thing about cloud computing that also became clear to me at the conference - all cloud implementations tend to have three key capabilities. The first is the capability to manage resources. This means the management tools implementing cloud must be able to recognize all types of resources – or in other words heterogeneous resources. Cloud management tools must be able to manage these heterogeneous resources through provisioning mechanisms, whether physical or virtual. The second is the capability to manage the software stack, or at least part of the software stack. Monitoring utilization data characteristics from physical resources is common, but monitoring utilization data in the software stack is less common, and certainly not generally inclusive of the entire software stack (IaaS, PaaS, and SaaS) under a single tool. But even if there were a single management tool that could monitor the entire software stack and had a dashboard that could clearly communicate the state of each tier within the stack, manual changes to the infrastructure to improve performance and efficiency would still be necessary. So this brings up a third capability that is needed: the ability to manage changes in the infrastructure through a set of policies. That is, to automatically make application and platform resource adjustments at any level in the software stack in order to improve utilization and efficient.

Well, at least this is what I learned. When I talk about cloud I will be thinking about all three capabilities in management solutions. It doesn’t matter which use case or how big or little your use case is, if you can include all three of these basic cloud capabilities you will have a more efficient cloud. Of course the ‘got to be open’, and ‘work with key partners’ practices are always important too. But for cloud, I’m basing the best solution based on principle.

This Month: What 30,000 miles, 30 Customers and Five 5 Experts Taught Me

Having spent time with our financial services customers in Europe and New York, our partners and customers at ISC ’10 and some of the leading thinkers on cloud computing, I come home tired but firm in my belief that:

Cloud is happening as we predicted. Grids are evolving to private clouds. Private clouds are being trialed first in test/dev. Private clouds to hybrid clouds to public clouds seems to be the plan for the majority of IT organizations.

HPC clouds (mostly bursting from private clouds or grids) are being driven by peak off-loading. Custom built/configured environments are very much required.

VM adoption will remain limited in HPC environments due to performance overheads (yes 5% overhead matters….)


“HPC as a Service” or "Grid as a Service" (suggest we just call it private cloud) is very attractive and a growing need for ISVs that want to offer hosted environments and/or SaaS.

Migrating to commodity architectures in the enterprise is a top 1, 2 or 3 long-term objective. Many enterprises have “Google envy” in this regard. Few companies realize understand that HPC environments have already shown the way there in many regards.

Clouds cannot be bought. The cloud is still cloudy in terms of offerings and reference architectures. Only discussions about specific architectures that support specific applications are productive.

The Debate Rages On.........

Network World is now running a new feature called “Tech Debate,” where they’re pitting industry experts against each other to debate current hot topics. Not surprisingly, this week’s topic was “Cloud: Public or Private,” which pitted Dave Malcolm, Chief Technology Officer of Surgient, arguing for private clouds against Siki Giunta, Global VP of Cloud Computing & Cloud Services of CSC, arguing for public.

We’re in Dave’s corner in this debate, but don’t feel that this conversation is as black and white as the media would like it to be. Ideally, private clouds provide IT and end users ways to manage, allocate and share resources and workloads that maximize the entire infrastructure while keeping information secure within the firewall. In addition, deploying a private cloud using an independent management layer also helps avoid lock-in issues because it provides the means to help make physical and virtual--as well as legacy and new systems--interoperable.

For companies with small IT teams or less complicated compute needs, the public cloud can be an easy answer. But companies with large IT teams and more complex infrastructures, however, can spend a small fortune for enterprise apps, and have often built intricate infrastructures that can be a tangled web of legacy systems and architectures. Add in a global presence and high performance compute requirements and the desire to manage that infrastructure securely and safely is an increasingly important element in evaluating cloud technologies. Can you trust your apps to the public cloud? If you are at one of these big global brands that I mentioned before, I can bet your legal departments probably don’t want you to.

As for the scale argument that Giunta poses when it comes to public clouds, scale isn’t limited to the public cloud domain. Having come from a distributing computing background and managed cluster and grid systems for everything from large financial institutions running complex trading algorithms to heavy scientific research calculations, we’ve been scaling huge systems within firewalls for years. Public clouds may be able to add capacity for hybrid situations and may give SMBs what they need to start easily and scale fast, but enterprises will ultimately need to build out their own systems that meet their needs for security, provisioning and management.

Multiple global datacenters => one virtual datacenter? I think not…

I was just reading Nick Heath’s article Private Cloud: Turning corporate datacentres into a global virtual machine posted on about the potential of private cloud turning global datacenters into a massive, global VM, and it made me recall a somewhat-famous quote from Sun Microsystems on cloud from a couple of years back – to paraphrase: All enterprise IT will be run out of facilities hosted by a handful of service providers within the next decade. (I don’t remember the exact timeframe, but I do recall chuckling out loud when I read it).

Now this particular “global VM” article by Nick Heath wasn’t worthy of a chuckle, but--seriously? Who in their right mind would be entertaining a near-term strategy to string together all of their datacenter assets into a single shared resource pool, where apps and workloads could cross the pond(s) and back on a whim? Don’t get me wrong – this is a wonderful vision, with incredible economics of scale and efficiency behind it – but cloud computing is in its infancy and it still has significant political, data security, and financial reporting issues that need to be resolved before a monumental project such as this could be entertained.

When it comes to cloud computing, organizations of all sizes need to start small, either within a specific group (e.g. developers and testers of a BU) or class of apps (eg. stateless Java apps performing simple request/response). From there, they can get the kinks out, work the politics, figure out the chargeback models, etc. You could certainly implement this approach across multiple datacenters/geographies to support these teams/businesses now (some of our early adopters are doing just that), but not for the sake of having an über-global cloud.

You don’t need some science-fiction, futuristic cloud model to realize the economics of private cloud – a small bite will do.