IBM’s Systems and Technology Group wants to remain a systems business, not a hardware business nor a parts business. Therefore, IBM’s renewed emphasis is on systems software in its own right. IBM and Platform, the two complementary market leaders in technical computing systems and management software respectively, are coming together to provide overall market leadership and help customers to do more cost effective computing. In IBM speak, it’s smarter systems for smarter computing enabling a Smarter Planet. Not smarter people. Just normal people doing smarter things supported by smarter systems.
Why Combine Platform Computing with IBM?
IBM’s Systems and Technology Group wants to remain a systems business, not a hardware business nor a parts business. Therefore, IBM’s renewed emphasis is on systems software in its own right. IBM and Platform, the two complementary market leaders in technical computing systems and management software respectively, are coming together to provide overall market leadership and help customers to do more cost effective computing. In IBM speak, it’s smarter systems for smarter computing enabling a Smarter Planet. Not smarter people. Just normal people doing smarter things supported by smarter systems.
ISC cloud 2011
The European conference concerning the intersection between cloud computing and HPC has just finished, and it's very pleasing to report that this conference delivered considerable helpings of useful and exciting information on the topic.
Though cloud computing and HPC have tended to stay separated, the HPC community starting with the sc2010 conference, interest has been gaining primarily because access to additional temporary resource is very temping. However, other reasons for HPC users and architects to evaluate the viability of cloud include total cost of ownership comparisons, and startup businesses which may need temporary access to HPC but do not have the capital to purchase dedicated infrastructure.
Conclusions varied from presenter to presenter, tough some things were generally agreed upon:
- if using Amazon EC2, HPC applications must use the cluster compute instance to achieve comparable performance to local clusters.
- fine grained MPI applications are not well suited to the cloud simply because none of the major vendors offer infiniband or other low latency interconnect on the back end
- running long term in the cloud, even with favorable pricing agreements is much more expensive than running in local data centers, as long as those data centers already exist. (no one presented a cost analysis which included the datacenter build costs as an amortized cost of doing HPC.)
Another interesting trend was the different points of view depending on where the presenter came from. Generally, researchers from national labs had the point of view that cloud computing was not comparable to their in-house supercomputers and was not a viable alternative for them. Also, compared to the scale of their in-house systems, the resources available from Amazon or others were seen as quite limited.
Conversely, presenters from industry had the opposite point of view (notably a presentation given by Guillaume Alleon from EADS). Their much more modest requirements seemed to map much better into the available cloud infrastructure and the conclusion was positive for cloud being a capable alternative to in-house HPC.
Perhaps this is another aspect of the disparity between capability and capacity HPC computing. One maps well into the cloud, the other doesn't.
Overall it was a very useful two days. My only regret was not being able to present Platform's view on HPC cloud. See my next blog for some technologies to keep an eye on for overcoming cloud adoption barriers. Also, if anyone is interested in HPC and the cloud, this was the best and richest content event I've ever attended. Highly recommended.
Congratulations to Rick Parker – a Platform Customer, Partner and SuperNova Semifinalist!
![]() |
| A Protostar |
VMworld Signage - "Are You Cloud Intense?"
Show Your Cloud Intensity
I'm not sure we are philosophers, but we definitely know how to help customers build private clouds with IaaS management software!
Best of VMworld Award for Private Cloud Management
![]() |
| Best of VMworld 2011 Private Cloud |
This award follows other recent successes such as being named the #1 private cloud vendor by Forrester Research. Customers want a trusted vendor that has proven themselves in large-scale, enterprise environments, and one that has been around for a few years. Companies just don't want to trust their data centers to a startup with an uncertain future. There is just too much at-risk.
| Platform team celebrates the win at VMworld! |
Our private cloud / IaaS management solution, called Platform ISF, was cited for the following key differentiators:
- Cloud in a cloud in a cloud – ability to support multiple hierarchical organizational structures with nested virtual private clouds for business units, groups, projects, etc.
- Physical deployment – includes native capability to do bare metal physical provisioning all the way to complex multi-tier application structures
- Application-centric framework – IT services are captured as standardized applications or application components that are multi-tiered, virtual and physical, private and public cloud, etc.
- Automated provisioning of VMware – deep integration that maintains investment in VMware with automatic discovery, import (all or partial), synchronization, and simplification of provisioning VMware environments
- Multi-hypervisor with KVM and Xen – adds a management layer similar to vCenter on top of KVM / Xen to normalize capabilities across hypervisors
- Integration with P2V (physical to virtual) – support for multiple provisioning tools plus a Platform native option to support the ongoing transition of more applications and workloads to virtual environments
- Managing resource pools on DRS clusters – adding control over resource pools on DRS clusters with end user self-service management aligned to hierarchical account management
- Policy management - depth and breadth of policy management capabilities for both initial placement and operational service level agreements (SLAs) such as flexing up and down
- One solution – Platform ISF is an integrated set of IaaS management capabilities that span self-service, billing, automation, etc. versus requiring multiple solutions from different vendors that require stitching together
Top 5 VMworld 2011 Takeaways for Private Cloud
From attending the keynotes and speaking with attendees, I had 5 key takeaways from the event:
2. VMware licensing has upset many
3. Multi-hypervisor is real
4. The rise of application-centric thinking
5. Physical is part of cloud
Forrester Evaluates Private Cloud Vendors – Platform Tops Media Scoring
Complimentary Full Copy of the Report
The report is significant as the first ‘apples-to-apples” comparison of vendors using customer references, a compulsory 30 minute demonstration and written responses. Only 15 vendors were able to meet this initial hurdle, suggesting a number of vendors where marketing far outstrips reality. Vendors were scored 0-4 across 10 criteria. Forrester did not add up the scores but the media quickly did the sub-analysis and published a ranking table. The result: Platform Computing received the most points.
Source: SYS-CON Media
The # 2 vendor was VMware and we congratulate them on their effort. Given that one of the key report findings is to “prepare for islands of hypervisors” – meaning an increase in the need to support other hypervisors such as KVM and Xen -- companies should pause to reconsider locking into a single hypervisor vendor. Cloud is much more than VM. Also, be careful about pricing models that penalize your ability to utilize your cloud environment more efficiently (i.e., per VM versus per server).
Read the Platform Computing press release
Another interesting point was discussed by the report’s author, James Staten, in his blog. Mr. Staten cautions readers to evaluate vendors on their specific requirements. We totally agree.
First, in the report, an attempt is made to catalog vendors based upon their origins, e.g., enterprise systems management vendors versus pure-play cloud solutions, etc. Our belief is that regardless of origins, customers should choose a vendor with a comprehensive technology footprint, solid customer base, strong financials, and global support system. Claiming you are # 1 among pure-play or grid vendors is meaningless and an artificial distinction. At the end of the day, one wants the best solution to solving a set of problems. Period.
Second, Mr. Staten recommends looking differently at either software-only versus ‘physical compute and storage included’ vendors. If you take those points away from HP, IBM, Dell, Microsoft, and BMC, the Platform lead just grows. And if you want an appliance, Platform has partners that offer a complete pre-integrated solution today.
Finally, the report recommends a strategy of trying before you buy. Since a private cloud project involves both technology and process change, we recommend running a proof-of-concept (POC) to gain buy-in from end user communities and different business units. A POC can also help validate assumptions against your business case for justifying the value of a private cloud.
Resources
- Learn more about Platform's private cloud solution here
- Download the Platform ISF product here
- Schedule a discovery session with one of our cloud consultants by contacting us at privatecloud@platform.com.
Are there any criteria that you would have added to the evaluation?
Private Cloud at State Street
There is nothing better than a real world customer case study to explain the value of deploying a product. Cut through the marketing and explain:
- The underlying problem / pain points
- The vendor selection process
- Key decisions and lesson learned
- Actual value being achieved
At the recent Wall Street & Technology Capital Markets Cloud Symposium, the Chief Infrastructure Architect from State Street, Kevin Sullivan, discussed how cloud – specifically private cloud – is being rolled out across 20+ projects.
For more details about Mr. Sullivan's presentation, there is a great article in Advanced Trading magazine by Phil Albinus. Mr. Albinus provides details about the decision making process, key vendors, and next steps. To learn more about Platform Computing's cloud solution, please visit our private cloud solution area.
Some additional metrics that were discussed include:
- Started in 2009
- Went from 400 to 150 use cases
- Operated a 6 month proof-of-concept (POC)
- Down-selected from 150 technology partners to a ‘handful’
- Provisioning has been reduced from 8 weeks to hours
- Unique Active-Active two data center configuration for 100% app high availability
One of the four key elements to the State Street cloud environment is the ‘cloud controller’. In their environment, the cloud controller manages self-service, configurations, provisioning, and automation of the application construction within the infrastructure. A key benefit for the developers was the ability to get access to more self-service capacity on-demand much faster. This easily outweighed any cultural concerns regarding not being able to specify to the n-th degree an IT environment request. If needed, IT could still request ‘custom’ but it would take longer.
In addition, Mr. Sullivan highlighted several key design principles driving the design, building, and management of the cloud:
- Simplification – creating of standards, consolidation, and self-service
- Automation – for deployment, metrics, elasticity, and monitoring
- Leverage – commodity hardware and software stack with a focus on reuse of the platform, services, and data
The benefits and successes of private cloud at State Street are becoming increasingly known as their CIO Christopher Perretta is fond of discussing the topic. Some additional resources to learn more about the State Street cloud project are:
What do you think can be learned from State Street’s successes?Not to burst your bubble…
Probably everyone who has ever been in an HPC environment as a user has run into resource constraints. Limitations like hardware, licenses, memory, or maybe even GPUs on hybrid compute engines rank at the high end of the limitations most users face are facing today. The problem comes when you run into these limitations in the middle of running a job. What do you do then? “Hitting the stops,” so to speak, will often cause another procurement cycle and consume lots of resources in terms of analysis, internal meetings, planning, as well as the purchase and deployment phases before any additional real work can get completed.
Cloud computing, or in this case, cloud bursting offers an approach to mitigate the process and limitations that most HPC consumer corporations go through today. Certainly, using resources outside the firewall can have its own challenges for corporate users, but those aren’t the focus of this blog.
Assume for a moment that security, licensing, provisioning latency and data access are not a problem. Of course, they’re all major issues that need to be addressed to make a cloud solution usable, but bear with me. There are still some important questions that need to be answered:
- What are the appropriate conditions to start up and provision cloud infrastructure?
- What jobs should be sent to the cloud once that infrastructure is provisioned and which should stay local and wait?
This second question is the focus here. Often in science, the hardest part of solving a tough problem is stating the question properly. In this case, the question is nicely represented by the below inequality where each entity represents a factor of elapsed time. For cloud computing to be advantageous from a performance perspective,
(Data upload to cloud) + (Cloud Pend time) + (Cloud Run time) + (Data download from cloud) < (Local Pend time) + (Local Run time)
Such a statement allows us to draw a few conclusions about the conditions for when cloud bursting is advantageous for the HPC user:
- When local job pend time estimates for a job get very large
- When local elapsed run time is large -- A corollary to this condition is that if the job can be parallelized, but there are insufficient resources locally to run the job quickly, then cloud bursting the job may return results to the user sooner than allowing the job to run on insufficient resources locally.
- When the job’s data transfer requirements into and out of the cloud are small
Additionally to those conditions, we start to see where several of the real challenges are for a scheduler to make the right decision about which jobs get sent to the cloud and which don’t. For instance, most schedulers today do not consider the data volume associated with a job. But, in a cloud scenario, the data transfer times associated could be 2-50x the runtime for a job and not only dependent upon the file size, but the available transfer bandwidth. Schedulers will need to evolve on several levels to tackle this challenge:
- Allow users to indicate the files (both input and output) required for each job.
- Estimate pending and run time for disparate infrastructures
- Estimate the run time for jobs which run parallel
Supercomputing goes commercial in China
Just having come back from the HPC China Conference in Beijing two weeks ago where TianHe 1A was announced, I am now packing for SC’10 in New Orleans.
For those of you unfamiliar with TianHe 1A, it is currently the world’s fastest supercomputer capable of computing 2.507 petaflops, or 2,507 trillion calculations per second. TianHe 1A, together with Nebulae (the word’s second fastest supercomputer according to the July 2010 TOP500) will be hosted in government funded supercomputing centers in China. However, unlike many other countries where the top systems are almost exclusively running code for research, supercomputing centers in China will be running like commercial companies to sell computing capacity to industries and research firms. Their model center, the Shanghai Supercomputing Center (#24 on July 2010 TOP500) has about 300 customers and keeps full capacity operations 24x7. As the result, those big systems will run more commercial code than the systems in national labs. Such commercialized operations will demand high software quality and strong local support.
The recently launched Platform HPC matches needs like these very well. With a complete management solution with a leading MPI implementation, Platform HPC brings performance and productivity to production environments. It provides all the necessary components to manage large systems including user web portal, application integrations, resource allocation and job scheduling, cluster management and monitoring, and job usage accounting. Most importantly, it is supported by a single vendor to dramatically decrease troubleshooting time.
I’m looking forward to seeing how such HPC cloud model on top systems shakes the future of HPC community.
SIBOS off and running
Tying all of these themes together is the role of technology in helping organizations overcome the obstacles they face. After all, data proliferation, and the need to better manage it, is at the root of many challenges being faced by financial institutions today. For example, our recent survey found that 66% of buy-side firms and 56% of sell-side firms are grappling with siloed data sources.
Looking at some of our recent work, it is interesting to see how financial services firms are trying to tackle these areas. SepSvenska Handelsbanken is using high-performance computing to run core applications for trading and risk management. New regulations around risk in Sweden are leading to more risk scenarios which need to be run so more compute power allows the bank to plan ahead much more easily. Other financial institutions including Citigroup are also running similar projects.
As the event continues through the week, it will be interesting to see the different ways financial organizations are tackling these challenges. Hopefully, we will finally begin to see an end to the stormy crisis clouds and a sunny future shining though.
By Jeff Hong, CFA
Director of Financial Services Industry Marketing
Platform Computing
Innovation in the cloud
Cloud really is sparking innovation and the promise of innovation in the cloud is a driver for adoption. This year we conducted a study around the uptake of cloud technology and found that while improving efficiency was the main motivator in 2009 (41%), the 2010 survey reveals that drivers for deploying private cloud included experimenting with cloud (19%).
Private cloud is a good home for test and development because as it offers a flexible, cost effective and safe environment to operate in. We recently wrote about a test and development cloud, this type of private cloud deployment provides a self-service test and development infrastructure.
Not only does cloud offer a platform for innovation, but as Ballmer states it also frees up resources by allowing executives time to innovate.
In order to make the most of the cloud it’s essential to approach a migration to cloud carefully. A phrase we like to use is “A cloud is built not bought” and as with all good constructions; comprehensive plans, firm foundations and the right materials are the essential building blocks.
Private clouds are likely to continue to outpace public cloud models and I expect the increase in private cloud adoption to be matched by an increase in innovation.













