Linux Interacting with Windows HPC Server

There are many interesting technology showcases at SC’11 this week. One of the Birds of a Feather sessions this week will discuss a solution implemented at Virginia Tech Transportation Institute (VTTI) that mixes Linux with Windows HPC Server in a cluster for processing large amount of data.

Without a proper tool or a lot of practice, getting Linux and Windows to work together seamlessly to provide a unified interface for end users is a very challenging task. Having both systems coexist in an HPC cluster environment adds an order of magnitude of additional complexity compared to an already complex enough HPC Linux cluster.

This is because Windows and Linux “speak very different languages” in many areas such as user account management, file path and directory structure, cluster management practice, application integrations etc.

The good news is the Platform Computing engineering team did some heavy lifting in product development for this project. Platform HPC integrates with the full software stack required to run an HPC Linux cluster. Its major differentiator compared to alternative solutions is that Platform HPC is application aware. When adding Windows HPC Server into the HPC cluster, the solution delivered by Platform HPC ensures it provides a unified user experiences across Linux and Windows, and hides the difference and complexity between the two OSs.

Platform HPC team has developed a step by step guide for implementing an end-to-end solution with provisioning both Windows and Linux, unified user authentication, unified job scheduling, automated workload driven OS switch, application integrations, and unified end-user interfaces.

This solution significantly reduces the complexity of a mixed Windows and Linux cluster, so users can focus on their applications and their productive work, as opposed to managing the complexity of the mixed Windows and Linux cluster.

The Release of Platform HPC 3.0.1 Dell Edition

Last week, Platform HPC 3.0.1 Dell Edition was released. One of the significant features of this release is the support of high availability for management nodes on the latest version of Red Hat Enterprise Linux 6.1. Why is support for management node HA special when many cluster management tools support it already?

Well, Platform HPC is not just a cluster management solution. It is an end-to-end cluster management and productivity tool. When handling management node failover, Platform HPC not only needs to ensure all the cluster management functionalities can fail over, it also has to ensure other functionalities fail over at the same time so that the end user and administrator won’t see the difference before and after the failover. The functionalities that failover handles include but are not limited to:

1. Provisioning
2. Monitoring & alerting
3. Workload management service
4. Web portal for users and administrators

In a heavy production environment, the failover function of the user web portal is far more critical than the failover of the cluster management functionality. This ensures users have non-stop access to the cluster through the web portal even if the primary management node running the web server is down.

Other capabilities included in Platform HPC 3.0.1 Dell Edition include:

1. Dell hardware specific setup for management through IPMI
2. One-To-Many BIOS configuration via the idrac-bios-tool
3. Dell OpenManage integration
4. Mellanox OFED 1.5.3-1 kit
5. CUDA 4 kit
6. Management software integration provided by QLogic and Terascale
7. Dell fully automated factory install

With the complete and integration software components, the cluster solution that Platform Computing delivers together with Dell has gone a long way since the release of Open Cluster Stack in 2007.

Is Platform Cluster Manager just commercial support of Kusu?

Many people think Platform Cluster Manager is just a commercial version of the open source software, Kusu. When Kusu was born a few years ago, that actually was the case. Since then, Platform Cluster Manager has evolved significantly, and by comparing the latest release of Platform Cluster Manager with Kusu today, one will find there are many differences now.

Kusu is open source cluster provisioning and management software developed by Platform Computing. The commercially supported open source version was initially called Open Cluster Stack (OCS), and that name was later changed to Platform Cluster Manager. In version 2.0, which was released in early 2010, we started to package proprietary code into Platform Cluster Manager for a better interface. In subsequent releases since then, more proprietary code has been added to Platform Cluster Manager. In the latest version 3.0, the original Kusu code is now just small part of the Platform Cluster Manager product. Installation, a graphical web interface, and the monitoring system are all proprietary code that have been added into the product. Although the Kusu code has gained enhancements for functionality and reliability release by release, Platform Cluster Manager 3 just uses Kusu to power its provisioning engine. The rest of the product’s functionality is not open source any more.

Today, Platform Cluster Manager as a licensed product is sold by many Platform Computing channel partners. It contains the following functional modules:

1. High quality and flexible open source provisioning engine developed by Platform Computing.
2. Web interface framework shared by most Platform Computing products
3. Web interface for cluster management
4. Monitoring framework based on reliable and scalable agent technology used in Platform LSF
5. Installer that supports un-attended or factory install

Some key new features added to Platform Cluster Manager 3 include:

1. An intuitive web interface
2. Management node high availability
3. Support of NIC bonding
4. Monitoring and alerting
5. More flexible network interface types
6. Enhanced kit building process

Platform Cluster Manager is also included in Platform HPC as the cluster management tool. It is used for deploying and managing the additional software modules in Platform HPC. With one package and one web interface, customers can easily expand their management functionalities from Platform Cluster Manager to Platform HPC by just adding Platform HPC licenses.

With the addition of these new features over the last two years, Platform Cluster Manager has truly evolved from a community supported open source package to a commercial grade product.