This article is primarily targeted at Czech state institutions and is based on recommendations of the Czech authority. However,...
Creating Microsoft HPC computing clusterLukas Beran
On the internet, there is no much information about Microsoft HPC cluster creation and configuration. Therefore, in the following article, I will describe the basic setup how to create Microsoft HPC computing cluster. The following guidance is based on the Microsoft HPC Pack 2008 R2 with Windows Server 2008 R2 HPC Edition as the head node. There must be an Active Directory, to which must be added all nodes of the cluster.
- Main node: Windows Server 2008 R2 Standard, Enterprise, Datacenter nebo HPC Edition
- Computing node: 64bit varianty Windows Server 2008 nebo Windows Server 2008 R2 Standard, Enterprise, Datacenter nebo HPC Edition
- Broker node: Windows Server 2008 R2 Standard, Enterprise, Datacenter nebo HPC Edition Workstation nebo Cycle
- Harvesting node: Windows 7 Professional nebo Enterprise, 64bit verze Windows Server 2008 R2 Standard, Enterprise, Datacenter
- CPU: 64bit architecture except Workstation nodes, which can be 32bit
- RAM: 512 MB
- Hard drive: At least 50 GB of available disk space
- Network: At least one network card
For high availability nodes is requirement of Enterprise edition of main and broker nodes.
Installing the main node
The first is to install the main node based on Windows Server 2008 R2 HPC Edition. This is a slightly modified version of Windows Server 2008 R2 for use in a computing cluster, which is extended of NetworkDirect RDMA, cluster management tools, SOA task manager, MPI library and the High Performance Computing Basic role. After installing and updating follows the inclusion in Active Directory. The next step is to install Microsoft HPC Pack 2008 R2 on the main node. Under license, it is possible to choose between the Express and Enterprise editions. Express Edition is available for free download, Enterprise Edition is available as a paid package Microsoft HPC Pack 2008 R2 Suite (VL or OEM) or as a standalone product (VL, OEM or MSDN). In addition, Enterprise Edition provides the ability to run Microsoft Excel tasks and ability to add workstations as a cluster node.
The first step of the installation of HPC Pack on the main node is to select the edition you want to install. The next step is to choose what we want to do with HPC Pack – Create a new HPC cluster by creating a head node, thus creating a new HPC cluster by creating a main node. Other possibilities are to connect to an existing cluster computing by creating a new node, connecting to an existing cluster, creating a WCF broker node, connecting to an existing cluster and installing only management cluster tools (HPC Cluster Manager, HPC Job Manager, and HPC PowerShell). We choose the first option.
The next step is to select the location of database. Microsoft’s recommendation is to use a local database on the primary node when the cluster consists of fewer than 256 nodes, or use a separate database server. If you choose the first option, you can install HPC Pack also with installing SQL Server 2008 and create a database instance.
It remains to complete the last settings, which are creating a cluster name and select the location of the database and other files of HPC Pack.
After installing HPC Pack is important to verify that you have installed the latest version of Service Pack. Currently it is version 4. Testing can be done for example in HPC Manager Help menu -> About wherein the second number indicates the version of the SP version (3.4.4169.0).
Cluster Configuration in HPC Manager
When configuring the network, it is possible to choose from five different topologies:
- Compute node isolated on a private network, which is used to separate private network dedicated only for the computing nodes. Computing nodes are not part of any other network and are connected to a dedicated network interface to the server. For this configuration, it is necessary to have at least two network interfaces on the primary node.
- All nodes on Enterprise and Private Networks, which is used in the case where all nodes are connected to a private and non-private network, the private network is used only for the cluster. This configuration requires at least two network interfaces on the primary node and on the compute nodes.
- Compute nodes isolated on private and application networks uses two cluster networks when one network is reserved for applications’ communication (calculations) and the second for cluster management. Configuration requires at least two network interfaces on the compute nodes and at least three network interfaces on the primary node.
- All nodes on enterprise, private, and application networks is based on three networks. One network is used for applications’ communication (calculations), the second for cluster management and the third is a common corporate network. In this configuration are required at least three network interfaces on the compute nodes and the main node.
- All nodes only on an enterprise network is the configuration that requires all nodes connected to a corporate network, so just one network interface on the primary node and on the compute nodes is needed.
The next step is to select the network interface that will be used for computing cluster. The last step is configuring the network firewall settings. There is a choice of two options. The first recommendation is automatic firewall configuration for computing nodes and primary node for needs of the cluster, the other option is to leave the firewall settings in its current state. The following is only an overview of the settings and confirmation.
Provide administrative credentials
In another part of the initial preparation of the cluster is necessary to enter domain account that has administrator privileges to install software and manage all stations.
Configuration names compute nodes
For easier management of all computing nodes set names that are automatically generated for all newly added compute nodes. Naming template has the shape of %number%, for example numbering template %00% will be from numbering 00 to 99.
Creating templates for nodes
Templates are used to simplify adding, configuration and management of nodes in the cluster. If any template is assigned, these nodes are automatically applied to the rules defined in this template.
Using templates can be defined, for example, the availability of computing nodes, manage updates using Windows Update or WSUS, add nodes to an Active Directory domain, manage shared drives that are connected to the computing nodes and more.
There are four basic types of templates:
- Compute node template is used for computational nodes, which are permanently available. This type of nodes can receive and execute computing tasks and accepts requests from SOA broker nodes.
- Broker node template now accepts requests from Service-Oriented Architecture (SOA) clients, distributes these requests to the computing nodes of the cluster, collects the answers and sends them back to the clients. Broker node must be connected to the corporate network.
- Workstation node template is used for workstations that will be used for calculations. This template is the only one where we can define availability of computing nodes. For Workstation node template you can use the schedule to define when the nodes are in this template available and when not. After installing SP1 on HPC Pack you can set the status change based on user activity detection. If the activity detection is activated, all computing tasks are running with priority BelowNormal or less.
- Windows Azure node template allows connection to Windows Azure and management of Windows Azure compute nodes.
Installing Microsoft HPC Pack computational nodes
On all computing nodes must be installed Microsoft HPC Pack. Installation is the same as the main node, and in the second-step of installation, select the option to connect to an existing cluster, and the next step select a previously defined cluster name.
Adding compute nodes to the cluster
Before the first use of cluster must all newly installed nodes be added to the HPC Cluster Manager and publish these nodes for computation. Newly installed nodes in HPC Cluster Manager are categorized as Unapproved and needs to allocate them to the template. Template can be assigned by selecting all nodes in the Unapproved category and choosing Assign Node Template, where you can select a previously created template. Applying a template will move all nodes within the category OK.
Adding authorized users and groups
Users who need to enter tasks must be in the Users group in HPC Cluster Manager. It is also necessary to add users and authorized for remote desktop to be able to connect to the cluster and manage their tasks.
Configuration Task Scheduler
In the Options menu -> Job Scheduler Configuration can be set as the scheduler to treat computing tasks. Here you can set, for example, allocation of task level preemption, e-mail notifications for tasks, error trapping, rules for keeping jobs in history.
For sending e-mail notifications it is important to mention that notifications work correctly only if the recipient has a mail box at Microsoft Exchange. In other cases, notifications are illegible, contain only a blank message.