Azure Sentinel is a cloud SIEM and SOAR. It is therefore used for the supervision of a customer environment, from which it...
Running computing tasks on Microsoft HPC – Job ManagerLukas Beran
In today’s article in a series of HPC I will focus on starting and managing computational tasks on Microsoft HPC cluster using HPC Job Manager. Article takes the form of straightforward instructions.
Tasks that should run parallel to the computing nodes, should be run with the association on computing nodes – not cores. Within the computing nodes are then used threads to run on all cores of the node. MPI is used to ensure communication of the app running on multiple nodes. So each app has to be run as a parameter of mpiexec, which cares about parallelization for the cluster.
The path, which is designed for data of computing tasks, including the actual executable file must be available for all computing nodes. It must therefore be either a network path specified using UNC.
Creating a new computing tasks
In the right actions menu, select New Job. In a new window, type in the box Job name name of the job. In Job template, leave the Default or select a previously created template. Project leave empty and Priority remains Normal.
You can set up sending notifications upon job start and / or end of the job using the check-boxes next to the Send a notification. When you check at least one option, box will be displayed with the default e-mail address retrieved from the logged account. This address can be change.
At the bottom of the window, select the type of Job Node because of the parallelization using MPI and you can optionally set the minimum and maximum number of nodes allocated for computational task. If you leave Auto, computing cluster itself determines the number of nodes assigned for the job, given the overall cluster utilization.
In the left pane, switch to Edit Tasks tab, and then in the right pane, select Add. You can add task name to the box Task name. To the box Command line, enter the name of the executable file of the calculation process, including any arguments. If you are running MPI job, enter mpiexec first, which is an application that will allow the process to run on multiple computing nodes and beyond mpiexec specify the name of the executable file you want to run. If the executable file of the calculation process is not directly in the working directory, set in the box below Working directory, and type the relative path to the working directory, or an absolute path. The path to the executable must be available to all computing nodes.
As a final, you can specify a minimum and maximum number of computing resources allocated to that process. The resource type is determined by the type chosen for the entire job in the previous section on the Job Details tab.
Resource Selection tab, you can manually choose the computing nodes on which a given task can run. If you do not select any computing nodes, the task will run on all nodes that are available.
By clicking Submit button, you add a new computing role. If there are available required computational resources to run a job, the job runs immediately. When computing resources are not available, the job will be queued and when you release the computing resources, the job will be automatically triggered.
View the status of computing tasks
Tasks can be filtered by their state or the owner using the left menu in the Job Manager.
Basic job status can be seen in the list of all tasks. To view more details about the task, double click on the task to open the details.
On the tab Job Progress, you can see the status of open tasks and jobs report detailing the current status.
Job Details tab shows the initial setup tasks.
On the tab View Tasks you can see details of all jobs and details of all running processes. Resource Selection tab allows you to view the selected resources to the task and tab Allocated Nodes shows the computational nodes assigned to the task.
Edit computing tasks
To edit the task, select the task in the list of tasks and in the right menu, select Modify Job.
Cancel computing tasks
If you want to cancel any computing task, you can do so using the Cancel Job in menu actions.