Share via

Windows HPC Server 2008 R2: Developer Resources

  • Article

Updated: December 2014

Applies To: Windows HPC Server 2008 R2

This topic provides information about developer resources specifically for Windows® HPC Server 2008 R2. For updated developer resources for the current version of , see Microsoft High Performance Computing for Developers.

In this topic:

  • The HPC SDKs and code samples (New SDK for SP2)

  • Visual Studio 2010 and HPC Server 2008 R2

  • The HPC Debugger Add-Ins for Visual Studio 2008 and Visual Studio 2010

  • Service-oriented architecture client development

  • Developing Excel workbooks that can run on a cluster

  • Diagnostics extensibility

  • Reporting Extensibility

  • Scheduler Integration

  • NetworkDirect service provider development

  • Resources for Azure integration

  • Resources for setting up a development cluster

The HPC SDKs and code samples

You can download the tools to write parallel applications for the Windows HPC Server 2008 R2 platform from the HPC Pack 2008 R2 Software Development Kit. Updated versions of the SDKs and code samples, which include new APIs and capabilities introduced in the Service Packs, are available here:

The following code samples are available:

The <HPCVersion>.SampleCode.zip (for example, HPC2008R2SP1.SampleCode.zip) file includes code samples and files as described in the following table. The 08R2, +SP1, and +SP2 columns indicate the availability of the samples in each code sample release. The icon indicates the sample is available, indicates it was updated, and indicates it was removed.

Folder File or sub-folder Description 08R2 +SP1 +SP2

Diagnostics

DriverVersionCheck

Verifies that the network driver version matches a specified value.

MPIPiCalculationBatch

Verifies that the cluster can run an MPI job by running a sample MPI application (uses a batch file in the PreStep to generate the MPI task for the RunStep).

MPIPiCalculationCS

Verifies that the cluster can run an MPI job by running a sample MPI application (uses a C# application in the PreStep to generate the MPI task for the RunStep).

PathAccessCheck

Verifies that compute nodes can access a specified path.

PingHistogram

Measures ping response times from the selected compute nodes and displays results as a histogram.

Diagnostics README.docx

Provides information and installation steps for the samples.

Excel

Exceludf

Includes a Visual Studio 2008 solution file that you can compile to generate a cluster-safe UDF and an Excel workbook that calls the UDF.

Excelworkbook

Includes several code samples and workbooks that demonstrate various workbook offloading methods including how to use the default ExcelService, create a custom SOA service, use the ExcelClient API from C#, use VBA code with the ExcelClient COM interface, and create custom SOA clients (durable or interactive).

HPCExcel.pdf

Provides a developer overview of the HPC Services for Excel, information about how to compile and run the samples, and further development options.

Scheduler

ActivationFilter

An example of a cluster-wide (executable) activation filter that checks for license availability against a FlexLM license file. Usage instructions are available online.

This sample is not included in the SP1 code samples.

In the SP2 code samples, this sample is available under the “Filters\exe” subfolder. The sample was also updated to reference the XML schema for R2 when parsing the job file (https://schemas.microsoft.com/HPCS2008R2/scheduler/).

DynamicNodeGroups

Submits jobs to demonstrate how dynamic node groups work on the cluster.

ExpandedPriorities

Demonstrates how to set the ExpandedPriority property for jobs.

Filters

Sample activation and submission filters that can be added to the HPC Job Scheduler Service to implement customized job scheduling policies, such as checking for availability of software licenses. Custom filters can be defined at the cluster-wide level (will run on all jobs) or at the job template level (will run only on jobs submitted to a particular job template).

The filters samples include examples of both types of filter implementations: the job-specific filter that must be defined as a DLL (and will run in the same process as the HPC Job Scheduler Service), and the cluster-wide filters that are defined as executables (which run in a separate process). Samples include:

  • DLL

    • Activation FlexLM: A sample activation filter that integrates with FlexLM.

    • Activation HoldUntil: A sample activation filter that causes the job scheduler to hold a job for a specified period of time before re-evaluating the job for activation.

    • ComboDiagLogging: A sample filter that serves as both an activation and submission filter and includes sample logging. This filter is useful for diagnostic purposes as well as demonstrating shared state between a submission filter and an activation filter.

    • Submission JobSize: A sample submission filter that modifies a job based on its size.

  • exe

    • Activation FlexLM: A sample activation filter that integrates with FlexLM.

    • Activation HoldUntil: A sample activation filter that causes the job scheduler to hold a job for a specified period of time before re-evaluating the job for activation.

    • Submission JobSize: A sample submission filter that modifies a job based on its size.

-

-

HoldUntilActivationFilter

An example of a cluster-wide (executable) activation filter that causes the job scheduler to hold a job for a specified period of time before re-evaluating the job for activation.

In the SP2 code samples, this sample is available under the “Filters\exe” subfolder. The sample was also updated to reference the XML schema for R2 when parsing the job file (https://schemas.microsoft.com/HPCS2008R2/scheduler/).

HPCSchedulerBasics

Creates a job, adds tasks, submits the job, and uses a callback to check the job state.

JobFinish

Demonstrates how to stop a running job and have the final state flagged as Finished rather than Canceled.

JobProgress

Demonstrates how to modify the job progress and progress message.

ReconnectEvents

Submits many jobs to the cluster, and monitors connection to the job scheduler by using an event handler. Stops submitting new jobs on disconnect, and resumes on reconnect.

REST

Includes sample client applications for accessing the HPC Job Scheduler Service through the HpcWebService. The HpcWebService is an HTTP web service that is based on the representational state transfer (REST) model. This enables you to submit jobs across platforms or across doMayns.

Samples are provided in C# and in Python, and demonstrate how to connect to the web service, parse responses, and create, submit, and cancel jobs.

-

-

SOA

AsianOptions

Runs an Asian Options pricing model from an Excel client, and uses SOA to offload calculations to the cluster.

CommonData

Sample SOA client and service that stage and access a set of common data that is required by all calculation requests within one or more sessions. These samples use APIs that were added in SP2, including a new type of client called a DataClient. The data client includes methods to upload data to the cluster (to the runtime user data shared folder) and to read and write data. If you want the data to be available to other cluster users, you can specify the list of users when you call DataClient.Create().

-

-

CustomBroker

Sample custom broker that you can build and deploy to a development cluster. See readme.txt for deployment steps.

-

-

DataCaching

Code samples and accompanying documentation for using Windows Server AppFabric caching as in-memory cache to support SOA applications that use large amounts of common data.

This sample is available as a separate download with the SP1 SDK (CommonDataCaching.Sample.zip).

-

-

GenericService

Sample generic SOA service and client that accesses the service by creating a durable session (V3ClientSample) and a volatile session (V2ClientSample).

-

-

HelloWorldR2

Includes several client samples that demonstrate how to use the Session APIs to send requests, get responses, register a callback, create file and recollect or multi-batch clients, cancel requests, and define exit behavior.

SP1 added Flush API samples.

Java

Demonstrates how to use a Java client to start a session on the HPC cluster. For more information, see the HPC Team blog post Interoping Windows HPC SOA from Java (or other non-.NET environment).

SP1 includes bug fixes to the sample code.

REST

Includes sample client and broker to enable REST (WebHttpBinding).

SapphireHost

Includes a service host created with C++/Sapphire (Windows Web Services). This allows you to run native C/C++ code as a service without using a C# wrapper. It is also as an example of how to create a custom service host.

TextEncoding

Demonstrates how to replace MessageEncodingBinding in net.tcp with custom TextMessageEncodingBinding to provide text encoding.

Additional code samples on the SDK download page:

  • CommonDataCaching.Sample.zip (available as of the SP1 SDK) includes sample SOA solutions that use AppFabric caching for common data that can be shared among compute nodes. Documentation is included in the zip file, or you can download just the white paper: Use AppFabric Caching for Common Data in Windows HPC Server 2008 R2 (white paper only).

  • Reporting.Sample.zip (available as of the SP1 SDK) includes sample code that demonstrates how to build web-based reports using Microsoft Silverlight. See also Reporting Extensibility in this topic.

Other code samples resources:

  • MPI: For a list of resources and code samples for MPI application development, see MPI Application Development for Windows HPC Server.

  • Excel: For information about resources and code samples for cluster-ready Excel see Developing Excel workbooks that can run on a cluster.

  • Azure: For information about code samples of applications (SOA, MPI, Excel, and parametric sweep) that can burst to Azure from an HPC cluster see Resources for Azure integration.

Visual Studio 2010 and HPC Server 2008 R2

This section includes information about developing applications in Visual Studio 2010 for a Windows HPC Server 2008 R2 cluster.

  • HPC Server 2008 R2 has the .NET 3.5 framework and the Microsoft Visual C++ 2008 runtime (VC++ 9.0) installed by default. If you are compiling in Visual Studio 2010, we recommend that you select .NET 3.5 when compiling your applications for the cluster (the default framework in Visual Studio 2010 is .NET 4.0). If you are compiling managed C++ code, you will also have to install the Microsoft Visual C++ 2008 runtime (VC++ 9.0) on your development computer.

    If you compile your applications with .NET 4.0, then the cluster administrator must install.NET 4.0 on the cluster. The service host on the compute nodes loads the framework and assemblies, so the administrator must also create an hpcservicehost.exe.config file (for 64-bit services) and optionally an hpcservicehost32.exe.config file (for 32-bit services) to make the service host load .NET 4.0. The file must be copied to %CCP_HOME%bin on the compute nodes. The file should contain the following XML:

    <?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
  </startup>
</configuration>

    For a .NET 3.5 application to run on .NET 4.0, the application must include a supported runtime element in the application configuration file. For example: <configuration> <startup> <supportedRuntime version="v4.0"/> </startup> </configuration>

    Note
    .NET 4.0 is not officially supported on an HPC Server 2008 R2 cluster and has not been thoroughly tested.

  • If you use Visual Studio 2010 to compile your parallel applications (managed C++) and target the Microsoft Visual C++ 2010 runtime (VC++ 10.0), you must include the C runtime assemblies (msvcrt) when you deploy your application to a Windows HPC Server 2008 R2 cluster. You can copy the CRT files to the same shared folder that includes your application. By default, the CRT files are located in C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist. If the correct CRT files are missing, you may see error code -1073741515.

The HPC Debugger Add-Ins for Visual Studio 2008 and Visual Studio 2010

The HPC Debugger includes add-ins to Visual Studio® 2008 SP1 and Visual Studio 2010 that help you debug Message Passing Interface (MPI) and service-oriented architecture (SOA) applications that are running on a Windows HPC 2008 or Window HPC 2008 R2 cluster. The add-ins extend the Visual Studio remote debugging functionality to simplify the process of debugging applications on a cluster.

The HPC Debuggers do the following:

  • Deploy your application or service files to the cluster nodes

  • Deploy any additional files that you specify

  • Launch the remote debugger on each node that is used for the debugging session

  • Clean up after the debugging session ends

Download the HPC Debuggers at HPC Debugger Add-on for MPI and Cluster-SOA.

For an overview of the HPC Debuggers, see Microsoft HPC Debugger Add-Ins for Visual Studio 2008 and Visual Studio 2010.

Service-oriented architecture client development

The HPC Class library includes the APIs that a client application can use to create sessions on the cluster. The Windows HPC 2008 R2 SDK includes support for creating reliable sessions and client interaction models such as file and recollect, batch, and shared session clients. These scenarios are supported by additions to the Microsoft.Hpc.Scheduler.Session namespace, including the BrokerClient class and the DurableSession class. Many of the new supported models do not require changes the service code, just to the client.

DurableSession

DurableSession supports reliable computations. When a DurableSession is created, request and response messages are persisted by the broker node so that in event of client or network failure, requests and responses are not lost. Clients can attach to a session with the same session ID to retrieve responses using the AttachSession() method.

The AttachSession() method is a non-blocking call that retrieves responses, but does not cause the broker node to release the responses (so they can be retrieved again in the case of failure). This allows you to attach to a crashed session to retrieve responses, or to restart a file and recollect session to retrieve responses.

Note
The C# Cluster-SOA Debugger for Windows HPC 2008 R2 includes a project template for a durable session client. For more information, see Walkthrough: Debugging a Durable Session Client on a Windows HPC 2008 R2 Cluster.
Note
Resilience in the case of broker node failure additionally requires a failover broker node configuration on the cluster.

Broker client

BrokerClient supports connecting to a session and the decoupled sending and receiving of typed messages to and from a WCF service. The BrokerClient constructor wraps your WCF client (client proxy) to override standard call behavior and create a message-based proxy. In the standard WCF client, a single call sends and receives responses. In the BrokerClient, sending and receiving are separate calls.

Important
When you include a service reference in your project, Visual Studio creates files that are required for the proxy class. To support asynchronous calls and the BrokerClient object, you must select the following advanced options for the service reference:
  • Generate asynchronous operations. This option generates methods such as BeginGetData and EndGetData in the resulting WCF client object.

  • Always generate message contracts. This option implements the GetDataRequest and GetDataResponse data types for the BrokerClient object.

A BrokerClient object provides the following methods for sending and receiving messages:

  • SendRequests() – Sends a batch of requests to the broker. In non-durable sessions, requests start being processed as soon they are received. In durable sessions, the broker does not start processing requests until EndRequests() is called. If you have a very large batch of request, you can break it up into separate BrokerClient instances so you do not have to wait for all requests to be sent before processing begins.

  • EndRequests() - Confirms that all messages have been sent and ends the enumerator for the responses. In durable sessions, this call signals that the broker node should start processing requests. If this call fails during a durable session, you must re-send the batch of requests.

  • GetResponses() – Returns a response enumerator so you can loop through to get the responses.

Within one session, you can create multiple BrokerClient objects. You can use each instance of the broker client to send a batch of requests and receive the corresponding responses. When you create multiple instances of a BrokerClient, you must give each instance a unique ID to identify the responses. You can create multiple BrokerClient instances to:

  • Send a batch of requests with one parameter set, change the parameter set, and then send a new set of requests in a different BrokerClient instance within the same session.

  • Run multiple models at the same time with different parameters using different BrokerClient instances.

  • In durable sessions, break up large batches of request into separate BrokerClient instances so you do not have to wait for all requests to be sent before processing begins.

  • Multiple client applications can share the same session by creating BrokerClient instances with unique identifiers. Each batch of computations is tagged with the BrokerClient ID to identify requests and retrieve results.

The BrokerClient is not thread safe, so the client should not call SendRequests() in multiple threads.

DataClient (Added in SP2)

DataClient support staging and accessing common data that is required by all calculation requests within one or more sessions. You can create a new type of client called a DataClient. The data client includes methods to upload data to the cluster (to the runtime user data shared folder) and to read and write data. If you want the data to be available to other cluster users, you can specify the list of users when you call DataClient.Create(). Optionally, you can associate the data to the session lifecycle, so that when the session ends, the data is automatically deleted from the share. Code samples are available in the SP2 SDK code sample download. Common data features are not supported on Azure Nodes.

InProcess Broker (Added in SP2)

The SP2 APIs include an option to enable an in-process broker. The in-process broker runs in the client process, and thereby eliminates the need for a broker node, reduces session creation time, and reduces the number of hops for each message. For example, one usage pattern for the in-process broker is as follows: Instead of running the client application on a client computer, you submit the client application to the cluster as a single-task job. The client application creates a session on the cluster, and instead of passing messages through a broker node, the client sends requests and receives responses directly from the service hosts (compute nodes). Code samples are available in the SDK code sample download. The in-process broker supports interactive sessions only, and is not supported on Azure Nodes.

You can enable the in-process broker by setting the SessionStartInfo.UseInprocessBroker Property property to true.

Changes to the service configuration file

The service configuration file is an XML file that is used to register the service on the cluster and configure service behavior. The service configuration file must be named servicename.config, where the servicename is the same as that passed into the SessionStartInfo constructor. At a minimum, the file must specify the registration information for the service, such as the path to the DLL for the service. You provide the registration information inside the <microsoft.Hpc.Session.ServiceRegistration> section which contains the <service> element.

In Windows HPC Server 2008, configuration settings that applied to a SOA service were defined in three separate configuration files. In Windows HPC Server 2008 R2, all of the settings are defined in a single file: the service configuration file (servicename.config). Even the broker settings, which define how the broker handles service sessions, are defined at the individual service level. In R2, the service configuration file can include the <microsoft.Hpc.Broker>, <system.serviceModel>, and <system.Diagnostics> sections. In Windows HPC Server 2008, these sections were defined in the HpcWcfBroker.exe.config and the serviceAssembly.dll.config files. For more information, see SOA Service Configuration Files in Windows HPC Server 2008 R2.

Note
The endpointNotFoundRetryTimeout setting in the service configuration file was replaced with two settings: the serviceInitializationTimeout attribute of the <service> element (in the <microsoft.Hpc.Session.ServiceRegistration> section), and the endpointNotFoundRetryPeriod attribute of the <loadBalancing> element (in the <Microsoft.Hpc.Broker> section). For usage information, see SOA Service Configuration Files in Windows HPC Server 2008 R2.

New items in the API

The new items in the Microsoft.Hpc.Scheduler.Session namespace include:

  • BrokerClient class

  • BrokerResponse class

  • BrokerResponseEnumerator class

  • BrokerResponseHandler delegate

  • DurableSession class

  • Session.AttachSession method

  • SessionAttachInfo class

  • SessionBase class

  • SessionStartInfo.SessionStartInfo.System.Runtime.Serialization.ISerialization.ISerializable.GetObjectData method

  • SessionStartInfo.Constructor (SerializationInfo, StreamingContext)

  • SessionStartInfo.Environments property

  • SessionStartInfo.SavePassword property

Added in SP1:

  • BrokerClient.Flush method

  • BrokerClient.RequestCount property

  • Session.ServiceVersion property

  • SessionStartInfo.CanPreempt property

Items in the Microsoft.Hpc.Scheduler.Session that are no longer supported in Windows® HPC Server 2008 R2 include:

  • SessionStartInfo.HasRuntime property

  • SessionStartInfo.BrokerSettingsInfo constructor

  • SessionStartInfo.BrokerSettingsInfo.AllocationGrowLoadRatioThreshold property

  • SessionStartInfo.BrokerSettingsInfo.AllocationShrinkLoadRatioThreshold property

Developing Excel workbooks that can run on a cluster

HPC Services for Excel supports a number of programming models for developing and running Excel calculations on a Windows HPC Server 2008 R2 cluster. Workbooks that are suitable for cluster acceleration include independent calculations that can run in parallel. Many complex and long-running workbooks run iteratively—that is, they perform a single calculation many times over different sets of input data. These workbooks might contain complex Microsoft Visual Basic for Applications (VBA) functions or calculation intensive XLL add-ins.

HPC Services for Excel support offloading workbooks to the cluster or offloading UDFs to the cluster. The following table describes when workbook or UDF offloading might be appropriate:

Offloading workbooks Offloading UDFs
  • There is a long-running workbook that calculates iteratively (that is, runs the same calculation many times over different sets of input data).

  • The individual calculations are independent: that is, the results of one single calculation don’t affect any of the other calculations.
  • There is a long-running workbook and much of the calculation time is consumed by UDFs in the spreadsheet.

  • Functions are self-contained: They take arguments and return a value but do not otherwise look up data or make changes to the spreadsheet.

  • There is an existing XLL containing UDFs that can be recompiled to support running in a Windows HPC cluster; or, a user has native code (C or C++) that he or she wants to call from Excel, and the user can build an XLL to provide UDFs.

To find or share development resources, see Windows HPC and Microsoft Excel: Resources for Building Cluster-Ready Workbooks on the TechNet wiki.

For an overview of the HPC Services for Excel solutions, see:

Note
Based on customer and partner feedback from the Beta version, we made a number of breaking changes from the Beta release. For more information about updating Excel solutions from Beta, see the Excel Workbook Offloading Beta Transition Guide.

SDK samples and documentation

The SDK download includes samples for workbook and UDF offloading in the HPC2008R2.SampleCode\Excel folder. The folder also includes the HPCExcel.pdf document that explains the HPC Services for Excel solutions and the associated samples in detail. You can download the SDK and the samples from the HPC Pack 2008 R2 Software Development Kit.

Workbook offloading

To run on the cluster, you define independent units of calculation (for example individual cells, rows, or columns) that can run asynchronously and independently on the cluster. To help distribute these units of calculation onto a cluster and merge the results into the workbook, HPC Services for Excel provides the HPC Macro framework which allows VBA macros to define how calculations are to be partitioned and their results merged. Additional .NET APIs allow implementation of more complex calculation patterns.

For workbook offloading, the HPC Services for Excel include a built-in SOA client (ExcelClient) and a built-in service (Microsoft.Hpc.Excel.ExcelService). The HPC macro framework integrates with the built-in ExcelClient. You can also create custom clients that use the built-in service, or you can build your own clients and services.

The following table lists the development options for workbook offloading and related samples and resources. The workbook offloading samples are in the HPC2008R2.SampleCode\Excel\excelworkbook folder. The HPCExcel.pdf sections listed in the table refer specifically to the related samples (HPCExcel.pdf is included in the SDK samples folder).

Development option SDK Sample code HPCExcel.pdf section Additional resources

VBA and the macro framework

VBASample.xlsm

Section 3.4.1

Accelerating Excel 2010 with Windows HPC Server 2008 R2: Building VBA applications and workbooks for a Windows HPC Cluster (White paper and additional code and workbook samples)

.NET and the macro framework

HPCExcelAddIn project

Section 3.4.2

Accelerating Excel 2010 with Windows HPC Server 2008 R2: Building .NET Applications that Run Excel Calculations on a Windows HPC Cluster (White paper and additional code and workbook samples)

.NET with custom client

MonteCarloCLI project

Section 3.4.3

Accelerating Excel 2010 with Windows HPC Server 2008 R2: Building Custom Clients and Services (White paper and additional code and workbook samples)

.NET with custom client and service

HelloWorld (CLI/DurableCLI/Service)

Section 3.5

Accelerating Excel 2010 with Windows HPC Server 2008 R2: Building Custom Clients and Services (White paper and additional code and workbook samples)

UDF offloading

Microsoft Excel 2010 extends the UDF model to the cluster by enabling Excel 2010 UDFs to run in a Windows HPC cluster. In the cluster, UDFs work much like traditional UDFs, except that the calculation is performed by one or more servers. The key benefit is parallelization. If a workbook contains calls to long-running UDFs, multiple servers can be used to evaluate functions simultaneously. In order to run on the cluster, the UDFs must be contained in a cluster-safe XLL file.

The Excel samples in the Windows HPC Pack 2008 R2 SDK include code that you can compile to build a cluster-safe UDF contained in an XLL file named ClusterUDF.xll (in the HPC2008R2.SampleCode \Excel\exceludf folder). Section 4.2 of HPCExcel.pdf explains the UDF sample in detail (HPCExcel.pdf is included in the SDK samples folder).

Resources:

Diagnostics extensibility

In Windows HPC Server 2008 R2, cluster administrators and partners can create custom diagnostic tests. Partners include independent software vendors (ISVs), independent hardware vendors (IHVs), original equipment manufacturers (OEMs), and system integrators. Cluster administrators can add these tests to the list of diagnostic tests for their HPC cluster, and then run them in the same way as the built-in diagnostic tests for Windows HPC Server 2008 R2. With custom diagnostic tests, cluster administrators can verify that custom and non-Microsoft software or hardware that they have added to their HPC cluster works correctly.

For more information about writing and installing custom diagnostic tests, see:

Diagnostics Extensibility in Windows HPC Server 2008 R2 CTP2 Step-by-Step Guide on MSDN (https://go.microsoft.com/fwlink/?LinkId=177604). This guide describes custom diagnostic tests, how to define and implement several types of custom diagnostic tests, and how to add custom diagnostic tests to an HPC cluster.

Code sample. The SDK download includes samples for diagnostics extensibility in the HPC2008R2.SampleCode\Diagnostics folder. You can download the SDK and the samples from the HPC Pack 2008 R2 Software Development Kit.

Reporting Extensibility

In Windows HPC Server 2008 R2, cluster administrators and partners can create custom reports from the same data sources that HPC Cluster Manager uses for the built-in reports. This allows cluster administrators to get information that is tailored to their individual needs about the use and performance of the HPC cluster.

For more information about creating custom reports, see:

Reporting Extensibility on MSDN. This guide describes the features that Windows HPC Server 2008 R2 includes to provide data for custom reports about cluster use and performance. This guide also presents use cases that demonstrate how to use Microsoft products to analyze and create custom reports from the reporting data that Windows HPC Server 2008 R2 provides.

Code sample. You can download a reporting extensibility sample that demonstrates how to build web-based reports using Microsoft Silverlight. These web-based reports provide the flexibility to view reports about your HPC cluster from anywhere that you have internet or intranet access. Additionally, you can choose to make the web-based reports available to cluster users or other stakeholders who are not cluster administrators. The sample includes the following three reports:

  • Charge back report: calculates the total core time used by each job owner times the charge rate per core. Results are per day, week, or month. The rate can be different for nodes belonging to the different node groups.

  • Capacity Planning Report: displays the trend of aggregated CPU utilization, available memory, and network utilization for each node group for a selected time range.

  • Cluster Utilization Report: displays the overall cluster utilization based on all available cores (online and reachable) and all cores for a selected time range. Utilization is calculated as allocated core time (allocated to a job) divided by total core time.

The code sample and accompanying documentation are available in the Reporting.Sample.zip file on the HPC Pack 2008 R2 Software Development Kit download page.

Scheduler Integration

The HPC APIs can help you integrate your application with the HCP Job Scheduler Service. The Using HPC section in the MSDN developer library for High Performance Computing includes code snippets for common tasks using the HPC APIs such as connecting to a cluster and submitting a job. In most cases, code snippets are available in C# and C++.

The SDK download also includes samples for integrating your applications with the HPC Job Scheduler service in the <HPCVersion>.SampleCode\Scheduler folder. You can download the SDK and the samples from the HPC Pack 2008 R2 Software Development Kit.

License aware scheduling

The HPC Job Scheduler Service can run a custom activation filter on jobs that are about to get cluster resources. A job activation filter is a custom application that you can write to provide additional checks and controls, such as checking for license availability. Depending on the return value from your filter, the HPC Job Scheduler Service takes the appropriate action on the job.

The HPC 2008 R2 SDK includes sample code for an activation filter that checks for license availability against a FlexLM license file.

For detailed information about the new exit codes for Activation Filters and more information about building the code sample, see the “Check for license availability before a job is started” section in the New Feature Evaluation Guide for Windows HPC Server 2008 R2.

Job progress information

Windows HPC Server 2008 R2 includes changes to support programming integration with the scheduler. Two new properties in Microsoft.Hpc.Scheduler namespace (ISchedulerJob.Progress and ISchedulerJob.ProgressMessage) allow applications to communicate progress to the job owner through the HPC job submission interfaces (HPC Job Manager, HPC PowerShell, and the command line interface). For more information, see Setting Custom Job Progress Information in Windows HPC Server 2008 R2 Step-by-Step Guide.

For more information about the Microsoft.Hpc.Scheduler namespace, see the Scheduler APIs in the HPC Class Library (https://go.microsoft.com/fwlink/?LinkID=167944).

Graceful exit during task cancellation

In Windows HPC Server 2008 R2, the HPC Node Manager Service stops a running task by sending a CTRL_BREAK signal to the application. If an application includes code to handle the CTRL_BREAK event, it can make use of the Task Cancellation Grace Period to exit gracefully. The Task Cancellation Grace Period is a cluster property that can be set by the cluster administrator. The default time is 15 seconds. If the application does not respond to the CTRL_BREAK event, it exits immediately.

NetworkDirect service provider development

The NetworkDirect service provider interface (NetworkDirect SPI) defines an interface that networking hardware vendors can implement to expose the advanced capabilities of their devices, such as remote direct memory access (RDMA) between computers, and that allows applications to leverage networking technologies such as iWARP and InfiniBand. This document describes version 2 of the NetworkDirect SPI, implementation requirements and suggestions, and usage of the NetworkDirect interface.

Download the document here: NetworkDirect Service Provider Interface Version 2 (https://www.microsoft.com/downloads/details.aspx?displaylang=en\&FamilyID=0fe51823-660d-489c-9c48-fd50d9f54377).

Download the NetworkDirect DDK here: HPC Pack 2008 R2 Software Development Kit (https://go.microsoft.com/fwlink/?LinkId=198809).

Resources for Azure integration

This section provides information about running your parallel applications in Azure.

Burst to Azure from an on-premises head node

Windows® HPC Server 2008 R2 SP1 and later enables administrators to increase the power of the on-premises cluster by adding computational resources in Azure. With the Azure “burst” scenario, various types of HPC applications can be deployed to Azure nodes and run on these nodes in the same way they run in on-premises nodes. For some types of applications, running on Azure nodes might require some additional considerations and steps on the part of the administrator, job owner, and application developer.

For more information, you can download the white paper and code sample from Windows HPC with Burst to Azure: Application Models and Data Consideration. The code sample download includes samples of parametric sweep, SOA, MPI and Excel UDF applications that can burst to Azure. The download also includes the AzureBlobCopy utility that is mentioned in the paper. This utility helps you copy files to and from Azure storage, Azure nodes, and on-premises computers.

An updated version of the code samples was uploaded in September 2011, new samples are denoted as “New!”. The MPI samples are supported on Azure Nodes that are deployed on a Windows HPC Server 2008 R2 cluster that has Service Pack 2 installed.

Samples include:

  • Parametric sweeps

    • CPU spinner

    • Image rendering on Renderman files (console client)

    • Image rendering on Renderman files (Windows Phone 7 client) - New!

    • BLAST - New!

  • SOA

    • Square Service

    • Asian Options service

    • Word-association service - New!

  • MPI

    • Pi (π) Calculator - New!

    • Tachyon ray-tracing - New!

  • Excel

    • Asian Options UDF

See the next section for information about trial Azure accounts and adding Azure Nodes to a development cluster.

Resources for setting up a development cluster

For information about downloading the Windows HPC Pack 2008 R2 client utilities and the evaluation suite for setting up a test cluster, see Download Windows HPC Server 2008 R2 software.

For information about deploying your cluster, see:

For information about signing up for a free Azure trial (or getting Azure hours as part of an MSDN subscription) go to the Microsoft Azure website.

For information about adding Azure Nodes to your HPC development cluster: