Edit

Share via

Initialize execution providers with Windows ML

This page discusses more advanced ways your app can gracefully handle downloading and registering execution providers (EPs) using Windows ML. Even if an EP is already downloaded on the device, you must register the EPs every time your app runs so they will appear in ONNX Runtime.

Download and register in one call

For initial development, it can be nice to simply call EnsureAndRegisterCertifiedAsync(), which will download any new EPs (or new versions of EPs) that are compatible with your device and drivers if they're not already downloaded, and then register all the EPs. Note that on first run, this method can take multiple seconds or even minutes depending on your network speed and EPs that need to be downloaded.

// Get the default ExecutionProviderCatalog
var catalog = ExecutionProviderCatalog.GetDefault();

// Ensure and register all compatible execution providers with ONNX Runtime
// This downloads any necessary components and registers them
await catalog.EnsureAndRegisterCertifiedAsync();

Tip

In production applications, wrap the EnsureAndRegisterCertifiedAsync() call in a try-catch block to handle potential network or download failures gracefully.

Register existing providers only

If you want to avoid downloading and only register execution providers that are already present on the machine:

var catalog = ExecutionProviderCatalog.GetDefault();

// Register only providers already present on the machine
// This avoids potentially long download times
await catalog.RegisterCertifiedAsync();

Discover if there are new EPs (without downloading)

If you want to see if there are new EPs compatible with your device and drivers available to download, but don't want to start the download, you can use the FindAllProviders() method and then see if any providers have a ReadyState of NotPresent. You can then decide to handle this however you would like (launching your users into an "Updating screen", asking them if they want to update, etc). You can choose to continue using the already-downloaded EPs (by calling RegisterCertifiedAsync() as shown above) if you don't want to make your users wait right now.

var catalog = ExecutionProviderCatalog.GetDefault();

// Check if there are new EPs that need to be downloaded
if (catalog.FindAllProviders().Any(provider => provider.ReadyState == ExecutionProviderReadyState.NotPresent))
{
    // TODO: There are new EPs, decide how your app wants to handle that
}
else
{
    // All EPs are already present, just register them
    await catalog.RegisterCertifiedAsync();
}

Download and register a specific EP

If there is a specific execution provider your app wants to use, you can download and register a particular execution provider without downloading all compatible EPs.

You'll first use FindAllProviders() to get all compatible EPs, and then you can call EnsureReadyAsync() on a particular ExecutionProvider to download the specific execution provider, and call TryRegister() to register the specific execution provider.

var catalog = ExecutionProviderCatalog.GetDefault();

// Get the QNN provider, if present
var qnnProvider = catalog.FindAllProviders()
    .FirstOrDefault(i => i.Name == "QNNExecutionProvider");

if (qnnProvider != null)
{
    // Download it
    var result = await qnnProvider.EnsureReadyAsync();

    // If download succeeded
    if (result != null && result.Status == ExecutionProviderReadyResultState.Success)
    {
        // Register it
        bool registered = qnnProvider.TryRegister();
    }
}

Production app example

For production applications, here's an example of what your app might want to do to give yourself and your users users control over when downloads occur. You can check if new execution providers are available and conditionally download them:

using Microsoft.Windows.AI.MachineLearning;

var catalog = ExecutionProviderCatalog.GetDefault();

// Filter to the EPs our app supports/uses
var providers = catalog.FindAllProviders().Where(p =>
    p.Name == "VitisAIExecutionProvider" ||
    p.Name == "OpenVINOExecutionProvider" ||
    p.Name == "QNNExecutionProvider" ||
    p.Name == "NvTensorRtRtxExecutionProvider"
);

if (providers.Any(p => p.ReadyState == ExecutionProviderReadyState.NotPresent))
{
    // Show UI to user asking if they want to download new execution providers
    bool userWantsToDownload = await ShowDownloadDialogAsync();

    if (userWantsToDownload)
    {
        // Download all EPs
        foreach (var p in providers)
        {
            if (p.ReadyState == ExecutionProviderReadyState.NotPresent)
            {
                // Ignore result handling here; production code could inspect status
                await p.EnsureReadyAsync();
            }
        }

        // And register all EPs
        await catalog.RegisterCertifiedAsync();
    }
    else
    {
        // Register only already-present EPs
        await catalog.RegisterCertifiedAsync();
    }
}
  • Last updated on