Deploy a model to Azure Functions

Article
12/17/2022

Learn how to deploy a pre-trained ML.NET machine learning model for predictions over HTTP through an Azure Functions serverless environment.

Prerequisites

Visual Studio 2022 with the .NET desktop development and Azure development workloads installed. The .NET 6 SDK is automatically installed when you select this workload.
Azure Functions Tools
PowerShell
Pre-trained model. Download this pre-trained sentiment analysis machine learning model or use the ML.NET Sentiment Analysis tutorial to build your own model.

Azure Functions sample overview

This sample is a C# HTTP Trigger Azure Functions application that uses a pretrained binary classification model to categorize the sentiment of text as positive or negative. Azure Functions provides an easy way to run small pieces of code at scale on a managed serverless environment in the cloud. The code for this sample can be found on the dotnet/machinelearning-samples repository on GitHub.

Create Azure Functions project

In Visual Studio 2022 open the Create a new project dialog.
In the "Create a new project" dialog, select the Azure Functions project template.
In the Name text box, type "SentimentAnalysisFunctionsApp" and select the Next button.
In the "Additional information dialog", leave all the defaults as is and select the Create button.
Install the Microsoft.ML NuGet Package
1. In Solution Explorer, right-click on your project and select Manage NuGet Packages.
2. Choose "nuget.org" as the Package source.
3. Select the "Browse" tab.
4. Search for Microsoft.ML.
5. Select that package in the list, and select the Install button.
6. Select the OK button on the Preview Changes dialog
7. Select the I Accept button on the License Acceptance dialog if you agree with the license terms for the packages listed.
Follow the same steps to install the Microsoft.Extensions.ML, Microsoft.Extensions.DependencyInjection, and Microsoft.Azure.Functions.Extensions NuGet packages.

Add pre-trained model to project

Create a directory named MLModels in your project to save your pre-build model: In Solution Explorer, right-click on your project and select Add > New Folder. Type "MLModels" and hit Enter.
Copy your pre-built model to the MLModels folder.
In Solution Explorer, right-click your pre-built model file and select Properties. Under Advanced, change the value of Copy to Output Directory to Copy if newer.

Create Azure Function to analyze sentiment

Create a class to predict sentiment. Add a new class to your project:

In Solution Explorer, right-click the project, and then select Add > New Azure Function.
In the Add New Item dialog box, select Azure Function and change the Name field to AnalyzeSentiment.cs. Then, select the Add button.

In the New Azure Function dialog box, select Http Trigger and choose Anonymous from the Authorization level dropdown. Then, select the OK button.

The AnalyzeSentiment.cs file opens in the code editor. Add the following using directive to the top of AnalyzeSentiment.cs:

using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using Microsoft.Extensions.ML;
using SentimentAnalysisFunctionsApp.DataModels;

By default, the AnalyzeSentiment class is static. Make sure to remove the static keyword from the class definition.

public class AnalyzeSentiment
{

}

Create data models

You need to create some classes for your input data and predictions. Add a new class to your project:

Create a directory named DataModels in your project to save your data models: In Solution Explorer, right-click on your project and select Add > New Folder. Type "DataModels" and hit Enter.
In Solution Explorer, right-click the DataModels directory, and then select Add > Class.
In the Add New Item dialog box, select Class and change the Name field to SentimentData.cs. Then, select the Add button.

The SentimentData.cs file opens in the code editor. Add the following using directive to the top of SentimentData.cs:
```
using Microsoft.ML.Data;
```
Remove the existing class definition and add the following code to the SentimentData.cs file:
```
public class SentimentData
{
    [LoadColumn(0)]
    public string SentimentText;

    [LoadColumn(1)]
    [ColumnName("Label")]
    public bool Sentiment;
}
```
In Solution Explorer, right-click the DataModels directory, and then select Add > Class.
In the Add New Item dialog box, select Class and change the Name field to SentimentPrediction.cs. Then, select the Add button. The SentimentPrediction.cs file opens in the code editor. Add the following using directive to the top of SentimentPrediction.cs:
```
using Microsoft.ML.Data;
```
Remove the existing class definition and add the following code to the SentimentPrediction.cs file:
```
public class SentimentPrediction : SentimentData
{

    [ColumnName("PredictedLabel")]
    public bool Prediction { get; set; }

    public float Probability { get; set; }

    public float Score { get; set; }
}
```
SentimentPrediction inherits from SentimentData which provides access to the original data in the SentimentText property as well as the output generated by the model.

Register PredictionEnginePool service

To make a single prediction, you have to create a PredictionEngine. PredictionEngine is not thread-safe. Additionally, you have to create an instance of it everywhere it is needed within your application. As your application grows, this process can become unmanageable. For improved performance and thread safety, use a combination of dependency injection and the PredictionEnginePool service, which creates an ObjectPool of PredictionEngine objects for use throughout your application.

The following link provides more information if you want to learn more about dependency injection.

In Solution Explorer, right-click the project, and then select Add > Class.
In the Add New Item dialog box, select Class and change the Name field to Startup.cs. Then, select the Add button.

Add the following using directives to the top of Startup.cs:

using Microsoft.Azure.Functions.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.ML;
using SentimentAnalysisFunctionsApp;
using SentimentAnalysisFunctionsApp.DataModels;
using System.IO;
using System;

Remove the existing code below the using directives and add the following code:

[assembly: FunctionsStartup(typeof(Startup))]
namespace SentimentAnalysisFunctionsApp
{
    public class Startup : FunctionsStartup
    {

    }
}

Define variables to store the environment the app is running in and the file path where the model is located inside the Startup class
```
private readonly string _environment;
private readonly string _modelPath;
```

Below that, create a constructor to set the values of the _environment and _modelPath variables. When the application is running locally, the default environment is Development.

public Startup()
{
    _environment = Environment.GetEnvironmentVariable("AZURE_FUNCTIONS_ENVIRONMENT");

    if (_environment == "Development")
    {
        _modelPath = Path.Combine("MLModels", "sentiment_model.zip");
    }
    else
    {
        string deploymentPath = @"D:\home\site\wwwroot\";
        _modelPath = Path.Combine(deploymentPath, "MLModels", "sentiment_model.zip");
    }
}

Then, add a new method called Configure to register the PredictionEnginePool service below the constructor.

public override void Configure(IFunctionsHostBuilder builder)
{
    builder.Services.AddPredictionEnginePool<SentimentData, SentimentPrediction>()
        .FromFile(modelName: "SentimentAnalysisModel", filePath: _modelPath, watchForChanges: true);
}

At a high level, this code initializes the objects and services automatically for later use when requested by the application instead of having to manually do it.

Machine learning models are not static. As new training data becomes available, the model is retrained and redeployed. One way to get the latest version of the model into your application is to restart or redeploy your application. However, this introduces application downtime. The PredictionEnginePool service provides a mechanism to reload an updated model without restarting or redeploying your application.

Set the watchForChanges parameter to true, and the PredictionEnginePool starts a FileSystemWatcher that listens to the file system change notifications and raises events when there is a change to the file. This prompts the PredictionEnginePool to automatically reload the model.

The model is identified by the modelName parameter so that more than one model per application can be reloaded upon change.

Tip

Alternatively, you can use the FromUri method when working with models stored remotely. Rather than watching for file changed events, FromUri polls the remote location for changes. The polling interval defaults to 5 minutes. You can increase or decrease the polling interval based on your application's requirements. In the code sample below, the PredictionEnginePool polls the model stored at the specified URI every minute.

builder.Services.AddPredictionEnginePool<SentimentData, SentimentPrediction>()
  .FromUri(
      modelName: "SentimentAnalysisModel",
      uri:"https://github.com/dotnet/samples/raw/main/machine-learning/models/sentimentanalysis/sentiment_model.zip",
      period: TimeSpan.FromMinutes(1));

Load the model into the function

Insert the following code inside the AnalyzeSentiment class:

public AnalyzeSentiment(PredictionEnginePool<SentimentData, SentimentPrediction> predictionEnginePool)
{
    _predictionEnginePool = predictionEnginePool;
}

This code assigns the PredictionEnginePool by passing it to the function's constructor which you get via dependency injection.

Use the model to make predictions

Replace the existing implementation of Run method in AnalyzeSentiment class with the following code:

public async Task<IActionResult> Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req,
    ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    // Parse HTTP Request Body
    string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
    SentimentData data = JsonConvert.DeserializeObject<SentimentData>(requestBody);

    //Make Prediction
    SentimentPrediction prediction = _predictionEnginePool.Predict(modelName: "SentimentAnalysisModel", example: data);

    //Convert prediction to string
    string sentiment = Convert.ToBoolean(prediction.Prediction) ? "Positive" : "Negative";

    //Return Prediction
    return new OkObjectResult(sentiment);
}

When the Run method executes, the incoming data from the HTTP request is deserialized and used as input for the PredictionEnginePool. The Predict method is then called to make predictions using the SentimentAnalysisModel registered in the Startup class and returns the results back to the user if successful.

Test locally

Now that everything is set up, it's time to test the application:

Run the application
Open PowerShell and enter the code into the prompt where PORT is the port your application is running on. Typically the port is 7071.
```
Invoke-RestMethod "http://localhost:<PORT>/api/AnalyzeSentiment" -Method Post -Body (@{SentimentText="This is a very bad steak"} | ConvertTo-Json) -ContentType "application/json"
```
If successful, the output should look similar to the text below:
```
Negative
```

Congratulations! You have successfully served your model to make predictions over the internet using an Azure Function.

Next Steps

Deploy to Azure

Share via