ML.NET model accuracy drops after deployment to production

PredictionEngine is not safe for concurrent ASP.NET requests.

PredictionEngine is explicitly documented as:

• A convenience API for single predictions.
• Not thread-safe.
• Acceptable only in single-threaded or prototype environments.

In a production ASP.NET Core application handling concurrent requests, using a shared PredictionEngine instance can cause race conditions and corrupt internal state, which can manifest as unstable or degraded prediction accuracy after deployment.

For production and concurrent scenarios, use PredictionEnginePool instead. PredictionEnginePool:

• Creates an ObjectPool of PredictionEngine instances.
• Provides improved performance and thread safety.
• Can automatically reload models when the underlying file changes (when configured with watchForChanges: true).

In an ASP.NET Core app, register it in Program.cs like this:

builder.Services.AddPredictionEnginePool<ModelInput, ModelOutput>()
    .FromFile(modelName: "SentimentAnalysisModel", filePath: "sentiment_model.zip", watchForChanges: true);

Then inject and use the pool in controllers/services instead of a single PredictionEngine instance.

Using PredictionEnginePool is the recommended approach for production, concurrent ASP.NET workloads and can help prevent the kind of reliability issues observed after deployment.

References:

• Deploy a model in an ASP.NET Core Web API
• Make predictions with a trained model
• Tutorial: Categorize support issues using multiclass classification with ML.NET
• Tutorial: Analyze sentiment of website comments with binary classification in ML.NET
• Tutorial: Train an ML.NET classification model to categorize images
• Tutorial: Categorize iris flowers using k-means clustering with ML.NET
• Tutorial: Build a movie recommender using matrix factorization with ML.NET