Foundry error 424 : ms-azureml-model-error-reason': 'too_few_model_instance'

Question

Foundry error 424 : ms-azureml-model-error-reason': 'too_few_model_instance'

Enzo Bianchi 0

We have been experiencing intermittent HTTP 424 errors when calling an Azure AI / Azure ML online endpoint for the past 2 days (model deployment: mistral-small-2503, region: East US). The issue occurs approximately 50% of the time. The error returned is: “424 upstream connect error or disconnect/reset before headers, latest reset reason: remote reset”. The response headers include ms-azureml-model-error-reason: too_few_model_instance, along with identifiers such as x-request-id and x-ms-client-request-id.

The requests are made using the official Azure/OpenAI SDK from a backend service. Could you help clarify what might be causing this intermittent behavior and how to investigate it further?

Thank you for your help

Amira Bedhiafi 41,291 Reputation points Volunteer Moderator

2026-04-15T17:39:18.24+00:00

Hello Enzo !

Thank you for posting on MS Learn Q&A.

What I understood from the error is that the serving layer doesn’t have enough ready model instances for some requests so this looks like a backend capacity issue rather than a payload problem.

Since it is intermittent, use retries with exponential backoff, then check Azure Monitor for the endpoint and deployment around the failing timestamps.

If it persists, open an Azure support ticket and include the failed x-request-id, your x-ms-client-request-id, UTC timestamps, region and model and failure rate.

If this is a quota based online deployment also review min instances and autoscale and if possible test another region to confirm whether the issue is East US specific.
Enzo Bianchi 0 Reputation points

2026-04-15T18:10:50.8266667+00:00

Hello, Thank you for your response.

We are currently using Azure AI Foundry, and as far as I can see, I do not have control over scaling settings such as min instances or autoscaling for this deployment (or at least it is not exposed in the interface).

We are also running the same workload in two regions: East US and Canada Central, and we are observing the same intermittent 424 errors in both regions. This suggests the issue is not region-specific.

Additionally, this problem started recently, but we have been using this setup for approximately 5 months without any issues, which makes this behavior unexpected.

Regarding opening a support ticket, we are currently unable to do so via the Azure portal. The support flow redirects us to Copilot troubleshooting, and we are not able to create a direct support request from there.

Could you please advise:

Whether scaling / instance management is expected to be configurable in Azure AI Foundry for this type of deployment and how to properly open a support ticket in this situation (without being blocked by the Copilot troubleshooting flow)

Thank you
SRILAKSHMI C 16,975 Reputation points Microsoft External Staff Moderator

2026-04-15T18:20:01.1066667+00:00
Hello Enzo Bianchi

Thank you for sharing the detailed information and error context.

Based on the error message HTTP 424 with ms-azureml-model-error-reason: too_few_model_instance

this indicates that the endpoint is intermittently running out of available model instances to serve incoming requests. In such cases, when the number of incoming requests exceeds the number of healthy/ready replicas, some connections are reset before a response can be returned, resulting in the behavior you’re observing.

This aligns with your observation of ~50% intermittent failures and is typically related to capacity or scaling constraints rather than an issue with the request payload or SDK.

Recommended checks and actions

1. Review endpoint metrics Please navigate to your Online Endpoint → Deployments → Metrics and review the following during the timeframe when failures occur:

Instance count

CPU and memory utilization

Queue length

If you notice instance count staying low or queue length increasing, this indicates the endpoint is under-provisioned for the current traffic.

2. Verify scaling configuration

If using autoscale:

Check minimum and maximum replica settings

The default minimum is often low (0 or 1), which can lead to cold-start delays

Increasing the minimum replica count can help maintain availability

If using fixed scaling:

Consider increasing the number of replicas

Alternatively, evaluate a larger SKU to handle higher concurrency per instance

3. Enable logging and diagnostics Enable Azure Monitor / Application Insights for the endpoint to review:

Scaling events (scale-in/scale-out)

Instance health

Any throttling or dropped requests

You may also correlate failed requests using x-request-id or x-ms-client-request-id for deeper analysis.

4. Mitigate cold-start impact If autoscaling is involved:

Ensure minimum replicas are set appropriately

Optionally send periodic lightweight requests to keep instances warm

5. Implement retry logic As this is a transient condition, it is recommended to implement retry with exponential backoff in your client to improve reliability.

Please refer this

I Hope this helps. Do let me know if you have any further queries.

Thank you!
Enzo Bianchi 0 Reputation points

2026-04-15T19:03:26.9666667+00:00

Hello,
Thank you we are using the deployment with Azure foundry, I don't have option to scale. But in metrics we have only 300 requests today. I tried from curl directly, I have the same 424 error, Is it possible that Azure isn't deploying enough Mistral instance ?
SRILAKSHMI C 16,975 Reputation points Microsoft External Staff Moderator

2026-04-17T08:19:53.7766667+00:00
Hi Enzo Bianchi,

Thank you for the additional details,

You are correct in your understanding, and based on what you’ve described, this is not something you can control from your side in Azure AI Foundry.

What is happening

The error ms-azureml-model-error-reason: too_few_model_instance indicates that there are not enough active model instances available on the service side to handle incoming requests at that moment.

In Azure AI Foundry (for models like mistral-small-2503):

Scaling, replica count, and instance management are fully managed by Microsoft

These settings are not exposed to customers

You cannot manually increase min/max instances or autoscale settings

Why this is happening in your case

Based on your inputs:

Only ~300 requests/day -- very low traffic

Issue reproduced via curl -- not SDK-related

Occurring in multiple regions (East US & Canada Central) -- not region-specific

Started recently after months of stable behavior

This strongly indicates a backend capacity or service-side regression, not a configuration issue on your end.

In simple terms: Even though your traffic is low, the platform is intermittently not maintaining enough ready instances for this model, leading to request drops (HTTP 424).

Is Azure deploying enough instances?

From your observation, it appears that The platform is intermittently under-provisioning model instances for this deployment

This can happen due to:

Temporary capacity pressure on specific models (e.g., Mistral)

Scaling delays or cold-start behavior in the managed service

Recent backend changes or regression

What you can do

Since scaling is not customer-controlled in Foundry, the options are:

1. Implement retry logic

Use exponential backoff retries

This helps absorb transient instance shortages

This is currently the most effective mitigation

2. Test with another model

If possible, test with a different model (e.g., GPT family or another Mistral variant)

This helps confirm if the issue is model-specific capacity

3. Continue multi-region usage

You are already doing this (which is good)

Even though both regions show issues, this still improves resiliency

You are not expected to manage scaling in Azure AI Foundry for this deployment

The error indicates insufficient backend model instances (service-side)

Your scenario strongly points to a platform-side capacity or scaling issue

Thank you!

1 answer

Your answer

Amira Bedhiafi 41,291 Reputation points Volunteer Moderator

2026-04-15T17:39:18.24+00:00

Hello Enzo !

Thank you for posting on MS Learn Q&A.

What I understood from the error is that the serving layer doesn’t have enough ready model instances for some requests so this looks like a backend capacity issue rather than a payload problem.

Since it is intermittent, use retries with exponential backoff, then check Azure Monitor for the endpoint and deployment around the failing timestamps.

If it persists, open an Azure support ticket and include the failed x-request-id, your x-ms-client-request-id, UTC timestamps, region and model and failure rate.

If this is a quota based online deployment also review min instances and autoscale and if possible test another region to confirm whether the issue is East US specific.
Enzo Bianchi 0 Reputation points

2026-04-15T18:10:50.8266667+00:00

Hello, Thank you for your response.

We are currently using Azure AI Foundry, and as far as I can see, I do not have control over scaling settings such as min instances or autoscaling for this deployment (or at least it is not exposed in the interface).

We are also running the same workload in two regions: East US and Canada Central, and we are observing the same intermittent 424 errors in both regions. This suggests the issue is not region-specific.

Additionally, this problem started recently, but we have been using this setup for approximately 5 months without any issues, which makes this behavior unexpected.

Regarding opening a support ticket, we are currently unable to do so via the Azure portal. The support flow redirects us to Copilot troubleshooting, and we are not able to create a direct support request from there.

Could you please advise:

Whether scaling / instance management is expected to be configurable in Azure AI Foundry for this type of deployment and how to properly open a support ticket in this situation (without being blocked by the Copilot troubleshooting flow)

Thank you
Enzo Bianchi 0 Reputation points

2026-04-15T19:03:26.9666667+00:00

Hello,
Thank you we are using the deployment with Azure foundry, I don't have option to scale. But in metrics we have only 300 requests today. I tried from curl directly, I have the same 424 error, Is it possible that Azure isn't deploying enough Mistral instance ?

Answer 1

Enzo Bianchi 0

--Wrong comment

0 comments

Share via

Foundry error 424 : ms-azureml-model-error-reason': 'too_few_model_instance'

1 answer

Your answer