APIM OpenAI loadbalancer - requests go to only two different locations

Hello @70708851,

The behavior you're experiencing with Azure API Management (APIM) routing requests primarily to West Europe and Sweden Central could be influenced by several factors. One possibility is that the APIM instance is configured to prioritize certain regions based on availability or performance metrics. If the backend services in Germany West Central, Poland Central, and Italy North are experiencing low availability or performance issues, the APIM may automatically redirect traffic to the more stable regions.

Additionally, Azure's load balancing mechanisms might favor regions that have lower latency or higher capacity at the time of the request. If the backend services in the other regions are not as responsive or are under heavy load, requests may be routed to the regions that can handle them more effectively. To ensure optimal routing, you might want to check the configuration of your APIM instance and the health of your backend services in the less utilized regions.

For further clarification, please refer to the following documentations:

https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-deploy-multi-region#route-api-calls-to-regional-backend-services

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/business-continuity-disaster-recovery#designing-for-consumption-through-the-microsoft-public-backbone

I hope this helps resolve your issue. Feel free to reach out if you have further concerns.

Hi @70708851,

Just checking in to see if above information was helpful. If you have any further updates on this issue, please feel free to post back.

Hi,

thank you very much for your response.

The general unavailability of GPT models is responsible for this, I think, and the Data Zone deployment type does this mechanism. Becuase we have nothing in West Europe region.

I will try to put only regional standard deployment types into APIM to confirm my theory in the future.

Thanks again.

Hi @70708851,
There seems to be a lot of General available models in OpenAI. Those models which are not containing preview in their names are mostly General available models.

You can check all General available models and their availability on different types of deployment's here. Azure OpenAI Service models Global deployment normally fall back to nearest data centers with low latency.

You can find more details about different deployment and their routing and regional persistence here. Azure OpenAI deployment types.

Hi,

by unavailability, I mean that there are no quotas for regional API

Example: France Central has no quota for regional API, so Data Zone Deployment from the same region redirects the requests to West Europe cluster or Sweden Central cluster only, because there quota is available.

Technically, West Europe and Sweden Central are the only two available EU regions for OpenAI deployments at the moment.