Azure APIM intermittently receives ASP.NET Core HTTP 500.30 response from Azure AI Search backend

Question

Azure APIM intermittently receives ASP.NET Core HTTP 500.30 response from Azure AI Search backend

Djay 0

We are experiencing an intermittent issue in Azure API Management when calling an Azure AI Search backend.

The APIM operation forwards the following request:

GET https://<search-service>.search.windows.net/indexes/document/docs/?$top=10&api-version=2019-05-06

The same request through the same APIM instance intermittently returns either HTTP 200 or HTTP 500, without any configuration changes.

Successful request

HTTP 200
Content-Type: application/json
OData-Version: 4.0
Azure Search request-id returned
Backend response time approximately 19 ms in the captured APIM trace

Failed request

HTTP 500
Content-Type: text/html
Response size: 2416 bytes
Backend response received in approximately 3 ms in the captured APIM trace
Response body contains: HTTP Error 500.30 - ASP.NET Core app failed to start

Additional investigation performed

Direct REST calls to the Azure Search endpoint return HTTP 200.
Azure Search Explorer works successfully.
Azure Search operation logs checked during the affected period showed successful operations.
Search throttling does not correlate with the failures.
APIM SNAT Port Analysis reports no SNAT allocation issue.
APIM gateway LastErrorReason, LastErrorMessage, LastErrorSource and LastErrorSection are empty.
The equivalent operation works in our lower environment.
Successful and failed APIM traces show the same backend URL and effectively the same request being forwarded.

The unusual part is that the failed request receives an ASP.NET Core 500.30 HTML response despite the configured backend being Azure AI Search.

What could cause APIM traffic addressed to an Azure AI Search hostname to intermittently receive an ASP.NET Core 500.30 response instead of the expected Azure Search response?

Could this indicate an issue somewhere in the DNS, private networking, routing, or backend platform path? What additional telemetry could help identify which component is generating the 500.30 response?

I've attached sanitised APIM traces for both a successful and failed request for comparison.

APIMError(2).txt
APIMSuccess(2).txt

0 comments

1 answer

Your answer

Answer 1

Likely cause: APIM is intermittently reaching a different backend target than Azure AI Search (or a differently routed path), which can explain fast HTML failures with ASP.NET Core 500.30 instead of Search JSON responses.

Why this fits your data:

Direct calls to Azure AI Search succeed.
Failed APIM responses are HTML with ASP.NET Core startup signature, not typical Search error payload.
The same APIM operation alternates between success and failure, which points to routing or resolution variance.

Diagnostic branches:

DNS/private endpoint resolution differs across APIM paths.
APIM policy conditionally changes backend URL or headers.
Mixed public/private network path to Search causes inconsistent backend reachability.

Recommended steps:

Compare successful vs failed APIM traces for final backend URL and Host header.
Enable APIM diagnostics/resource logs and correlate failed calls by timestamp.
Validate Search endpoint networking model is consistent:
- public endpoint + firewall allow rules, or
- private endpoint + correct private DNS and VNet path.
Review APIM policies for conditional backend rewrite rules.
If issue persists, open Azure support with sanitized traces and exact UTC times.

Validation checks:

Run repeated test calls and classify by response content-type and status.
Confirm all failed calls map to one backend path pattern.
After DNS/policy fixes, verify failures disappear across APIM units.

References:

Azure API Management virtual network concepts: https://learn.microsoft.com/en-us/azure/api-management/virtual-network-concepts
Monitor APIs in API Management: https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-use-azure-monitor
Configure network access for Azure AI Search: https://learn.microsoft.com/en-us/azure/search/service-configure-firewall
Create private endpoint for Azure AI Search: https://learn.microsoft.com/en-us/azure/search/service-create-private-endpoint

“If you find this answer helpful, please upvote it. If this resolves your issue, please mark it as the accepted answer so that it can help others in the community.”