Container Instance private IP changed unexpectedly in VNet

Shreyas Waikar (Persistent Systems Inc) 0 Reputation points Microsoft External Staff
2026-05-21T19:04:21.82+00:00

Issue type: Technical
Resource Group: <Pii>
Service: Container Instances (Compute)
Problem type: Networking, IP, and DNS issues
Problem subtype: Container group is unable to communicate with another resource in the same VNet

Summary

We have a customer-impacting outage on a demo environment caused by what we believe is a platform-initiated restart of a VNet-delegated Azure Container Instance that resulted in a private IP reassignment within the delegated subnet.

Symptom

  • App Service (frontend) proxies /api/* to BACKEND_URL = http://10.0.4.4:8000 (the ACI's expected private IP at deploy time).
  • All /api/* requests started returning HTTP 502 after ~2 minutes (consistent with Linux TCP SYN retry timeout) on or around <YYYY-MM-DD HH:MM UTC>.
  • /.auth/me (App Service EasyAuth, no backend hop) remained 200 OK throughout.
  • No customer-initiated restart, redeployment, scale, or config change was performed on the ACI in the preceding 7 days.

What We Need from Microsoft

  1. Confirmation from platform telemetry of any restart, host migration, node replacement, planned/unplanned maintenance, or container group recreation events on this container group between the deploy date and the bug report date.
  2. Exact timestamps of any such event, and the IP held by the container group before and after each event.
  3. IP conflict check — confirmation of whether the IP 10.0.4.4 was held by any other resource in the subnet during that window (i.e., whether a transient conflict forced the new container group instance to allocate a different IP).
Azure Container Instances

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Manish Deshpande 8,055 Reputation points Microsoft External Staff Moderator
    2026-05-27T21:03:34.5466667+00:00

    @Shreyas Waikar (Persistent Systems Inc)

    Thanks for the detailed write-up super helpful! From what you’ve shared, it sounds like Azure probably kicked off a restart (like a host migration or maintenance), which led to your Container Group being recreated and getting a new private IP in the delegated subnet.

    If you’re running Azure Container Instances in a VNet using subnet delegation to Microsoft.ContainerInstance/containerGroups, you won’t get a guaranteed IP address every time the container group restarts or gets recreated by the platform. Basically, whenever the group is taken down and spun up again, it gets treated like a brand-new allocation, so DHCP will just hand out whatever IP is free at that moment.

    1. Use a Static/Reserved Approach via Azure DNS or Internal Load Balancer

    Rather than relying on the raw IP, front your ACI with an Azure Internal Load Balancer (ILB) or use an Azure Private DNS Zone to resolve a stable hostname to the container group. This way, even if the IP changes, the FQDN stays consistent and your App Service proxy can target the DNS name instead of a hardcoded IP.

    https://learn.microsoft.com/en-us/azure/container-instances/container-instances-vnethttps://learn.microsoft.com/en-us/azure/dns/private-dns-overview

    1. Pin the IP via a Static IP Workaround (Application Gateway / NAT)

    Deploy an Azure Application Gateway (internal) or NAT Gateway in front of the ACI. The gateway holds the stable IP; ACI can change behind it without affecting upstream services.

    https://learn.microsoft.com/en-us/azure/application-gateway/overview

    1. Update App Service's BACKEND_URL Dynamically

    As a short-term mitigation, you can use a startup script or Azure Automation Runbook that detects IP changes (via Azure Resource Graph or ARM API) and updates the App Service application setting automatically.

    https://learn.microsoft.com/en-us/azure/governance/resource-graph/overview

    Thanks,
    Manish.

    Was this answer helpful?

    0 comments

  2. AI answer

    2026-05-21T19:04:39.7133333+00:00

    Azure Container Instances (ACI) container groups can be restarted by the platform for maintenance or other reasons, and when this happens the container group’s IP address can change. This behavior is documented and expected for ACI, especially when running in a virtual network.

    From the available information:

    • Container groups may restart due to platform maintenance events, which are performed to improve the underlying infrastructure. These restarts can occur without explicit user action.
    • When a container group restarts, its IP address (private or public) is not guaranteed to remain the same. The documentation explicitly advises against relying on a hard-coded IP address for ACI container groups.
    • For scenarios that require a stable endpoint, the recommended approaches are:
      • Use an Application Gateway in front of the container group to provide a static frontend IP and route traffic to the container group’s current private IP.
      • Use a NAT gateway for outbound connectivity when the container group is deployed into a virtual network, as this is the only supported configuration for outbound traffic from ACI in a VNet.

    Regarding the specific requests:

    1. Platform telemetry confirmation of restart/maintenance events
      Access to platform telemetry (host migration, node replacement, planned/unplanned maintenance, container group recreation) is not exposed through the public documentation or APIs referenced here. For confirmation of such events with exact timestamps, a support request must be opened so that the Azure support team can review internal telemetry for the specific container group and time window.
    2. Exact timestamps and IP before/after events
      Exact event timestamps and the before/after IP assignments for a specific container group are not available in the provided documentation. These details can only be obtained by Azure support through internal logs. The documented guidance is that IP changes are expected when a container group is stopped, started, or restarted.
    3. IP conflict check for 10.0.4.4 in the subnet
      There is no public mechanism in the referenced documentation to query historical IP ownership within a VNet/subnet or to confirm whether a specific private IP was temporarily held by another resource. An IP conflict investigation for a specific address and time window would also require an Azure support case so that internal networking logs can be examined.

    Mitigation and design guidance based on the documentation:

    • Do not hard-code the container group’s IP address in upstream services (such as App Service configuration). Instead, place the container group behind an Application Gateway or similar component that provides a stable endpoint and can be updated if the backend IP changes.
    • Be aware that container groups may restart due to platform maintenance and that their IP addresses may change as a result.
    • If the container group is in a VNet and requires outbound connectivity, ensure a NAT gateway is configured, as this is the only supported configuration for outbound traffic from ACI in a VNet.
    • If the container group stops working or becomes unreachable, follow the ACI troubleshooting guidance first, and then open a support request if the issue appears to be platform-related (for example, suspected platform-initiated restarts causing IP changes).

    For this specific incident (confirmation of restarts, timestamps, and IP conflict analysis), the next step is to open an Azure support request for the affected subscription and container group, referencing the time window and symptoms described, so that support can pull the internal telemetry and provide definitive answers.

    References:

    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.