How to resolve high latency for the on-premise Document Intelligence Invoice container?

Hello @Rahul Bhanose ,

Welcome to Microsoft Q&A .Thank you for reaching out to us.

A noticeable increase in latency compared to the cloud service is typically associated with compute configuration, resource allocation, deployment topology, workload characteristics, or container version alignment rather than a functional issue in the service itself.

On-premises containers run entirely on customer-managed infrastructure, while the cloud service operates on fully managed, highly optimized and auto-scaled compute resources. Because of this architectural difference, some latency variation is expected. The primary objective is to identify bottlenecks and optimize the environment.

Please check if the following helps-

1. Container Version Validation The first step is to ensure both Invoice and Layout containers are running supported and compatible images from the official registry. Current documentation indicates:
   • Document Intelligence v4.0 containers are available for Read and Layout models
   • Invoice containers are part of the v3.x GA container family depending on availability and release support
   suggested actions :
   1. Verify the exact image names and tags for both containers
   2. Ensure both containers are pulled from supported GA releases
   3. Avoid using outdated or mismatched container generations
   Please try- docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.ID}}"
2. CPU, Memory, and Throttling Validation One of the most common causes of high latency in on-prem deployments is resource misconfiguration or CPU throttling. Even when the host meets general sizing guidance, performance issues can occur due to:
   • Insufficient CPU or memory allocated per container
   • Kubernetes or Docker CPU throttling (CFS quota enforcement)
   • Oversubscription of compute nodes
   • Use of burstable or shared VM instances
   Suggested checks -
   1. Ensure CPU and memory limits are explicitly defined per container (--cpus, --memory)
   2. Validate no CPU throttling at container/pod level
   3. Confirm compute-optimized VM or node pool is used
   4. Avoid running multiple heavy workloads on shared CPU cores
3. Deployment Topology and Data Path Latency can also be influenced by how containers are deployed and how requests flow through the system. Please validate:
   • Whether Invoice and Layout containers are deployed on the same node or across different nodes
   • Whether requests pass through proxies, firewalls, or load balancers introducing additional network hops
   • Whether temporary processing storage is SSD-backed (recommended) rather than network-mounted storage
   • Whether horizontal scaling is enabled for handling concurrent workloads
   While Layout functionality is part of the Invoice processing pipeline, infrastructure routing and resource contention can still significantly impact performance.
4. Latency Baseline and Workload Validation To accurately compare on-premises and cloud performance, identical test conditions must be used. Recommended metrics:
   • File size
   • Page count
   • Latency per page (key normalization metric)
   • Total processing time (steady-state only)
   • p50 / p95 latency under consistent load
   Please also validate -
   • Whether documents are scanned images or digital PDFs
   • Average page complexity (tables, handwriting, noise)
   • Concurrent request volume during testing
   • Whether slowdown occurs during steady-state or only under peak load

The following references might be helpful , please check them out

• Install and run Docker containers for Document Intelligence - Foundry Tools | Microsoft Learn
• Service quotas and limits - Document Intelligence - Foundry Tools | Microsoft Learn

Thank you