Function App deadlock when merge >10k entities to azure data table

Hello Phillip,

Thank you for reaching out to the Microsoft Q&A community. Experiencing a "silent hang" where your Function App stops processing and logging without throwing an explicit error can definitely be frustrating to troubleshoot.

When dealing with ~50,000 sequential inserts to Azure Table Storage via a Node.js Flex Consumption Function, the behavior you are seeing (stalling out around ~10k operations) typically points to one of two common platform constraints:

1. SNAT Port / Socket Exhaustion: Even though you are awaiting operations sequentially, creating tens of thousands of individual HTTP requests (which the Azure Tables REST API relies on under the hood) can exhaust the allocated outbound connections (SNAT ports) for your function's sandbox. When SNAT ports are exhausted, outbound requests simply "hang" waiting for a socket to open, which leads to the silent deadlock behavior you observed.

According to the official documentation on Azure Functions scale and hosting: "Connections might be exhausted... if you make a large number of outbound requests. To avoid this, you should share client instances and reuse connections."

2. Function Execution Timeout: If you execute 50,000 table merges sequentially, and each operation takes even a modest 20-30 milliseconds, the total execution time will exceed 15-25 minutes. In a Flex Consumption plan, functions are subject to strict execution timeouts. If the timeout is reached, the host environment may forcefully shut down the worker process before Application Insights has a chance to flush the final batch of telemetry, resulting in a sudden drop-off in logs.

How to Resolve This Issue:

Your discovery that batching rows in parallel (e.g., using Promise.all for 10 rows at a time) works is actually the recommended best practice for this scenario. Since your entities have different Partition Keys, you cannot use native Azure Table Batch Transactions, but you can control concurrency natively in Node.js.

To ensure stability, I recommend combining concurrency chunking with HTTP keepAlive to reuse sockets:

• Implement Concurrency Control: Continue grouping your promises. Libraries like p-limit or simply chunking your array into groups of 10-50 and yielding them via Promise.all reduces the overall execution time significantly, bypassing the timeout limits.
• Enable keepAlive on the TableClient: Ensure your SDK client reuses TCP connections rather than opening a new socket for every merge.

const { TableClient } = require("@azure/data-tables");
const https = require("https");

// Create a custom HTTPS agent with keepAlive enabled to prevent socket exhaustion
const agent = new https.Agent({ keepAlive: true });

const client = new TableClient(
  "https://<your-account>.table.core.windows.net",
  "tableName",
  credential,
  {
      // Pass the keepAlive agent into the client options
      requestOptions: {
          agent: agent
      }
  }
);

References:

• Managing connections in Azure Functions
• Troubleshoot SNAT port exhaustion

Could you check the configured functionTimeout in your host.json file to see if the silent failure correlates with the maximum execution limit of your Flex Consumption app?

Note: This response is drafted with the help of AI systems.

Thank you! I will give it a try. Does it also work with?

TableClient.fromConnectionString()

I tested it with new TableClient, but the options object does not seem to be right.

const keepAliveAgent = new https.Agent({ keepAlive: true });
const baseHttpClient = createDefaultHttpClient();
const httpClient = {
    sendRequest: (request: any) => {
        request.agent = keepAliveAgent;
        return baseHttpClient.sendRequest(request);
    },
};
const client = new TableClient(
    `https://${accountName}.table.core.windows.net`,
    tableName,
    new DefaultAzureCredential(),
    { httpClient }
);

tried like that, but stopped after 16k, same behaviour

Hello Phillip,

Thanks for the update and for sharing your configuration! Implementing the keepAlive custom HTTP client is a great step, and doing so successfully eliminates SNAT port (socket) exhaustion from the equation.

However, since your function is an HTTP Trigger and it appears you are still iterating through the table merges sequentially (one await at a time), your function has hit a different, hardcoded architectural wall: the Azure Load Balancer timeout.

The 230-Second Limit

Let's do some quick math: If a single Azure Table merge takes roughly 15 milliseconds, processing 16,000 records sequentially will take about 240 seconds (4 minutes). This duration aligns perfectly with a known platform limit.

According to the official Azure Functions scale and hosting documentation:

"Regardless of the function app timeout setting, 230 seconds is the maximum amount of time that an HTTP triggered function can take to respond to a request. This limit exists because of the default idle timeout of Azure Load Balancer."

When your HTTP Trigger runs for more than 230 seconds without returning an HTTP response to the caller, the underlying Azure Load Balancer forcibly drops the connection. The process is terminated abruptly, which perfectly explains why the function simply "stops" at the 16k mark with no errors logged in Application Insights.

The Fix: You Must Implement Chunking

While your keepAlive agent is keeping the single connection healthy, it isn't making the code run fast enough. To safely bypass the 230-second load balancer limit, you must process the rows concurrently in batches using Promise.all(). This will compress the total execution time from several minutes down to just a few seconds.

Here is how you apply concurrency to your existing setup:

const { DefaultAzureCredential } = require("@azure/identity");
const { TableClient } = require("@azure/data-tables");
const { createDefaultHttpClient } = require("@azure/core-rest-pipeline");
const https = require("https");

// 1. Your custom keepAlive logic
const keepAliveAgent = new https.Agent({ keepAlive: true, maxSockets: 50 });
const baseHttpClient = createDefaultHttpClient();
const httpClient = {
    sendRequest: (request) => {
        request.agent = keepAliveAgent;
        return baseHttpClient.sendRequest(request);
    },
};

const client = new TableClient(
    `https://${accountName}.table.core.windows.net`,
    tableName,
    new DefaultAzureCredential(),
    { httpClient }
);

module.exports = async function (context, req) {
    const entitiesToUpload = req.body.entities; // Assuming 50k rows passed in
    
    // 2. Chunking Logic (DO NOT process 1 by 1)
    const chunkSize = 100; // Process 100 concurrently
    
    for (let i = 0; i < entitiesToUpload.length; i += chunkSize) {
        const chunk = entitiesToUpload.slice(i, i + chunkSize);
        
        // Fire 100 HTTP requests in parallel, and await the whole batch
        await Promise.all(chunk.map(entity => client.upsertEntity(entity, "Merge")));
        
        if (i % 5000 === 0) {
            context.log(`Successfully merged ${i} entities...`);
        }
    }
    
    context.res = { body: "All entities merged successfully!" };
}

By adding Promise.all() to fire off 100 requests concurrently, your 50,000 records will finish processing well before the 230-second Azure Load Balancer timeout drops the connection. Note: This code is generated with the help of AI systems.