Azure OpenAI Intermittent 400 Error: Item with id not found

I am periodically getting a 400 invalid_request_error when using a simple agentic tool flow with the responses api using a gpt-4o Global Standard 2024-11-20 model deployment. This happens somewhat rarely, but enough that is is becoming an issue. Specifically, I am getting something that looks like this:

{
  "error": {
	"message": "Item with id 'fc_68cc1e9f1d588190a2e6d1a435025685092272e73adc6b55' not found.",
	"type": "invalid_request_error",
	"param": "input",
	"code": null
  }
}

I send two requests: 1 initial request where the LLM decides to use a tool, and 2 that returns the function_call_output from that tool. I understand the error is telling me the tool call in the payload has an id that is not found. However, this was the id provided by the endpoint in the function_call response object.

Here is the payload that was sent to produce the error. I have omitted the model name and schema link

{
  "model": "...",
  "instructions": "You are a friendly agent that always calls a tool. Output the result of the tool.",
  "input": [
{
  "role": "user",
  "content": "Wonder what the tool call returns"
},
{
  "id": "fc_68cc1e9f1d588190a2e6d1a435025685092272e73adc6b55",
  "type": "function_call",
  "status": "completed",
  "arguments": "{\"input\":\"Hello there!\"}",
  "call_id": "call_tZAqFpxMlsWYMnQ5FSflQmxf",
  "name": "myToolCall"
},
{
  "type": "function_call_output",
  "call_id": "call_tZAqFpxMlsWYMnQ5FSflQmxf",
  "status": "completed",
  "output": "This is some output from our sample tool."
}
  ],
  "include": [],
  "tools": [
{
  "type": "function",
  "name": "myToolCall",
  "description": "Tool to Call",
  "parameters": {
    "type": "object",
    "properties": {
      "input": {
        "type": "string",
        "description": "An input into the tool"
      }
    },
    "required": [
      "input"
    ],
    "additionalProperties": false,
    "$schema": "..."
  },
  "strict": true
}
  ],
  "stream": false
}

To add more confusion, when I re-send this payload, it works and does not give me a 400. This leads me to believe there is some race condition where I am sending the second request with the tool result before the id from the first request is cached or saved on the backend. However, I would expect to not receive a response from the first request until the id is properly created. Putting a sleep between requests seems to solve the issue which furthers my suspicion.

The error does not happen consistently and the only way I can reproduce is by running many of the same agentic flows until I see the error, but I do see it intermittently when running just one flow at a time. Different times of day also seem to affect how often I see the error. Sometimes I am unable to reproduce entirely. I have also seen this error using the typescript Azure OpenAI SDK.

Here is a Python script that I have been using to reproduce the issue:

import requests
import json
import concurrent.futures
import time

API_KEY = "..."
MODEL = "..."
RESOURCE_NAME = "..."

def send(i):
  print(f"Starting {i}")
  querystring = {}

  function_result = {
    "type": "function_call_output",
    "status": "completed",
    "output": "This is some output from our sample tool."
  }

  url = f"https://{RESOURCE_NAME}.openai.azure.com/openai/v1/responses"

  payload = {
    "model": MODEL,
    "instructions": """You are a friendly agent that always calls a tool. Output the result of the tool.""",
    "input": [
      {
        "role": "user",
        "content": "Wonder what the tool call returns"
      }
    ],
    "include": [],
    "tools": [
      {
        "type": "function",
        "name": "myToolCall",
        "description": "Tool to Call",
        "parameters": {
          "type": "object",
          "properties": {
            "input": {
              "type": "string",
              "description": "An input into the tool"
            }
          },
          "required": [
            "input"
          ],
          "additionalProperties": False,
          "$schema": "http://json-schema.org/draft-07/schema#"
        },
        "strict": True
      }
    ],
    "stream": False
  }
  headers = {
    'user-agent': "Agents/JavaScript 0.1.1",
    'content-type': "application/json",
    'accept': "application/json",
    'api-key': API_KEY
  }

  response = requests.request("POST", url, data=json.dumps(payload), headers=headers, params=querystring)
  if not response.ok:
    print(json.dumps(payload, indent=2))
    print(response.text)
    raise AssertionError(response.text)
  response_json = response.json()

  output = response_json['output'][0]
  function_result["call_id"] = output["call_id"]
  payload["input"] = payload["input"] + [output, function_result]

  # time.sleep(5)
  response = requests.request("POST", url, data=json.dumps(payload), headers=headers, params=querystring)
  if not response.ok:
    print(json.dumps(payload, indent=2))
    print(response.text)
    raise AssertionError(response.text)
  response_json = response.json()

def main():
  with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
    future_to_i = {executor.submit(send, i): i for i in range(50)}
    futures = concurrent.futures.wait(future_to_i, return_when=concurrent.futures.FIRST_EXCEPTION)
    executor.shutdown(cancel_futures=True)

if __name__ == '__main__':
  try:
    main()
  except Exception as e:
    print('Error:', e)

How can I resolve this issue?