GPT-5.4 (Responses API) rejects dense prompts above 272k tokens with context_length_exceeded, but accepts English prompts up to 1M

Question

GPT-5.4 (Responses API) rejects dense prompts above 272k tokens with context_length_exceeded, but accepts English prompts up to 1M

Chris Cook 0

Environment

Model: gpt-5.4, Azure AI Foundry, region Sweden Central, Responses API (/openai/v1/responses), streaming
reasoning.effort: none, small max_output_tokens

Problem

Some requests above 272k input tokens fail with:

{"type":"error","error":{"type":"invalid_request_error","code":"context_length_exceeded","message":"Your input exceeds the context window of this model. Please adjust your input and try again.","param":"input"},"sequence_number":2}

The error arrives inside the HTTP 200 stream, after response.created and response.in_progress. Whether a request fails does not depend on the token count. It depends on how many characters the prompt has per token.

Synthetic prompts, same deployment, token counts from the OpenAI token-count endpoint:

Prompt	Tokens	Characters	Chars/token	Chars ÷ 4	Result
English	285,008	1,725,828	6.06	431,457	accepted
Dense ASCII	285,018	424,223	1.49	106,056	context_length_exceeded
Japanese	284,996	471,727	1.66	117,932	context_length_exceeded
Chinese	285,004	393,590	1.38	98,398	context_length_exceeded
Japanese	699,986	1,158,607	1.66	289,652	accepted

A Japanese prompt of 285k tokens is rejected, but a Japanese prompt of 700k tokens is accepted. In production we also see English prompts accepted up to ~536k tokens on the same deployment. The same rejected prompt is accepted by OpenAI's API directly.

All results agree with this: the service estimates the prompt size as characters ÷ 4 to choose between a backend with a 272k limit and a long-context backend. Text with few characters per token (CJK, dense data) is estimated too low, goes to the 272k backend, and is rejected there when the real token count is above 272k.

Reproduction script (Node 18+, no dependencies, dry run by default): https://gist.github.com/zirkelc/7222464315271b3f43cf6f9488e80b2f

Questions

Is this a known issue? Is a fix planned?
Is there a supported way to make sure a request goes to the long-context backend (a header, a deployment type or a setting)?

A similar issue for gpt-4.1 (context_length_exceeded at ~300k tokens) was fixed in July 2025: https://learn.microsoft.com/en-us/answers/questions/2280883/

Suhail Ahamed J 0 Reputation points Microsoft External Staff Moderator

2026-09-29T14:02:35.5733333+00:00
Hello @Chris Cook ,
Greetings,

This issue appears to stem from the behavior of the Azure OpenAI Service’s routing mechanism, where requests are assigned to different backends based on the estimated prompt size (calculated as characters ÷ 4). Dense character prompts. such as those in Japanese, Chinese, or dense ASCII- result in a lower estimate of prompt size than their actual token count, causing the request to be routed to a backend with a stricter context length limit. Below is a detailed analysis paired with suggestions:

Root Cause Analysis:

Prompt Size Estimation Behavior:

The backend routes requests based on the "character count ÷ 4" heuristic.

This method works well for English prompts but underestimates context size for languages with low characters-per-token ratios (e.g., CJK languages or dense ASCII data). When the prompt exceeds the real token count limit (e.g., 272k), the request fails with the context_length_exceeded error.

Behavior of Long-Context Backend:

Prompts that are properly estimated (e.g., using English or some Japanese prompts) are routed to the long-context backend and can handle up to ~1M tokens.

Dense prompts mistakenly routed to the standard backend fail when the true token count exceeds its limit.

Answers to your queries:
Is this a known issue? Is a fix planned?
Yes, this behavior has been observed and reported previously, particularly for deployments of GPT models in Azure OpenAI Service with dense character prompts. It aligns with a similar issue for gpt-4.1 in July 2025 (Microsoft documentation reference ).

While a specific fix isn’t officially documented for gpt-5.4 yet, I recommend tracking updates via the Azure OpenAI Service release notes and your deployed Azure region for potential fixes.

Known workarounds are discussed below to mitigate the issue for now.

Is there a supported way to enforce routing to the long-context backend?
Currently, there is no documented method, such as an API header or specific setting, to explicitly route requests to the long-context backend. Routing decisions remain automatic based on prompt sizing heuristics.
To work around this limitation:
1) Pre-processing workaround:

Pad dense character prompts (e.g., adding English text headers or sections) to manipulate the prompt’s "character count ÷ 4" estimate upward. This forces better backend routing.

Example: Add a brief sentence/metadata in English to the prompt.

This padding does not add significant tokens but adjusts character estimation for routing.

2)Token Count API Validation:
Utilize the /v1/tokenizer endpoint to predict whether your prompt will exceed the backend limit before sending the request.

If the token count approaches the 272k threshold, reformat the prompt for better backend routing or truncate excess data.

3) Split Large Prompts:
Break prompts into smaller chunks (below the 272k threshold) and send them as sequential requests. Aggregate responses later in your application logic to recompose the complete output.

Next Steps:

For long-term stability:

Raise a support case via the Azure Portal so the Azure product team is aware of how this impacts your use case and requests for improvements.

Stay updated with the Azure AI Foundry release notes: Azure OpenAI documentation .

Recommendations for Immediate Action:
To avoid prompt rejection during production calls:

Use gpt-5.4's tokenizer API to predict token count and assess routing risks.

Pad dense character prompts with lightweight English headers to improve routing heuristics.

Consider splitting overly dense prompts across smaller request sizes or restructuring content formats to avoid backend constraints.

Thanks,
Suhail Ahamed J.

1 answer

Your answer

Answer 1

Based on your findings, it doesn't look like a GPT-5.4 context-window limitation but an Azure-side routing bug.

It looks like the correlation between your chars/4 is a solid indication that Azure's routing and size estimation might be mistakenly relying on character-based estimation for certain inputs. To address this, the best course of action would be to open a support ticket with Azure. Be sure to include your reproduction steps, token-count measurements, and request IDs, while clearly outlining the suspected chars/4 routing issue.

References:

https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure

https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/provisioned-throughput-sizing