(Resolved - Service seems back) DeepSeek-V3(x) model only works with stream option True, otherwise timeout

Question

(Resolved - Service seems back) DeepSeek-V3(x) model only works with stream option True, otherwise timeout

Xiwei Zhou 0

DeepSeek-V3 and DeepSeek-V3-0324 model stop working on non-streaming request.

Simple use the example shows timeout. Using "stream=True" then works.

import os
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential

endpoint = "https://XXXXXX.services.ai.azure.com/models"
model_name = "DeepSeek-V3-0324"

client = ChatCompletionsClient(
    endpoint=endpoint,
    credential=AzureKeyCredential("<API_KEY>"),
)

response = client.complete(
    messages=[
        SystemMessage(content="You are a helpful assistant."),
        UserMessage(content="I am going to Paris, what should I see?")
    ],
    max_tokens=2048,
    temperature=0.8,
    top_p=0.1,
    presence_penalty=0.0,
    frequency_penalty=0.0,
    model=model_name
)

print(response.choices[0].message.content)

Nam Tran 0 Reputation points

2025-04-23T11:17:28.85+00:00

experiencing the same issue

2 answers

Your answer

Nam Tran 0 Reputation points

2025-04-23T11:17:28.85+00:00

experiencing the same issue

Answer 1

Xiwei Zhou 0

now:

HttpResponseError: Operation returned an invalid status 'Internal Server Error' Content: Model service is unavailable.

JAYA SHANKAR G S 2,180 Reputation points Microsoft External Staff

2025-04-23T11:50:07.1033333+00:00

Hello @Xiwei Zhou ,

Initially I was getting timeout error then later after few minutes the DeepSeek-V3-0324 model works fine and giving outputs in non-stream.

So, please retry once again now and if still the issue persists, please share your apim-request-id in private message which will be in your response header.

Thank you
Xiwei Zhou 0 Reputation points

2025-04-23T14:34:28.8066667+00:00

Thanks, now seems works.

Answer 2

Hello @Xiwei Zhou ,

I am converting the comment to answer so that it will help the community to find better solution.

Issue:
The DeepSeek-V3 and DeepSeek-V3-0324 models were returning timeouts or internal server errors for non-streaming (stream=False) requests. However, streaming requests (stream=True) continued to work as expected.

Solution:
This issue appears to be transient and related to temporary backend unavailability. If you encounter similar errors with non-streaming requests (such as timeouts or internal server errors), please try the following steps:

Retry the request after a short wait, as the service may take time for fully available after deployment.
If the issue persists beyond a reasonable period, capture the apim-request-id from the response headers and share it with Azure support for further investigation.

Please do accept the solution and give feedback by clicking on yes.

Thank you

Share via

(Resolved - Service seems back) DeepSeek-V3(x) model only works with stream option True, otherwise timeout

2 answers

Your answer