(Resolved - Service seems back) DeepSeek-V3(x) model only works with stream option True, otherwise timeout

Question

(Resolved - Service seems back) DeepSeek-V3(x) model only works with stream option True, otherwise timeout

Xiwei Zhou 10

DeepSeek-V3 and DeepSeek-V3-0324 model stop working on non-streaming request.

Simple use the example shows timeout. Using "stream=True" then works.

import os
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential

endpoint = "https://XXXXXX.services.ai.azure.com/models"
model_name = "DeepSeek-V3-0324"

client = ChatCompletionsClient(
    endpoint=endpoint,
    credential=AzureKeyCredential("<API_KEY>"),
)

response = client.complete(
    messages=[
        SystemMessage(content="You are a helpful assistant."),
        UserMessage(content="I am going to Paris, what should I see?")
    ],
    max_tokens=2048,
    temperature=0.8,
    top_p=0.1,
    presence_penalty=0.0,
    frequency_penalty=0.0,
    model=model_name
)

print(response.choices[0].message.content)

Xiwei Zhou 10 Reputation points

2025-04-22T23:30:30.9266667+00:00

now:

HttpResponseError: Operation returned an invalid status 'Internal Server Error' Content: Model service is unavailable.
Nam Tran 0 Reputation points

2025-04-23T11:17:28.85+00:00

experiencing the same issue
JAYA SHANKAR G S 3,960 Reputation points Microsoft External Staff Moderator

2025-04-23T11:50:07.1033333+00:00

Hello @Xiwei Zhou ,

Initially I was getting timeout error then later after few minutes the DeepSeek-V3-0324 model works fine and giving outputs in non-stream.

So, please retry once again now and if still the issue persists, please share your apim-request-id in private message which will be in your response header.

Thank you
Xiwei Zhou 10 Reputation points

2025-04-23T14:34:28.8066667+00:00

Thanks, now seems works.

1 answer

Your answer

Xiwei Zhou 10 Reputation points

2025-04-22T23:30:30.9266667+00:00

now:

HttpResponseError: Operation returned an invalid status 'Internal Server Error' Content: Model service is unavailable.
Nam Tran 0 Reputation points

2025-04-23T11:17:28.85+00:00

experiencing the same issue
JAYA SHANKAR G S 3,960 Reputation points Microsoft External Staff Moderator

2025-04-23T11:50:07.1033333+00:00

Hello @Xiwei Zhou ,

Initially I was getting timeout error then later after few minutes the DeepSeek-V3-0324 model works fine and giving outputs in non-stream.

So, please retry once again now and if still the issue persists, please share your apim-request-id in private message which will be in your response header.

Thank you
Xiwei Zhou 10 Reputation points

2025-04-23T14:34:28.8066667+00:00

Thanks, now seems works.

Answer 1

JAYA SHANKAR G S 3,960 Microsoft External Staff Moderator

Hello @Xiwei Zhou ,

I am converting the comment to answer so that it will help the community to find better solution.

Issue:
The DeepSeek-V3 and DeepSeek-V3-0324 models were returning timeouts or internal server errors for non-streaming (stream=False) requests. However, streaming requests (stream=True) continued to work as expected.

Solution:
This issue appears to be transient and related to temporary backend unavailability. If you encounter similar errors with non-streaming requests (such as timeouts or internal server errors), please try the following steps:

Retry the request after a short wait, as the service may take time for fully available after deployment.
If the issue persists beyond a reasonable period, capture the apim-request-id from the response headers and share it with Azure support for further investigation.

Please do accept the solution and give feedback by clicking on yes.

Thank you

JAYA SHANKAR G S 3,960 Reputation points Microsoft External Staff Moderator

2025-04-25T05:54:37.1433333+00:00

Hello @Xiwei Zhou ,

Please check above and do accept the solution. Let us know if you have any further query.

Thank you
Xiwei Zhou 10 Reputation points

2025-04-25T05:57:53.0733333+00:00

I feel there is a service reliability issue, as it was down again for couple minutes today, can you add a link to share the information to Azure support a bit more detail. Thanks. @JAYA SHANKAR G S
JAYA SHANKAR G S 3,960 Reputation points Microsoft External Staff Moderator

2025-04-25T08:50:52.0866667+00:00

Hello @Xiwei Zhou ,

Will check with internal team if there is any ongoing issue and update you with details.

Thank you
JAYA SHANKAR G S 3,960 Reputation points Microsoft External Staff Moderator

2025-04-29T09:21:47.92+00:00

Hello @Xiwei Zhou ,

There was no ongoing issue related to timeouts, are you still facing the issue? If yes, please provide the details asked in private chat.

Thank you

Share via

(Resolved - Service seems back) DeepSeek-V3(x) model only works with stream option True, otherwise timeout

1 answer

Your answer