What issue is really being encountered by the Azure Key Vault VM extension on my Azure Arc Server?

Question

What issue is really being encountered by the Azure Key Vault VM extension on my Azure Arc Server?

Thomas Zook 25

When installing the Azure Key Vault VM extension on my Azure Arc Server, it keeps failing with the error message:

Message: Extension Message: Failed to download one or more certificates and retries exhausted. The last error was: Refreshing 'REDACTED' failed with exception: Chain validation failed with critical error: CERT_TRUST_IS_NOT_TIME_VALID

I've independently verified that the Certificate and its chain certificates are valid(both as the certificate is being used elsewhere without issue, such as in a Azure Application Gateway, and I've used "Test-Certificate" on the certificate the extension downloaded and it passed without issue).

I've made sure that the Azure Connected Machine Agent is all the way up to date, the server has external internet access to verify the chain certificates, and I've tried uninstalling and reinstalling the extension without success.

Suchitra Suregaunkar 11,150 Reputation points Microsoft External Staff Moderator

2026-03-23T17:24:42.2566667+00:00

Hello Thomas Zook

Thank you for posting your query on Microsoft Q&A platform.

We are looking into this issue and will keep you posted updates.

Thanks,
Suchitra.
Suchitra Suregaunkar 11,150 Reputation points Microsoft External Staff Moderator

2026-03-23T20:18:10.3666667+00:00
Hello Thomas Zook

Can you confirm the Arc server’s system time, time zone, and NTP source, and verify there is no clock skew from UTC?

Is the Arc server behind a proxy, firewall, or SSL/TLS inspection device that could re‑sign outbound HTTPS traffic to Azure Key Vault?

Does the failure occur for all certificates in the Key Vault, or only for a specific certificate/version?

What OS and exact version is running on the Arc server, and is the root/intermediate CA store fully up to date on the machine?

Does the certificate have a very recent NotBefore timestamp (recent issuance) that could conflict with the server’s local time validation?

Thanks,

Suchitra.
Thomas Zook 25 Reputation points

2026-03-23T21:32:46.0266667+00:00
Hello @Suchitra Suregaunkar

Can you confirm the Arc server’s system time, time zone, and NTP source, and verify there is no clock skew from UTC?

No issue with the Arc Server's system time, time zone, and NTP source. No clock skew identified on the server.

Is the Arc server behind a proxy, firewall, or SSL/TLS inspection device that could re‑sign outbound HTTPS traffic to Azure Key Vault?

It is not behind anything that would re-sign outbound HTTPS traffic to Azure Key Vault.

Does the failure occur for all certificates in the Key Vault, or only for a specific certificate/version?

The Failure occurs for all certificates in the Key Vault.

What OS and exact version is running on the Arc server, and is the root/intermediate CA store fully up to date on the machine?

Windows Server 2025 Standard 24H2 OS Build 26100.32522

Does the certificate have a very recent NotBefore timestamp (recent issuance) that could conflict with the server’s local time validation?

It does not. The latest NotBefore Timestamp is from 4 months ago.
Suchitra Suregaunkar 11,150 Reputation points Microsoft External Staff Moderator

2026-03-24T01:41:16.11+00:00
Hello Thomas Zook

You’re on Windows Server 2025 Standard 24H2, and your error is produced by the extension’s chain validation step.

Version 4.0 is the first one stating Windows Server 2025-specific behavior (KeyGuard private key storage) and explicitly stating it performs chain validation as part of install logic.

Check which KeyVaultForWindows extension version is installed:

In Azure Portal (Arc machine → Extensions) you can see the installed handler version.

Upgrade the extension to TypeHandlerVersion 4.0

For Arc-enabled servers, you typically deploy/update extensions using PowerShell cmdlets like:

New-AzConnectedMachineExtension

Set-AzConnectedMachineExtension

Key point: ensure the handler version is set to 4.0 (not 3.x).

Re-test certificate sync:

After upgrading, the extension will retry downloads and installs multiple times on startup up to 25 retries.

If it still fails, collect the correct logs for further investigation:

The Key Vault extension writes detailed logs under the WindowsAzure plugins log path for the KeyVaultForWindows extension, which is the standard place to validate the chain validation error details.

Please try to Upgrade the Key Vault VM extension to version 4.0 (or later). Version 4.0 explicitly includes Windows Server 2025 support behavior by storing private keys using KeyGuard and using CNG key storage, and it performs chain validation prior to install.

If your Arc machine is currently on extension 3.0, simply having autoUpgradeMinorVersion=true typically won’t jump major versions (3.x → 4.x). So you must explicitly deploy/upgrade to the 4.0 handler version.

Thanks,
Suchitra.
Thomas Zook 25 Reputation points

2026-03-24T17:13:42.3466667+00:00

Hello @Suchitra Suregaunkar

The indication in the portal and on the server showed that the version of the extension was 4.0.333.280, but for the sake of troubleshooting, I uninstalled the extension and re-installed it while specifying the type handler version to be 4.0.

The issue continues to occur, but I did find further logging in C:\ProgramData\GuestConfig\extension_logs\Microsoft.Azure.KeyVault.KeyVaultForWindows and found the following entry:

2026-03-24 09:57:41: <info> [CertificateManager] retrieved '##REDACTED##'; the retrieved version is '##REDACTED##/3af023b7a68e4ddb81a78f0fa57d4bdd'; the previous version is 'b4cc645b89834eaaab593f14d20a4eea'; the ContentType is 'application/x-pkcs12'

2026-03-24 09:57:41: <info> [CertificateManager] Installing previous version 'b4cc645b89834eaaab593f14d20a4eea' of '##REDACTED##'

Why is the extension attempting to install the previous version of the certificate that is expired instead of the current version? I have not specified the version of the certificate in the secrets URL that points to the certificate.
Suchitra Suregaunkar 11,150 Reputation points Microsoft External Staff Moderator

2026-03-25T17:57:05.8966667+00:00
Hello Thomas Zook

Thanks for sharing the additional logs — they’re very helpful.

When the Key Vault VM extension is configured with a versionless secrets URL (for example, https://<vault>.vault.azure.net/secrets/<certName>), the extension always retrieves the latest version first. However, before installing it, the extension performs local certificate chain validation.

If validation of the latest version fails, the extension will fall back to the previously installed version instead of leaving the machine without a certificate. This fallback logic is intentional and is part of the extension’s certificate lifecycle handling.

This behavior aligns with the documented design of the Key Vault VM extension, which monitors certificates, validates them, and only installs versions that pass validation checks:

Azure Key Vault VM extension for Windows https://learn.microsoft.com/azure/virtual-machines/extensions/key-vault-windows

In your log, this sequence is visible:

The extension successfully retrieves the current version of the certificate

Validation fails for that version

The extension then installs the previous version, even though it is expired, as a safety fallback

For you scenario, on Windows Server 2025, the extension uses updated cryptographic handling when validating certificates. Even though the certificate is valid in Key Vault and passes normal Windows validation tools, the extension’s internal validation step can still fail. When that happens, the extension does not consider the new version installable and reverts to the last known version.

This explains why:

You did not pin a certificate version in the URL

The extension still attempts to install an older version

The error surfaces as CERT_TRUST_IS_NOT_TIME_VALID

To prevent the extension from falling back to an expired version, you can take one of the following supported approaches:

Disable or delete the expired certificate version in Key Vault This removes the fallback option, forcing the extension to either install the current version or clearly fail. This is often the simplest way to confirm whether validation of the new version is the root issue.

Temporarily pin the secrets URL to the current certificate version By specifying the full versioned secret URI, you can verify that the extension consistently fails only on the latest version, which helps isolate validation behavior.

Try above two workarounds and let me know if it works.
If not then for further troubleshooting on this issue, I have asked few details via Private message, Please look into it.

Thanks,

Suchitra.
Suchitra Suregaunkar 11,150 Reputation points Microsoft External Staff Moderator

2026-03-26T17:02:04.7866667+00:00

Hello Thomas Zook

Please let us know if the suggested resolution helped/ addressed your concern. If it did, we’d appreciate your feedback on the proposed resolution as it helps others with similar issues.

If the resolution did not help, feel free to share more details about the challenge you’re still facing. Any additional information such as your current issue or specific errors will help us perform a deeper analysis and provide more targeted guidance.

We’re happy to continue assisting and work with you toward the right solution.

Thanks,

Suchitra.
Thomas Zook 25 Reputation points

2026-03-26T20:07:25.1366667+00:00

Hello @Suchitra Suregaunkar

Unfortunately neither of those workarounds worked.

The extension doesn't allow you to specify a specific version URL and requires the non-versioned URL. It will error out if you attempt to provided a versioned URL. Additionally, disabling the other versions of the certificate still causes it to fail, just with an error about not being able to retrieve a disabled certificate version:

[CertificateManager] retrieved 'REDACTED'; the retrieved version is 'REDACTED/3af023b7a68e4ddb81a78f0fa57d4bdd'; the previous version is 'b4cc645b89834eaaab593f14d20a4eea'; the ContentType is 'application/x-pkcs12'

[CertificateManager] Installing previous version 'b4cc645b89834eaaab593f14d20a4eea' of 'REDACTED'

[AuthClient] AcquireTokenCallback invoked

[AuthClient] acquiring token

[CertificateManager] Refreshing 'REDACTED' failed with RequestException: 403; desc: {"error":{"code":"Forbidden","message":"Operation get is not allowed on a disabled secret.","innererror":{"code":"SecretDisabled"}}}

[CertificateManager] Failed to install previous version 'b4cc645b89834eaaab593f14d20a4eea' of 'REDACTED'
Suchitra Suregaunkar 11,150 Reputation points Microsoft External Staff Moderator

2026-03-30T18:43:01.15+00:00

Hello Thomas Zook

Could you please have a look into private message and respond back?

Thanks,
Suchitra.
Thomas Zook 25 Reputation points

2026-03-30T18:54:59.5033333+00:00

Hello @Suchitra Suregaunkar

The private message has been responded to.
Suchitra Suregaunkar 11,150 Reputation points Microsoft External Staff Moderator

2026-03-30T19:20:58.78+00:00

Thanks for responding back. We will check with backend team and keep you posted updates.