Share via

What is customer managed key rotation behavior on Ephemeral OS disk?

Gong, Robin 60 Reputation points
2026-04-21T03:07:34.95+00:00

Hi, I'm using the CMK to encrypt the AKS OS disk and data disk, and I met some interesting things that do not match what is described in the Azure document.

I followed https://learn.microsoft.com/en-us/azure/aks/azure-disk-customer-managed-keys to create an AKS with node is ephemeral OS disk and use the Disk Encryption Set(DES) to encrypt the disk using CMK. Besides, I also enabled the "Encryption at host" for the AKS and "Auto rotation" for the DES.

While according to the limitation described in the doc, the rotation on the CMK of the ephemeral OS disk would require nodepool rescale or AKS upgrade:

When encrypting an ephemeral OS disk-enabled node pool with customer-managed keys, if you want to rotate the key in Azure Key Vault, there are two options to consider:

  • Immediate usage of new CMK
    • Scale down the node pool count to 0.
    • Rotate the key.
    • Scale up the node pool to the original count.
  • Gradual usage of new CMK
    • Allow AKS node image upgrades or version upgrades to naturally adopt the new CMK over time.
    • Until all nodes in the pool are upgraded, the existing CMK will continue to function without disruption.
    • Once the upgrade process is complete across all nodes, the new CMK takes effect seamlessly.

But when I actually rotated the key, I saw the DES linked to the new version of the key automatically in about 20 mins, and even though the old key version had expired and I disabled it, the node is still working, and I didn't trigger the nodepool rescale or AKS upgrade. So I wonder if it is because of the "Auto rotation" setting on the DES? Is it the expected behavior? Is the rescaling on the node pool or the AKS upgrade still mandatory when enabling the "Auto rotation" on DES?

Another question is, I used a separate key to only encrypt the pvc data disk of the AKS, when I delete this key, the node to which this data disk was attached becomes unavailable. Is this also an expected behavior?

Azure Kubernetes Service
Azure Kubernetes Service

An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.

Answer accepted by question author
  1. Jilakara Hemalatha 12,660 Reputation points Microsoft External Staff Moderator
    2026-04-23T10:16:08.13+00:00

    Hello Robin,
    Thank you for the detailed explanation and for your patience while we reviewed this with the engineering teams.

    We have checked this scenario with engineering team, and the behavior you observed is expected. When auto-rotation is enabled on the Disk Encryption Set, the DES will automatically update to the latest Key Vault key version after CMK rotation, and this does not require a node pool rescale or an AKS upgrade. The existing nodes continue to function normally, and encryption at host or any internal key caching mechanism does not change this behavior.

    Regarding the documentation reference you mentioned, the rescale or upgrade steps apply in scenarios where auto-rotation is not enabled or when a manual key adoption process is required. In your case, since auto-rotation is enabled, the platform handles the transition seamlessly in the background.

    For the second scenario, where a separate CMK is used for PVC-backed data disks, the behavior you observed is also expected. If the CMK is deleted or becomes unavailable, the associated disk cannot be decrypted, which can lead to the node becoming unavailable. Because of this dependency, it is important to ensure that customer-managed keys used for disk encryption are not deleted or disabled while they are still in use.

    Overall, your observations are aligned with the expected platform behavior.

    Hope this resolved your query. Please let me know if you have any queries.

    1 person found this answer helpful.
    0 comments
Answer accepted by question author
  1. Vinodh247 42,201 Reputation points MVP Volunteer Moderator
    2026-04-23T10:42:02.8733333+00:00

    Hi ,

    Thanks for reaching out to Microsoft Q&A.

    Yes, your understanding can be expressed as a single paragraph:

    When using customer managed keys with an ephemeral OS disk in AKS, enabling auto-rotation on the disk encryption Set allows the DES to automatically reference the new key version without requiring immediate node pool rescaling or upgrades, which is why your nodes continued to run even after the old key was disabled; however, this does not mean the new key is actively used by existing nodes, as they continue operating with the previously established encryption context until they are reimaged through scaling or upgrades, which is still required to fully enforce the new key, whereas for PVC backed data disks, the key must remain continuously accessible, and deleting or disabling it will immediately make the disk unavailable, leading to node or workload failure.

    Please 'Upvote'(Thumbs-up) and 'Accept' as answer if the reply was helpful. This will be benefitting other community members who face the same issue.

    0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Q&A Assist
    2026-04-21T03:07:57.08+00:00
    AI generated content. This question contains an answer created with AI from Q&A Assist. Learn more

    For ephemeral OS disks with customer-managed keys (CMK) in AKS, there are two separate behaviors to distinguish:

    1. Disk Encryption Set (DES) key version rotation behavior
      • For managed disks in general, when using CMK via a Disk Encryption Set, the platform uses envelope encryption: data is encrypted with a data encryption key (DEK), and the DEK is wrapped with the CMK. When the CMK is rotated, the service re-wraps the DEKs with the new key version; the underlying disk data is not re-encrypted. Both old and new key versions must remain enabled until re-wrapping is complete.
      • DES can be configured for auto-rotation: if rotationToLatestKeyVersionEnabled is true, the DES detects the key version change in Key Vault and automatically points to the latest key version. This is why the DES updated to the new key version automatically without any AKS node pool action.
      • This envelope-encryption behavior explains why the node continued to run even after the DES started referencing the new key version. As long as the DEKs have been successfully re-wrapped and the service no longer needs the old key version, the VM/AKS node can continue to function.
    2. Ephemeral OS disk + CMK rotation limitations
      • For ephemeral OS disks, there is an explicit limitation: key rotation is not supported without deleting/recreating the VM. For standalone VMs, documentation states that updating or rotating the CMK for ephemeral OS disks requires deleting the VM and recreating it; key rotation is not supported in-place.
      • For AKS node pools with ephemeral OS disks, the AKS documentation reflects this by requiring either:
        • Scaling the node pool to 0 and back up (immediate adoption of new CMK), or
        • Letting node image/version upgrades gradually recreate nodes (gradual adoption).
      • These requirements are about which CMK version is used when creating new ephemeral OS disks for nodes, not about the DES object itself pointing to a new key version. The DES auto-rotation does not change the fact that ephemeral OS disk VMs must be recreated to be provisioned under a new CMK version. Therefore, rescaling or upgrading remains the supported/mandatory path to ensure nodes are created with the new CMK version for ephemeral OS disks, even if DES auto-rotation is enabled.
    3. Behavior when deleting or disabling keys
      • For managed disks encrypted with CMK, the CMK protects the DEKs. If the CMK (or the specific key version) used to wrap those DEKs is disabled or deleted before re-wrapping completes or while still needed, the platform can no longer unwrap the DEKs, and the disks become inaccessible.
      • This applies equally to data disks, including those backing AKS PVCs. If a separate CMK is used for PVC data disks and that key is deleted, any VM/AKS node that needs to access those disks will fail to read them, and the node or workloads depending on those disks can become unavailable. This is expected behavior for CMK-protected disks.

    In summary:

    • DES auto-rotation explains the automatic move to the new key version and continued node operation, but does not remove the documented requirement to rescale/upgrade AKS node pools with ephemeral OS disks to adopt a new CMK version in a supported way.
    • Deleting the CMK used for PVC data disks causing node unavailability is expected, because the platform can no longer decrypt those disks.

    References:

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.