Attach a Kubernetes cluster to Azure Machine Learning workspace

The following CLI v2 commands show how to attach an AKS and Azure Arc-enabled Kubernetes cluster, and use it as a compute target with managed identity enabled.

AKS cluster

az ml compute attach --resource-group <resource-group-name> --workspace-name <workspace-name> --type Kubernetes --name k8s-compute --resource-id "/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.ContainerService/managedclusters/<cluster-name>" --identity-type SystemAssigned --namespace <Kubernetes namespace to run Azure Machine Learning workloads> --no-wait

Arc Kubernetes cluster

az ml compute attach --resource-group <resource-group-name> --workspace-name <workspace-name> --type Kubernetes --name amlarc-compute --resource-id "/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Kubernetes/connectedClusters/<cluster-name>" --user-assigned-identities "subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<identity-name>" --no-wait

Set the --type argument to Kubernetes. Use the identity_type argument to enable SystemAssigned or UserAssigned managed identities.

Attaching a Kubernetes cluster makes it available to your workspace for training or inferencing.

1. Navigate to Azure Machine Learning studio.

2. Under Manage, select Compute.

3. Select the Kubernetes clusters tab.

4. Select +New > Kubernetes

   

5. Enter a compute name and select your Kubernetes cluster from the dropdown.

   • (Optional) Enter Kubernetes namespace, which defaults to default. All machine learning workloads are sent to the specified Kubernetes namespace in the cluster. Compute attach doesn't create the Kubernetes namespace automatically or validate whether the kubernetes namespace exists. You need to verify that the specified namespace exists in your cluster, otherwise, any Azure Machine Learning workloads submitted to this compute would fail.

   • (Optional) Assign system-assigned or user-assigned managed identity. Managed identities eliminate the need for developers to manage credentials. For more information, see the Assign managed identity section of this article.

   

6. Select Attach

   In the Kubernetes clusters tab, the initial state of your cluster is Creating. When the cluster is successfully attached, the state changes to Succeeded. Otherwise, the state changes to Failed.

   

The following python SDK v2 code shows how to attach an AKS and Azure Arc-enabled Kubernetes cluster, and use it as a compute target with managed identity enabled.

AKS cluster

from azure.ai.ml import load_compute

# for AKS cluster, the resource_id should be something like '/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.ContainerService/managedClusters/<CLUSTER_NAME>''
compute_params = [
    {"name": "<COMPUTE_NAME>"},
    {"type": "kubernetes"},
    {
        "resource_id": "/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.ContainerService/managedClusters/<CLUSTER_NAME>"
    },
]
k8s_compute = load_compute(source=None, params_override=compute_params)
ml_client.begin_create_or_update(k8s_compute).result()

Arc Kubernetes cluster

from azure.ai.ml import load_compute

# for arc connected cluster, the resource_id should be something like '/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.ContainerService/connectedClusters/<CLUSTER_NAME>''
compute_params = [
    {"name": "<COMPUTE_NAME>"},
    {"type": "kubernetes"},
    {
        "resource_id": "/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.ContainerService/connectedClusters/<CLUSTER_NAME>"
    },
]
k8s_compute = load_compute(source=None, params_override=compute_params)
ml_client.begin_create_or_update(k8s_compute).result()