Quickstart: Create an Azure Kubernetes Fleet Manager and join member clusters using Azure CLI

Applies to: ✔️ Fleet Manager ✔️ Fleet Manager with hub cluster

Get started with Azure Kubernetes Fleet Manager by using the Azure CLI to create a Fleet Manager and join supported Kubernetes clusters as members.

Fleet Manager supports an optional hub cluster that is used for intelligent Kubernetes resource placement and is required to use Managed Fleet Namespaces.

For more information on Fleet Manager configurations, see the conceptual overview of fleet types.

Before you begin

If you don't have an Azure account, create a free account before you begin.

• Read the conceptual overview of Fleet Manager, which provides an explanation of fleets and member clusters referenced in this document.

• Read the conceptual overview of fleet types, which provides a comparison of different fleet configuration options.

• An Azure account with an active subscription. Create an account for free.

• An identity (user or service principal) with the following permissions:

  • Fleet Manager:
    • Microsoft.ContainerService/fleets/read
    • Microsoft.ContainerService/fleets/write
    • Microsoft.ContainerService/fleets/members/read
    • Microsoft.ContainerService/fleets/members/write
    • Microsoft.ContainerService/fleetMemberships/read
    • Microsoft.ContainerService/fleetMemberships/write
  • AKS member clusters:
    • Microsoft.ContainerService/managedClusters/read
    • Microsoft.ContainerService/managedClusters/write
    • Microsoft.ContainerService/managedClusters/listClusterUserCredential/action
  • Arc-enabled Kubernetes member clusters:
    • Microsoft.Kubernetes/connectedClusters/read
    • Microsoft.KubernetesConfiguration/extensions/read
    • Microsoft.KubernetesConfiguration/extensions/write
    • Microsoft.KubernetesConfiguration/extensions/delete

• Have the Azure CLI version 2.82.0 or later installed. To install or upgrade, see Install the Azure CLI.

• You also need the fleet Azure CLI extension version 1.8.3 or later, which you can install by running the following command:

  az extension add --name fleet
  

  Run the following command to update to the latest version of the extension released:

  az extension update --name fleet
  

• Set the following environment variables:

  export SUBSCRIPTION_ID=<subscription_id>
  export GROUP=<resource_group_name>
  export FLEET=<fleet_name>
  export LOCATION=<azure_region_name>
  

• Install kubectl using the az aks install-cli command.

  az aks install-cli
  

• Member clusters must run supported versions: see AKS cluster version support policy and Azure Arc-enabled Kubernetes validation.

Create a resource group

Set the Azure subscription and create a resource group using the az group create command.

az account set -s ${SUBSCRIPTION_ID}
az group create --name ${GROUP} --location ${LOCATION}

az fleet create \
    --resource-group ${GROUP} \
    --name ${FLEET} \
    --location ${LOCATION} \
    --enable-managed-identity

az fleet create \
    --resource-group ${GROUP} \
    --name ${FLEET} \
    --location ${LOCATION}  \
    --enable-hub \
    --enable-managed-identity

export SUBSCRIPTION_ID=<subscription_id>
export GROUP=<resource_group_name>
export FLEET=<fleet_name>
export LOCATION=<azure_region_name>
export UAMI-NAME=<user_managed_identity>
export VNET-NAME=<virtual_network_name>
export VNET-CLUSTER-SUBNET-NAME=<vnet_cluster_subnet_name>
export VNET-API-SUBNET-NAME=<vnet_api_subnet_name>

az identity create \
    --resource-group ${GROUP} \
    --name ${UAMI-NAME}

UAMI_ID=$(az identity show --resource-group ${GROUP} --name ${UAMI-NAME} --query id --output tsv)

az network vnet create \
    --resource-group ${GROUP} \
    --name ${FLEET} \
    --address-prefixes 192.168.0.0/16

az network vnet subnet create \
    --resource-group ${GROUP} \
    --vnet-name ${FLEET} \
    --name ${VNET-CLUSTER-SUBNET-NAME} \
    --address-prefixes 192.168.0.0/23

CLUSTER_SUBNET_ID=$(az network vnet subnet show --resource-group ${GROUP} --vnet-name ${FLEET} -n ${VNET-CLUSTER-SUBNET-NAME} -o tsv --query id)

az network vnet subnet create \
    --resource-group ${GROUP} \
    --vnet-name ${FLEET} \
    --name ${VNET-API-SUBNET-NAME} \
    --delegations "Microsoft.ContainerService/managedClusters" \
    --address-prefixes 192.168.2.0/23

API_SUBNET_ID=$(az network vnet subnet show --resource-group ${GROUP} --vnet-name ${FLEET} -n ${VNET-API-SUBNET-NAME} -o tsv --query id)

 az fleet create \
    --resource-group ${GROUP} \
    --name ${FLEET} \
    --location ${LOCATION}  \
    --enable-hub \
    --enable-managed-identity \
    --agent-subnet-id ${CLUSTER_SUBNET_ID} \
    --apiserver-subnet-id ${API_SUBNET_ID} \
    --assign-identity ${UAMI_ID}

Join member clusters

Fleet Manager supports joining existing AKS clusters or Arc-enabled Kubernetes clusters (Preview) as member clusters.

1. Set the following environment variables for member clusters:

   export MEMBER_NAME_1=flt-member-cluster-1
   
   # For an AKS cluster
   export MEMBER_CLUSTER_ID_1=/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${GROUP}/providers/Microsoft.ContainerService/managedClusters/${MEMBER_NAME_1}
   
   # For an Arc-enabled cluster
   export MEMBER_CLUSTER_ID_1=/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${GROUP}/providers/Microsoft.Kubernetes/connectedClusters/${MEMBER_NAME_1}
   

2. Join the existing clusters to the Fleet Manager using the az fleet member create command.

   az fleet member create \
       --resource-group ${GROUP} \
       --fleet-name ${FLEET} \
       --name ${MEMBER_NAME_1} \
       --member-cluster-id ${MEMBER_CLUSTER_ID_1}
   

   Your output should look similar to the following example output:

   {
     "clusterResourceId": "/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<GROUP>/providers/Microsoft.ContainerService/managedClusters/aks-member-x",
     "etag": "...",
     "id": "/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<GROUP>/providers/Microsoft.ContainerService/fleets/<FLEET>/members/aks-member-x",
     "name": "aks-member-1",
     "provisioningState": "Succeeded",
     "resourceGroup": "<GROUP>",
     "systemData": {
       "createdAt": "2022-10-04T19:04:56.455813+00:00",
       "createdBy": "<user>",
       "createdByType": "User",
       "lastModifiedAt": "2022-10-04T19:04:56.455813+00:00",
       "lastModifiedBy": "<user>",
       "lastModifiedByType": "User"
     },
     "type": "Microsoft.ContainerService/fleets/members"
   }
   

3. Verify that the member clusters successfully joined the Fleet Manager using the az fleet member list command.

   az fleet member list \
       --resource-group ${GROUP} \
       --fleet-name ${FLEET} \
       -o table
   

   If successful, your output should look similar to the following example output:

   ClusterResourceId                                                                                                          Name          ProvisioningState    ResourceGroup
   -------------------------------------------------------------------------------------------------------------------------- ------------  -------------------  ---------------
   /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<GROUP>/providers/Microsoft.ContainerService/managedClusters/aks-member-1  aks-member-1  Succeeded            <GROUP>
   /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<GROUP>/providers/Microsoft.ContainerService/managedClusters/aks-member-2  aks-member-2  Succeeded            <GROUP>
   /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<GROUP>/providers/Microsoft.ContainerService/managedClusters/aks-member-3  aks-member-3  Succeeded            <GROUP>
   

Next steps

• Access Fleet Manager hub cluster Kubernetes API.
• Deploy cluster-scoped resources across multiple clusters.
• Deploy namespace-scoped resources across multiple clusters.
• Create and configure Managed Fleet Namespaces.
• How-to: Upgrade multiple clusters using Azure Kubernetes Fleet Manager update runs.