az hdinsight

Commands

Name	Description	Type	Status
az hdinsight application	Manage HDInsight applications.	Core	GA
az hdinsight application create	Create an application for a HDInsight cluster.	Core	GA
az hdinsight application delete	Deletes the specified application on the HDInsight cluster.	Core	GA
az hdinsight application list	Lists all of the applications for the HDInsight cluster.	Core	GA
az hdinsight application show	Gets properties of the specified application.	Core	GA
az hdinsight application wait	Place the CLI in a waiting state until an operation is complete.	Core	GA
az hdinsight autoscale	Manage HDInsight cluster's Autoscale configuration.	Core	GA
az hdinsight autoscale condition	Manage schedule condition of the HDInsight cluster which enabled Schedule-based Autoscale.	Core	GA
az hdinsight autoscale condition create	Add a new schedule condition.	Core	GA
az hdinsight autoscale condition delete	Delete schedule condition.	Core	GA
az hdinsight autoscale condition list	List all schedule conditions.	Core	GA
az hdinsight autoscale condition update	Update a schedule condition.	Core	GA
az hdinsight autoscale condition wait	Place the CLI in a waiting state until an operation is complete.	Core	GA
az hdinsight autoscale create	Enable Autoscale for a running cluster.	Core	GA
az hdinsight autoscale delete	Disable Autoscale for a running cluster.	Core	GA
az hdinsight autoscale list-timezones	List the available timezone name when enabling Schedule-based Autoscale.	Core	GA
az hdinsight autoscale show	Get the Autoscale configuration of a specified cluster.	Core	GA
az hdinsight autoscale update	Update the Autoscale configuration.	Core	GA
az hdinsight autoscale wait	Place the CLI in a waiting state until an operation is complete.	Core	GA
az hdinsight azure-monitor	Manage Azure Monitor logs integration on an HDInsight cluster.	Core	GA
az hdinsight azure-monitor-agent	Manage Azure Monitor Agent logs integration on an HDInsight cluster.	Core	GA
az hdinsight azure-monitor-agent disable	Disable the Azure Monitor Agent logs integration on an HDInsight cluster.	Core	GA
az hdinsight azure-monitor-agent enable	Enable the Azure Monitor Agent logs integration on an HDInsight cluster.	Core	GA
az hdinsight azure-monitor-agent show	Get the status of Azure Monitor Agent logs integration on an HDInsight cluster.	Core	GA
az hdinsight azure-monitor disable	Disable the Azure Monitor logs integration on an HDInsight cluster.	Core	GA
az hdinsight azure-monitor enable	Enable the Azure Monitor logs integration on an HDInsight cluster.	Core	GA
az hdinsight azure-monitor show	Get the status of Azure Monitor logs integration on an HDInsight cluster.	Core	GA
az hdinsight create	Create a new cluster.	Core	GA
az hdinsight delete	Deletes the specified HDInsight cluster.	Core	GA
az hdinsight host	Manage HDInsight cluster's virtual hosts.	Core	GA
az hdinsight host list	List the hosts of the specified HDInsight cluster.	Core	GA
az hdinsight host restart	Restart the specific hosts of the specified HDInsight cluster.	Core	GA
az hdinsight list	List HDInsight clusters in a resource group or subscription.	Core	GA
az hdinsight list-usage	Lists the usages for the specified location.	Core	GA
az hdinsight monitor	Manage Classic Azure Monitor logs integration on an HDInsight cluster.	Core	GA
az hdinsight monitor disable	Disable the Classic Azure Monitor logs integration on an HDInsight cluster.	Core	GA
az hdinsight monitor enable	Enable the Classic Azure Monitor logs integration on an HDInsight cluster.	Core	GA
az hdinsight monitor show	Get the status of Classic Azure Monitor logs integration on an HDInsight cluster.	Core	GA
az hdinsight resize	Resize the specified HDInsight cluster to the specified size.	Core	GA
az hdinsight rotate-disk-encryption-key	Rotate the disk encryption key of the specified HDInsight cluster.	Core	GA
az hdinsight script-action	Manage HDInsight script actions.	Core	GA
az hdinsight script-action delete	Deletes a specified persisted script action of the cluster.	Core	GA
az hdinsight script-action execute	Execute script actions on the specified HDInsight cluster.	Core	GA
az hdinsight script-action list	Lists all the persisted script actions for the specified cluster.	Core	GA
az hdinsight script-action list-execution-history	Lists all scripts' execution history for the specified cluster.	Core	GA
az hdinsight script-action promote	Promotes the specified ad-hoc script execution to a persisted script.	Core	GA
az hdinsight script-action show-execution-details	Gets the script execution detail for the given script execution ID.	Core	GA
az hdinsight show	Gets the specified cluster.	Core	GA
az hdinsight update	Update the tags or identity of the specified HDInsight cluster. Setting the identity property will override the existing identity configuration of the cluster.	Core	GA
az hdinsight wait	Place the CLI in a waiting state until an operation is complete.	Core	GA

az hdinsight create --name
                    --resource-group
                    --type
                    [--assign-identity]
                    [--autoscale-count]
                    [--autoscale-max-count]
                    [--autoscale-min-count]
                    [--autoscale-type {Load, Schedule}]
                    [--cluster-admin-account]
                    [--cluster-admin-password]
                    [--cluster-configurations]
                    [--cluster-tier {Premium, Standard}]
                    [--cluster-users-group-dns]
                    [--component-version]
                    [--compute-isolation {false, true}]
                    [--days {Friday, Monday, Saturday, Sunday, Thursday, Tuesday, Wednesday}]
                    [--domain]
                    [--edgenode-size]
                    [--enable-private-link {false, true}]
                    [--encryption-algorithm {RSA-OAEP, RSA-OAEP-256, RSA1_5}]
                    [--encryption-at-host {false, true}]
                    [--encryption-in-transit {false, true}]
                    [--encryption-key-name]
                    [--encryption-key-version]
                    [--encryption-vault-uri]
                    [--esp]
                    [--headnode-size]
                    [--host-sku]
                    [--http-password]
                    [--http-user]
                    [--idbroker]
                    [--kafka-client-group-id]
                    [--kafka-client-group-name]
                    [--kafka-management-node-count]
                    [--kafka-management-node-size]
                    [--ldaps-urls]
                    [--location]
                    [--minimal-tls-version {1.2}]
                    [--no-validation-timeout]
                    [--no-wait]
                    [--outbound-dependencies-managed-type {External, Managed}]
                    [--private-link-config]
                    [--public-ip-tag-type]
                    [--public-ip-tag-value]
                    [--resource-provider-connection {Inbound, Outbound}]
                    [--ssh-password]
                    [--ssh-public-key]
                    [--ssh-user]
                    [--storage-account]
                    [--storage-account-key]
                    [--storage-account-managed-identity]
                    [--storage-container]
                    [--storage-filesystem]
                    [--subnet]
                    [--tags]
                    [--time]
                    [--timezone]
                    [--version]
                    [--vnet-name]
                    [--workernode-count]
                    [--workernode-data-disk-size]
                    [--workernode-data-disk-storage-account-type {premium_lrs, standard_lrs}]
                    [--workernode-data-disks-per-node]
                    [--workernode-size]
                    [--zones]
                    [--zookeepernode-size]

Examples

Create a cluster with an existing storage account.

az hdinsight create -t spark -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" \
--storage-account MyStorageAccount

Create a cluster with minimal tls version.

az hdinsight create -t spark -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" \
--storage-account MyStorageAccount --minimal-tls-version 1.2

Create a cluster which enables encryption in transit.

az hdinsight create -t spark -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" \
--storage-account MyStorageAccount --encryption-in-transit true

Create a cluster with encryption at host.

az hdinsight create -t spark -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" \
--storage-account MyStorageAccount --encryption-at-host true

Create a cluster with the Enterprise Security Package (ESP).

az hdinsight create --esp -t spark -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" \
--storage-account MyStorageAccount \
--subnet "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyRG/providers/Microsoft.Network/virtualNetworks/MyVnet/subnets/subnet1" \
--domain "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyRG/providers/Microsoft.AAD/domainServices/MyDomain.onmicrosoft.com" \
--assign-identity "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/MyMsiRG/providers/Microsoft.ManagedIdentity/userAssignedIdentities/MyMSI" \
--cluster-admin-account [email protected] \
--cluster-users-group-dns MyGroup

Create a cluster with the Enterprise Security Package (ESP) and enable HDInsight ID Broker.

az hdinsight create --esp --idbroker -t spark -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" \
--storage-account MyStorageAccount \
--subnet "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyRG/providers/Microsoft.Network/virtualNetworks/MyVnet/subnets/subnet1" \
--domain "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyRG/providers/Microsoft.AAD/domainServices/MyDomain.onmicrosoft.com" \
--assign-identity "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/MyMsiRG/providers/Microsoft.ManagedIdentity/userAssignedIdentities/MyMSI" \
--cluster-admin-account [email protected] \
--cluster-users-group-dns MyGroup

Create a Kafka cluster with disk encryption. See https://docs.microsoft.com/azure/hdinsight/kafka/apache-kafka-byok.

az hdinsight create -t kafka -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" --workernode-data-disks-per-node 2 \
--storage-account MyStorageAccount \
--encryption-key-name kafkaClusterKey \
--encryption-key-version 00000000000000000000000000000000 \
--encryption-vault-uri https://MyKeyVault.vault.azure.net \
--assign-identity MyMSI

Create a kafka cluster with kafka rest proxy.

az hdinsight create -t kafka -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" --workernode-data-disks-per-node 2 \
--storage-account MyStorageAccount \
--kafka-management-node-size "Standard_D4_v2" \
--kafka-client-group-id MySecurityGroupId \
--kafka-client-group-name MySecurityGroupName
--component-version kafka=2.1

Create a cluster with Azure Data Lake Storage Gen2

az hdinsight create -t spark -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" \
--storage-account MyStorageAccount \
--storage-account-managed-identity MyMSI

Create a cluster with configuration from JSON string.

az hdinsight create -t spark -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" \
--storage-account MyStorageAccount \
--cluster-configuration {'gateway':{'restAuthCredential.username':'admin'}}

Create a cluster with configuration from a local file.

az hdinsight create -t spark -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" \
--storage-account MyStorageAccount \
--cluster-configuration @config.json

Create a cluster which Load-based Autoscale.

az hdinsight create -t spark --version 3.6 -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" --storage-account MyStorageAccount \
--autoscale-type Load --autoscale-min-workernode-count 3 --autoscale-max-workernode-count 5

Create a cluster which Schedule-based Autoscale.

az hdinsight create -t spark --version 3.6 -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" --storage-account MyStorageAccount \
--autoscale-type Schedule --timezone "Pacific Standard Time" --days Monday \
--time 09:00 --autoscale-workernode-count 5

Create a cluster with Relay Outbound and Private Link feature.

az hdinsight create -t spark --version 3.6 -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" --storage-account MyStorageAccount \
--subnet "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/fakevnet/subnets/default" \
--resource-provider-connection Outbound --enable-private-link

Create a cluster with Compute Isolation feature.

az hdinsight create -t spark --version 3.6 -g MyResourceGroup -n MyCluster \
-p "HttpPassword1234!" --storage-account MyStorageAccount \
--enable-compute-isolation --workernode-size "Standard_E8S_V3" --headnode-size "Standard_E8S_V3"

Required Parameters

Optional Parameters

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

--output -o

Output format.

Accepted values: json, jsonc, none, table, tsv, yaml, yamlc

Default value: json

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

az hdinsight delete --name
                    --resource-group
                    [--no-wait]
                    [--yes]

Required Parameters

Optional Parameters

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

--output -o

Output format.

Accepted values: json, jsonc, none, table, tsv, yaml, yamlc

Default value: json

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

az hdinsight list [--resource-group]

Optional Parameters

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

--output -o

Output format.

Accepted values: json, jsonc, none, table, tsv, yaml, yamlc

Default value: json

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

az hdinsight list-usage --location

Required Parameters

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

--output -o

Output format.

Accepted values: json, jsonc, none, table, tsv, yaml, yamlc

Default value: json

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

az hdinsight resize --name
                    --resource-group
                    --workernode-count
                    [--no-wait]

Examples

Resize the cluster's workernode.

az hdinsight resize --name MyCluster --resource-group rg --workernode-count 5

Required Parameters

Optional Parameters

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

--output -o

Output format.

Accepted values: json, jsonc, none, table, tsv, yaml, yamlc

Default value: json

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

az hdinsight rotate-disk-encryption-key --encryption-key-name
                                        --encryption-key-version
                                        --encryption-vault-uri
                                        --name
                                        --resource-group
                                        [--no-wait]

Required Parameters

Optional Parameters

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

--output -o

Output format.

Accepted values: json, jsonc, none, table, tsv, yaml, yamlc

Default value: json

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

az hdinsight show --name
                  --resource-group

Required Parameters

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

--output -o

Output format.

Accepted values: json, jsonc, none, table, tsv, yaml, yamlc

Default value: json

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

az hdinsight update --name
                    --resource-group
                    [--assign-identity]
                    [--assign-identity-type]
                    [--no-wait]
                    [--tags]

Examples

Update the tags.

az hdinsight update --name MyCluster --resource-group rg --tags key=value

Update manage identity with single UserAssigned msi.

az hdinsight update --name MyCluster --resource-group rg --assign-identity-type UserAssigned --assign-identity MyMsi

Update manage identity with multiple UserAssigned msi.

az hdinsight update --name MyCluster --resource-group rg --assign-identity-type UserAssigned --assign-identity MyMsi1 MyMsi2

Update SystemAssigned manage identity.

az hdinsight update --name MyCluster --resource-group rg --assign-identity-type SystemAssigned

Update manage identity with SystemAssigned,UserAssigned msi.

az hdinsight update --name MyCluster --resource-group rg --assign-identity-type "SystemAssigned,UserAssigned" --assign-identity MyMsi1

Required Parameters

Optional Parameters

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

--output -o

Output format.

Accepted values: json, jsonc, none, table, tsv, yaml, yamlc

Default value: json

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

az hdinsight wait --name
                  --resource-group
                  [--created]
                  [--custom]
                  [--deleted]
                  [--exists]
                  [--interval]
                  [--timeout]
                  [--updated]

Required Parameters

Optional Parameters

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

--output -o

Output format.

Accepted values: json, jsonc, none, table, tsv, yaml, yamlc

Default value: json

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.