Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
1 answer
Unable to create a new hdi cluster with Internal server error
Got the error below but I've validated that the load balancer settings are already in place. Showing 1 out of 1 error(s).Status Message: Internal server error occurred while processing the request. Please retry the request or contact support. The…
Azure HDInsight
asked 2026-03-25T23:14:01.1866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 72%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHC%3C/text%3E%3C/svg%3E)
Hanyuan Cui
0
Reputation points Microsoft Employee
answered 2026-03-29T02:01:00.2666667+00:00
Yutaka_K_JP
1,570
Reputation points
0 answers
Outbound ports to HDInsight management
Hello, What are the required ports for outbound connection of HDInsight to management IPs and any other needed Azure services when configured with a resource provider connection set to ‘Outbound’? …
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2021-03-23T14:03:02.817+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 53%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAB%3C/text%3E%3C/svg%3E)
Alex Boata
21
Reputation points
commented 2026-03-02T10:50:05.7+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 67%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAM%3C/text%3E%3C/svg%3E)
Akshita Mittal
0
Reputation points
2 answers
HDInsights quota help request
when trying to get quota increases for HDInsights, all are 'ineligible' for increase - why?
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2026-02-18T21:38:58.92+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 88%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EUT%3C/text%3E%3C/svg%3E)
Urbach, Tara
0
Reputation points
commented 2026-02-19T20:44:50.92+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 2%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMK%3C/text%3E%3C/svg%3E)
Manoj Kumar Boyini
11,200
Reputation points Microsoft External Staff
Moderator
2 answers
One of the answers was accepted by the question author.
Unable to launch HDInsight (5.1, Hadoop 3.3.4) cluster
I am unable to create an HDInsight 5.1 cluster in the US EAST 2 region. The deployment fails with: Error Code: FailedToPrepareClusterDocumentsErrorCode Tried other regions (West Europe, US EAST) as well, with same results. I double checked quota in my…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2026-01-14T10:28:04.22+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 79%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKK%3C/text%3E%3C/svg%3E)
Karel Kolman
20
Reputation points
recommended 2026-01-24T12:46:17.8766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 40%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAT%3C/text%3E%3C/svg%3E)
Alexandra Tanasescu
0
Reputation points
Moderator
2 answers
HBase Accelerated Writes in HDInsight, Failure Recovery
I'm trying to understand the durability and performance trade-offs of enabling HBase Accelerated Writes in HDInsight. Specifically What happens if the local disk copy fails or becomes corrupted before the asynchronous sync to ADLS Gen2 occurs? Is…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-11-26T10:14:59.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 61%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
prathap sagar
0
Reputation points
commented 2025-12-08T06:47:34.14+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 8%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anonymous
1 answer
Can we use the same Vnet and a new subnet for deploying new Hbase as a part of Azure basic to standard load balancer?
https://learn.microsoft.com/en-us/azure/hdinsight/load-balancer-migration-guidelines Can we use the same Vnet and a new subnet for deploying new Hbase as a part of Azure basic to standard load balancer?
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-08-12T09:27:04.7866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 78%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Mall Ashi
5
Reputation points
commented 2025-08-18T05:27:47.9366667+00:00
Anonymous
1 answer
One of the answers was accepted by the question author.
Connecting to an HDInsight Kafka Cluster from a Local Machine via Python Script
Hello, I've configured my Kafka cluster on Azure HDInsight with a virtual network (VNet) that includes a NAT gateway and a Network Security Group (NSG). Now, I’m trying to connect to this cluster from my local Windows machine using a Python script to…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-07-01T12:55:51.3266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 93%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EOH%3C/text%3E%3C/svg%3E)
Ons Hanafi
20
Reputation points
accepted 2025-07-03T10:01:04.6166667+00:00
Ons Hanafi
20
Reputation points
1 answer
No response to HDInsight quota request – SR# 2506200030006435 (Southeast Asia)
Hi Microsoft Team, I had submitted a quota increase request to enable 10 cores for HDInsight cluster in Southeast Asia under Pay-As-You-Go subscription. S: 2506200030006435 . Raised on: June 20, 2025 . Subscription: ******@gmail.com I received a…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-06-24T06:14:16.2233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 43%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVM%3C/text%3E%3C/svg%3E)
Vinaykumar Marali
20
Reputation points
answered 2025-07-01T04:32:56.45+00:00
Vinaykumar Marali
20
Reputation points
1 answer
One of the answers was accepted by the question author.
Quota approved but still unable to create HDInsight cluster — no available cores
Hi Team, I raised a request for HDInsight core quota increase and got confirmation via email that the quota was approved for AV2 and Dads v5 families. However, I am still unable to create even the minimum configuration HDInsight cluster. I tried…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-06-25T07:15:21.57+00:00
Vinaykumar Marali
20
Reputation points
accepted 2025-06-27T10:57:25.7433333+00:00
Vinaykumar Marali
20
Reputation points
1 answer
NullPointerException in RegionSizeCalculator when using Hive Warehouse Connector on HDInsight (5.1) / Azure Spark
Hello, I'm encountering a NullPointerException when running a Spark job on Azure HDInsight (Spark with Hive Warehouse Connector). The job fails during input split generation from an HBase table. The relevant portion of the log is: 25/06/01 06:28:18 ERROR…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-06-23T13:43:11.2466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 96%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EET%3C/text%3E%3C/svg%3E)
ECSEKI Tibor
5
Reputation points
commented 2025-06-25T19:00:30.41+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
15,470
Reputation points Microsoft External Staff
Moderator
1 answer
Issue with On Demand HDInsight in Hive Activity.
I have been facing this for a lot of days now, I am trying to use the Hive Activity in ADF to create and On Demand HDInsight Cluster, with no customization, no VNET or subnet. But for some reason I am still facing the issue of the Provisioning Failing…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-06-13T06:47:28.4366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(169.60000000000002, 46%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVA%3C/text%3E%3C/svg%3E)
Vishesh Agarwal
0
Reputation points
edited an answer 2025-06-17T05:16:31.6233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 80%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKB%3C/text%3E%3C/svg%3E)
Krupal Bandari
780
Reputation points
Moderator
1 answer
HD Insights não me disponibiliza nós para começar um cluster
Não consigo criar nenhum nó no HDInsights. Nenhuma região possui nós disponíveis. Preciso de cotas
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-06-03T21:08:50.81+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(48, 64%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGB%3C/text%3E%3C/svg%3E)
Gustavo Bonifácio Pereira Guida
0
Reputation points
commented 2025-06-05T17:09:33.4633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
7,235
Reputation points
Moderator
1 answer
HDInsight cluster creation Fails
{ "code": "InvalidNetworkConfigurationErrorCode", "message": "Virtual Network configuration is incompatible with HDInsight requirements. Error: 'Cluster Validation timed out.', Please follow…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-05-01T18:57:29.3733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 96%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKK%3C/text%3E%3C/svg%3E)
Kavin Kingslin
0
Reputation points
answered 2025-05-02T07:29:16.65+00:00
Krupal Bandari
780
Reputation points
Moderator
2 answers
Unable to delete HDInsight cluster and resource group.
Unable to delete HDInsight cluster and resource group. I am getting and error and the deletion process keeps running indefinitely.
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-04-17T20:44:02.2033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 24%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
Abhinandan Gupta
0
Reputation points
answered 2025-05-01T11:15:49.28+00:00
Anonymous
1 answer
I want to delete my HDInsight cluster
I cannot ssh into the HD cluster, and Ambari UI is not loading either. The cluster has the status deallocated. I want to delete the cluster and create a new one
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-03-25T21:29:32.3133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 5%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIM%3C/text%3E%3C/svg%3E)
Isaac McSey
0
Reputation points
commented 2025-03-27T19:07:47.3033333+00:00
Chandra Boorla
15,470
Reputation points Microsoft External Staff
Moderator
1 answer
How to Disable or Remove Hive Warehouse Connector from HDInsight 5.1
Hello, We are running Spark on HDInsight 5.1 and are not using Hive Metastore at all. However, our Spark application is failing with the following exception: User class threw exception: java.lang.NoClassDefFoundError:…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-03-12T11:51:57.01+00:00
ECSEKI Tibor
5
Reputation points
commented 2025-03-14T07:33:53.6866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
34,640
Reputation points Microsoft External Staff
Moderator
1 answer
Azure HDInsight Cluster Not getting Deleted
Hi !!! I have a HDInsight Cluster which I am not able to delete. It is running for more than 30 mins. The name of the cluster is "sayakban-cc-spark-cluster". This is a high priority task as it is consuming my credits from my educational…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-03-03T23:24:43.18+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 97%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Sayak Banerjee
0
Reputation points
edited an answer 2025-03-12T06:47:17.5+00:00
Ganesh Gurram
7,235
Reputation points
Moderator
0 answers
HDInsight Spark-type cluster failing to be deployed due to failed start up of SPARK3_THRIFTSERVER service
In Microsoft Azure the Spark-type HDInsight cluster (with 5.1 stack version) is failing to be deployed with an AmbariClusterCreationFailedErrorCode error-code due to an internal error of Spark3 hriftserver service on Ambari: Execute['!…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-02-24T15:42:41.6233333+00:00
Tibor, Ecseki
0
Reputation points
commented 2025-02-27T12:38:37.71+00:00
Chandra Boorla
15,470
Reputation points Microsoft External Staff
Moderator
1 answer
Creating Azure HDInsight process is stuck at "Status: HDInsight Configuration"
Hey, During the creation of Azure HDinsight (Hadoop Cluster) The process seems to be stuck at "Status: HDInsight Configuration" For more than 20 minutes; after it did manage to somewhat create it (note the dashboard itself is up and running),…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-02-13T09:40:47.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 42%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
lidorbenshitrit
0
Reputation points
commented 2025-02-17T11:53:28.9766667+00:00
Ganesh Gurram
7,235
Reputation points
Moderator
2 answers
Issue Creating HDInsight Cluster with Virtual Network
I am trying to create an HDInsight cluster using Terraform. However, I encounter the following error when connecting it to a virtual network: │ Error: creating Hadoop HDInsight Cluster (Subscription: "XXX" │ Resource Group Name:…
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
236 questions
asked 2025-01-28T18:41:16.3233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 13%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEJ%3C/text%3E%3C/svg%3E)
Emily Jil Sauer
0
Reputation points
commented 2025-02-17T09:40:38.7333333+00:00
AnnuKumari-MSFT
34,566
Reputation points Microsoft Employee
Moderator