Share via

Issue with On Demand HDInsight in Hive Activity.

Vishesh Agarwal 0 Reputation points
2025-06-13T06:47:28.4366667+00:00

I have been facing this for a lot of days now, I am trying to use the Hive Activity in ADF to create and On Demand HDInsight Cluster, with no customization, no VNET or subnet. But for some reason I am still facing the issue of the Provisioning Failing with the error:

"statusMessage": "{\"status\":\"Failed\",\"error\":{\"code\":\"ResourceOperationFailure\",\"message\":\"The resource operation completed with terminal provisioning state 'Failed'.\",\"details\":[{\"code\":\"AmbariClusterCreationFailedErrorCode\",\"message\":\"Internal server error occurred while processing the request. Please retry the request or contact support. The following host components failed to start up: OOZIE_SERVER. There is a possibility that this issue occurred due to HDInsight's transition to Standard Load Balancer. Please refer to https://learn.microsoft.com/en-us/azure/hdinsight/load-balancer-migration-guidelines to recreate the cluster with standard load balancers if not done yet, or contact support for assistance if it remains unresolved.\"}]}}",

For short use HDInsight Clusters with no custom requirements, this issue shouldn't occur.

Please help.

Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
231 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Krupal Bandari 770 Reputation points Microsoft External Staff Moderator
    2025-06-13T07:43:12.48+00:00

    Hi @Vishesh Agarwal
    Based on the message you’re seeing during the provisioning of your on-demand HDInsight cluster via the Hive Activity in Azure Data Factory, the failure is related to the OOZIE_SERVER component not starting. This is a known issue that can occur due to HDInsight’s platform-wide transition from Basic to Standard Load Balancers.

    Even if you’re not using any custom configurations, HDInsight now expects the use of Standard Load Balancers as part of the default deployment model. When the older configuration (Basic Load Balancer) is used intentionally or by default it can result in the kind of provisioning failure you're encountering.

    1. Review and apply Microsoft’s official guidelines for using Standard Load Balancers HDInsight Load Balancer Migration Guide
    2. Update your setup to ensure it provisions with a Standard Load Balancer
      • Even in default setups, ensure that the HDInsight version and cluster configuration are compatible with this newer requirement.
      • If you’re using a linked service or any template, double-check that no outdated properties are being used.
    3. Retry the Hive activity in ADF after confirming the above. In the majority of similar cases, this has resolved the issue.
    4. You can also review the deployment logs under Resource Group => Deployments in the Azure portal to see more detailed error output.

    If this is helpful, please click Accept Answer and kindly upvote it so that other people who faces similar issue may get benefitted from it.

    Let me know if you have any further Queries.
    User's image

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.