An Apache Spark-based analytics platform optimized for Azure.
Hello zybertech1sql,
Welcome to the Microsoft Q&A and thank you for posting your questions here.
I understand that you would like to disable Serverless Compute in our Azure Databricks workspace to prevent unexpected compute charges.
Serverless Compute is available by default in many Unity Catalog-enabled Azure Databricks workspaces, and several Databricks services such as Notebooks, Jobs, SQL Warehouses, Lakeflow Pipelines, and Model Serving can independently consume Serverless resources. As a result, traditional cluster policies and compute governance controls may not fully prevent Serverless usage. - https://learn.microsoft.com/en-us/azure/databricks/compute/serverless/, and https://docs.azure.cn/en-us/databricks/compute/serverless/
What you need to do is to: - https://community.databricks.com/t5/administration-architecture/how-do-you-disable-serverless-interactive-compute-for-all-users/td-p/102880
- Identify the exact Serverless workload generating charges using Databricks Usage and Billing data.
- Remove or migrate all Serverless SQL Warehouses.
- Convert Serverless Jobs to customer-managed clusters.
- Review Notebook compute selections.
- Remove unused Model Serving Endpoints.
- Review Lakeflow Pipelines configured for Serverless execution.
- Restrict user permissions for creating Warehouses, Jobs, Pipelines, and Endpoints.
- Implement Budget Policies for ongoing visibility and accountability.
- If no Serverless disable option exists in the workspace, engage Databricks Support via your immediate Portal to verify whether Serverless can be disabled at the backend for that workspace or account. - https://learn.microsoft.com/en-us/azure/databricks/compute/serverless/, and https://learn.microsoft.com/en-us/azure/databricks/compute/serverless/dependencies gives more insight.
After identifying the actual Serverless consumer and removing all Serverless-enabled resources, future Serverless usage can be minimized. However, if the workspace is provisioned with Serverless enabled by platform design, Databricks Support is required to confirm whether complete disablement is supported for that environment.
I hope this is helpful. Please! Do not hesitate to let me know if you have any other questions, steps or clarifications.
Please do not close the thread by upvoting and accepting the answer if any part of it is helpful.