Share via

Databricks network connectivity

Anushree N 40 Reputation points
2026-05-21T09:50:26.2666667+00:00

For both private and public subnet, the virtual network traffic and internet traffic is routed to Azure firewall. So what are the Azure firewall rules required for the databricks workspace to work correctly Also should there be rules for connectivity between public and private subnet, since the vnet traffic is also routed to Azure firewall

Azure Databricks
Azure Databricks

An Apache Spark-based analytics platform optimized for Azure.

3 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Manoj Kumar Boyini 15,570 Reputation points Microsoft External Staff Moderator
    2026-05-25T13:55:55.5466667+00:00

    Hi @Anushree N

    If your storage account is intended to be accessed via a Private Endpoint, the primary area to validate here is DNS resolution and Private Endpoint configuration, rather than Azure Firewall rules.

    For Private Link scenarios:

    • Traffic to the storage account is expected to use the private IP inside the VNet
    • The storage firewall governs access to the public endpoint
    • Therefore, Azure Storage service tag access is generally not required for that same storage account when Private Endpoint connectivity is functioning correctly

    For Azure Databricks, please validate:

    • Both dfs and blob private endpoints are created (if applicable) and are in Approved state
    • The correct private DNS zones are configured:
      • privatelink.dfs.core.windows.net
        • privatelink.blob.core.windows.net
        • These DNS zones are linked to the Databricks VNet
        • If custom DNS is used, ensure name resolution correctly returns the private endpoint IPs

    Since traffic is routed through Azure Firewall:

    • Confirm routing and firewall policies allow traffic to the private endpoint IP range
    • For private endpoint traffic, validate network path and network rules rather than relying on FQDN filtering

    If name resolution does not return the private endpoint IP, connections may continue to target the public endpoint even though the private endpoint exists.

    Answer to your question:

    No, Azure Storage service tag is generally not required for a storage account being accessed through Private Endpoint. It may still be needed if additional storage dependencies are accessed through public endpoints or if other storage accounts are not using Private Link.

    Please let us know if you have any questions or concerns.

    References:
    https://learn.microsoft.com/en-us/azure/databricks/security/network/storage/firewall-support
    https://learn.microsoft.com/en-us/azure/storage/common/storage-private-endpoints
    https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-dns

    Was this answer helpful?

    0 comments
  2. Manoj Kumar Boyini 15,570 Reputation points Microsoft External Staff Moderator
    2026-05-22T14:15:29.99+00:00

    Hi @Anushree N

    When both the public and private Databricks subnets are configured to route virtual network traffic and internet traffic through Azure Firewall (using UDRs), the key requirement is to allow the required outbound connectivity from the Databricks data plane to the Databricks control plane and dependent Azure services.

    Azure Databricks does not require broad open communication between the public and private subnets. The focus should be on allowing the documented outbound dependencies while keeping east-west traffic restricted unless your workload specifically requires it.

    Recommended firewall considerations:

    1. Allow Databricks control-plane connectivity Allow outbound connectivity from both Databricks subnets to the Azure Databricks control plane using the AzureDatabricks service tag where possible instead of maintaining static IP ranges.
    2. Allow Azure platform dependencies used by Databricks Depending on the workload, allow outbound access to:
    • Azure Storage (DBFS, logs, artifacts)
    • Event Hubs (cluster telemetry and platform communication where applicable)
    • Microsoft Entra ID (authentication/token acquisition)
    • Azure Key Vault (if integrated)
    • Azure Container Registry or external package repositories if clusters install dependencies
    1. DNS connectivity Ensure the Databricks subnets can resolve required Azure and Databricks endpoints through the configured DNS path.
    2. Public ↔ Private subnet communication Additional broad firewall rules between the Databricks public and private subnets are generally not required solely because VNet traffic is routed through Azure Firewall. Azure Databricks manages the workspace networking model and NSG requirements for delegated subnets. Only permit inter-subnet traffic if your own applications require it.
    3. If Private Link is enabled Additional documented Private Link connectivity requirements apply for the private endpoint subnet and should be configured accordingly.

    Recommendation: Use service tags + UDR + Azure Firewall rather than static IP allowlists wherever possible to reduce operational overhead and avoid connectivity issues during service updates.

    References:
    https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/udr?
    https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/vnet-inject
    https://learn.microsoft.com/en-us/azure/databricks/resources/ip-domain-region
    https://learn.microsoft.com/en-us/azure/databricks/security/network/concepts/private-link
    https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/secure-cluster-connectivity
    https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/on-prem-network

    Please let us know if you have any questions or concerns.

    Was this answer helpful?

    0 comments

  3. AI answer

    2026-05-21T09:50:52.4833333+00:00

    For an Azure Databricks workspace where both private and public subnets send all traffic to Azure Firewall, the following rule sets are required so the workspace and UI function correctly.

    1. Front-end (control plane and UI) access Configure outbound rules from the Databricks subnets to allow HTTPS and WebSocket traffic to Databricks control-plane domains and UI CDNs: a. Databricks workspace domains
      • Option 1 (simpler, more permissive):
        • Allow HTTPS (TCP 443) and WebSocket traffic to:
          • *.azuredatabricks.net (or *.databricks.azure.us for Azure Government)
      • Option 2 (more restrictive, per-workspace):
        • Identify the two workspace-specific domains from the Azure Databricks resource blade in the Azure portal.
        • Allow HTTPS (TCP 443) and WebSocket traffic to those two FQDNs only.
      b. Databricks UI/CDN domains
      • Allow HTTPS (TCP 443) and all asset types (CSS, JS, fonts, images) to:
        • https://databricks-ui-assets-v2-gsd4bmhzapcyenec.b02.azurefd.net
        • https://ui-assets.azuredatabricks.net
        • https://*.cloud.databricksusercontent.com
      • Do not selectively block CSS, fonts, or images; doing so breaks the UI.
    2. SCC relay and service endpoints (egress from workspace subnets)
      • When using Secure Cluster Connectivity (No Public IP / NPIP), always allowlist the documented SCC relay FQDNs (not individual IPs) in Azure Firewall.
      • Ensure outbound HTTPS (TCP 443) from both workspace subnets to those SCC relay FQDNs is allowed.
      • Do not rely on static IPs for these endpoints; IPs change and will cause outages if hardcoded.
    3. Data plane / storage and other Azure services
      • If using storage accounts, Event Hubs, or other Azure PaaS services:
        • Allow outbound traffic from Databricks subnets to those services according to the chosen pattern:
          • Via service endpoints + service endpoint policies, or
          • Via private endpoints (in a dedicated subnet) with firewall rules on the PaaS resource.
      • For Event Hubs or similar services where FQDN filtering is required, use Azure Firewall to allow only the required FQDNs for that region.
    4. Internet egress via NAT/Firewall
      • For classic compute with secure cluster connectivity:
        • Use an Azure NAT Gateway on the workspace subnets or route 0.0.0.0/0 through Azure Firewall (in a hub or DMZ subnet) using UDRs.
        • If routing 0.0.0.0/0 to the firewall, ensure the firewall has outbound rules allowing:
          • Required public repositories (OS repos, container registries, etc.) that clusters need.
          • Any additional external services required by jobs.
    5. IP access lists (optional, Databricks-side)
      • Independently of Azure Firewall, Databricks IP access lists can restrict which client IPs can reach the workspace.
      • Ensure that any corporate egress IPs (for admins/users) are allowed in Databricks IP access lists if that feature is enabled.
    6. General firewall configuration guidance for Databricks
      • Use FQDN-based rules for Databricks control-plane and SCC relay endpoints.
      • Apply consistent rules across all required CDN domains.
      • Allow HTTPS (443) for all listed Databricks and CDN domains.
      • If using a hub-and-spoke model, ensure UDRs from Databricks subnets send 0.0.0.0/0 to the firewall and that return traffic is allowed.

    These rules, combined with correct routing (UDRs to the firewall and/or NAT Gateway) and any required private endpoints/service endpoints for storage and other PaaS services, allow the Databricks workspace to operate correctly while all traffic is inspected by Azure Firewall.

    References:

    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.