An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
Thank you for providing such a detailed analysis of your architecture and the troubleshooting steps you've already completed.
Based on the behavior you've described and the current Microsoft documentation, your findings appear consistent with the expected design of Mirrored Azure Databricks Catalog.
When the underlying ADLS Gen2 account is protected by a storage firewall, Fabric uses the Workspace Identity to access the storage account, regardless of the authentication method configured in the Databricks cloud connection. The cloud connection is used to authenticate to Databricks/Unity Catalog, while storage access relies on the Fabric Workspace Identity.
The documented method for allowing Workspace Identity access through a storage firewall is Trusted Workspace Access (TWA) via a resource instance rule on the storage account. However, the TWA documentation currently states that Trusted Workspace Access is not supported for cross-tenant requests.
As a result, in a scenario where the Fabric workspace resides in Tenant A and the firewalled ADLS Gen2 account resides in Tenant B, the observed behavior is expected:
- Mirrored Catalog refresh succeeds when the storage firewall is disabled.
- Mirrored Catalog refresh fails when the storage firewall is enabled because the Fabric Workspace Identity cannot be authorized through TWA across tenants.
Based on the currently published documentation, I am not aware of a supported configuration that enables Mirrored Azure Databricks Catalog to access a cross-tenant, firewall-enabled ADLS Gen2 account using Trusted Workspace Access.
The available options would therefore be to:
- Align Fabric and the Databricks/ADLS resources within the same tenant so TWA can be used as designed, or
- Consider alternative data-sharing or replication architectures that do not require Fabric Workspace Identity to traverse a cross-tenant storage firewall.
For reference:
- Fabric Mirrored Azure Databricks Catalog tutorial: https://learn.microsoft.com/fabric/mirroring/azure-databricks-tutorial
- Secure Fabric mirrored databases from Azure Databricks: https://learn.microsoft.com/fabric/mirroring/azure-databricks-security
- Trusted Workspace Access: https://learn.microsoft.com/fabric/security/security-trusted-workspace-access
If this cross-tenant scenario is a business requirement, I would recommend opening a Microsoft Support ticket to confirm whether there are any recently introduced capabilities or roadmap updates that are not yet reflected in the public documentation.
I hope this helps clarify the limitation.