How can I access the Synapse REST API from Spark notebooks in a Synapse workspace deployed with a private endpoint?

Question

How can I access the Synapse REST API from Spark notebooks in a Synapse workspace deployed with a private endpoint?

Kartik Rana 21

I'm trying to access the Synapse REST API from Spark notebooks in my Synapse workspace, which is deployed with a private endpoint and has public network access disabled. When I attempt to call the API from within a notebook, I get the following error:

PublicNetworkAccessDenied

However, if I run the same code locally while connected to our corporate VPN, it works without any issues.

Just to clarify, the private endpoint setup generally works as expected. For example:

I need to be connected to the VPN to even load items in the Synapse Studio.

Data exports from the serverless SQL pool to Power BI also work seamlessly through the private endpoint.

So, the private networking seems correctly configured and is functioning well for most services.

The issue arises only when trying to access the Synapse REST API from Spark notebooks.

How can I configure my environment to allow REST API calls from within Spark notebooks to route through the private endpoint as well?

Note: I need a solution where I don't have to use the Web activity in a pipeline

1 answer

Your answer

Answer 1

Hello , Welcome to MS Q&A

To enable private access to an Azure Synapse workspace via REST calls using Managed Private Endpoints and Private DNS, you'll first configure a Managed Private Endpoint within your Synapse workspace, which creates a private IP address in your Managed Virtual Network. Then, you'll integrate this endpoint with a private DNS zone to ensure that your application uses the private IP for communication. This setup allows secure, private access to your Synapse workspace without exposing it to the public internet.

Here's a more detailed breakdown:
1. 1. Enable Managed Virtual Network: If your Synapse workspace doesn't already have a Managed Virtual Network, ensure you enable it. This is a prerequisite for using Managed Private Endpoints.
2. 2. Create a Managed Private Endpoint:
  - Open your Azure Synapse workspace in the Azure portal.
  - Navigate to the Managed private endpoints section within the Manage tab.
  - Create a new Managed private endpoint, specifying the target resource (e.g., Azure Storage, Azure Cosmos DB, etc.).
  - The endpoint will be assigned a private IP address within your Managed Virtual Network.
3. 3. Configure Private DNS Integration:
  - In your Synapse workspace's settings, locate the "Private endpoint connections" section under "Security".
  - For each private endpoint connection, select "Integrate with private DNS zone" to link it to your existing or new private DNS zone.
  - This will ensure that your application uses the private IP address when resolving the FQDN of your Synapse workspace.
4. 4. Update DNS settings:
  - You'll need to update your DNS settings to resolve the FQDN of your Synapse workspace to the private IP address of the Managed Private Endpoint.
  - This can be done using Private DNS Zones, Azure Private Resolver, or by modifying the host file (for testing).
5. 5. Test the connection:
  - Use a REST client (like Postman or the curl command) to make requests to your Synapse workspace, ensuring that the connection is using the private IP address and DNS records.

Pls check this link for ref -->https://docs.azure.cn/en-us/synapse-analytics/security/how-to-connect-to-workspace-with-private-links

Kindly let us know if any further ques

Pls accept if it helps

Thanks

Deepanshu

Share via

How can I access the Synapse REST API from Spark notebooks in a Synapse workspace deployed with a private endpoint?

1 answer

Your answer