Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,304 questions
0 answers
Issue with Creating Storage Event Trigger in Azure Synapse – Access Token Missing Required Claims
I have Owner access on: The Azure Storage Account The Azure Synapse Workspace My storage account is a general-purpose v2 account. The Synapse workspace has system-assigned managed identity enabled. I assigned the following roles to the Synapse…
asked 2025-04-24T08:25:26.6333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 22%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Anil Kumar Nagar
0
Reputation points
asked 2025-04-24T08:25:26.6333333+00:00
Anil Kumar Nagar
0
Reputation points
0 answers
Create DB identity in synapse notebook
Hi, I want to create a Synapse notebook where can read data from Azure SQL server DB. So I need to connect the DB via jdbc and enter the user name and password like code below. Is there a way where I can store this username and password somewhere so that…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 14.000000000000002%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
1 answer
Pipelines are failing is Synapse Environment without any proper error specified?
Issue Description: Pipelines are failing in the Synapse environment without any specific error message or detailed logs. The pipelines run as usual but fail silently, and no error or warning is provided that can help diagnose the root cause of the…
asked 2025-04-22T16:55:26.9233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 65%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERY%3C/text%3E%3C/svg%3E)
Rohith Yantrapalli
0
Reputation points
answered 2025-04-24T07:41:23.3466667+00:00
Rohith Yantrapalli
0
Reputation points
1 answer
How can I access the Synapse REST API from Spark notebooks in a Synapse workspace deployed with a private endpoint?
I'm trying to access the Synapse REST API from Spark notebooks in my Synapse workspace, which is deployed with a private endpoint and has public network access disabled. When I attempt to call the API from within a notebook, I get the following…
asked 2025-04-24T06:02:59.9733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 36%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKR%3C/text%3E%3C/svg%3E)
Kartik Rana
21
Reputation points
edited an answer 2025-04-24T06:57:44.0466667+00:00
Deepanshukatara-6769
15,275
Reputation points
1 answer
Synapse Create Lake Database Table using a Parquet file from python
I have a Lake Database in Synapse - Not a SQL DatabaseTo date I have been creating tables manually in the workspace GUI choosing "Create external table from data lake" and selecting a parquet file from my Azure Storage.I want to use Python to…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 27%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EW%3C/text%3E%3C/svg%3E)
2 answers
Issues Connecting to Infor Data Lake Using Spark JDBC in Azure Synapse Spark Notebook
Hi Team, I'm encountering difficulties while attempting to connect to Infor Data Lake using Spark JDBC within Azure Synapse. Despite configuring the necessary JDBC driver and ionapi file path, I consistently encounter the following…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 15%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 30%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
1 answer
How to improve data flow sink performance
Hi, I am using Synapse for ETL. When I only updated 3 records, my sink processing time still need 7 mins. My target table contains about 6 millions records. I tried to optimize the sink by setting round robin and hash with 8 partitions, but I feel the…
1 answer
How can I use the Incremental Column property in the Sink activity in Azure Synapse Analytics Data Flow?
I am trying to transfer data from my source table to my destination table within a Data Flow. Since my table contains a large volume of data, I attempted to use the Incremental Column feature shown in the screenshot below by selecting a suitable column…
asked 2025-04-17T16:05:15.3933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 20%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBY%3C/text%3E%3C/svg%3E)
Burak Yoleri
0
Reputation points
commented 2025-04-23T16:31:01.08+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 99%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJN%3C/text%3E%3C/svg%3E)
J N S S Kasyap
1,620
Reputation points Microsoft External Staff
0 answers
Pipeline to wait until spark cluster is available
Hello, We have a very small node cluster and due to multiple spark notebooks running at the same time the notebooks or pipeline fails with 6002 timedout error. Some notebooks do not start at all and fails. Is there a way we can queue the notebooks or…
asked 2025-04-23T15:31:40.7533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 51%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKA%3C/text%3E%3C/svg%3E)
Kumar, Arun
336
Reputation points
commented 2025-04-23T16:01:57.2066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
11,665
Reputation points Microsoft External Staff
1 answer
Does Azure Synapse Dedicated SQL Pools support group-based authentication?
Requirement: Grant read-only (or specific) access to all tables in an Azure Synapse Dedicated SQL Pool for a set of users by assigning them to a group in Azure Entra ID. Execution: For example, say I've created a group on Entra ID:…
asked 2025-04-23T11:26:48.31+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 53%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVK%3C/text%3E%3C/svg%3E)
Vivek Komarla Bhaskar
936
Reputation points
commented 2025-04-23T13:44:08.0566667+00:00
Vivek Komarla Bhaskar
936
Reputation points
1 answer
What are the best practices or recommendations for setting up Disaster Recovery (DR) for services hosted on Azure?
I'm looking for recommendations on how to set up disaster recovery (DR) for the following Azure services, which are deployed in both test and production resource groups. I’ve been asked to plan a comprehensive DR strategy (Particularly Synapse dedicated…
asked 2025-04-23T12:43:03.0066667+00:00
Vivek Komarla Bhaskar
936
Reputation points
edited an answer 2025-04-23T13:36:37.2733333+00:00
Nandan Hegde
34,941
Reputation points MVP
0 answers
Unable to generate MSI token from Synapse while writing data to ServiceBus and EventHub
As per doc https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/microsoft-spark-utilities?pivots=programming-language-scala only the services for which Audience is available in list are allowed to generate a token using MsSparkUtils credential…
asked 2025-03-21T09:02:50.2433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 41%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Ravi Shankar Gupta
0
Reputation points Microsoft Employee
commented 2025-04-23T12:25:39.0933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 57.00000000000001%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDR%3C/text%3E%3C/svg%3E)
Dileep Raj Narayan Thumula
0
Reputation points Microsoft External Staff
0 answers
Error : External table 'habitant_gold' is not accessible because location does not exist or it is used by another process.
When I run my code : CREATE DATABASE datalake_gold_db; CREATE MASTER KEY ENCRYPTION BY PASSWORD ='Welcome@MS92'; CREATE DATABASE SCOPED CREDENTIAL MyCred WITH IDENTITY='SHARED ACCESS SIGNATURE', SECRET='SAS Token'; CREATE EXTERNAL DATA SOURCE…
asked 2025-04-13T18:12:00.97+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 16%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMS%3C/text%3E%3C/svg%3E)
Mehdi SAADI
0
Reputation points
commented 2025-04-23T12:21:37.24+00:00
Dileep Raj Narayan Thumula
0
Reputation points Microsoft External Staff
0 answers
There are not enough vcores resources available in the workspace while running the synapse dataflows
Hi Team, we are getting the below error dataflow while running in for each activity, Could you please help us to fix this **There are not enough resources available in the workspace, details: 'Your job requested 8 vcores. However, the workspace…
asked 2025-04-15T11:09:50.6333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 22%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
SaiSekhar, MahasivaRavi (Philadelphia)
140
Reputation points
commented 2025-04-23T12:10:58.0166667+00:00
Dileep Raj Narayan Thumula
0
Reputation points Microsoft External Staff
0 answers
How to create a incremental pipeline in dataflow need to delete records if that is not present in source
Hi Team, Could you please help us, we have to create incremental pipeline in synapse dataflow, we should have to handle insert, update and delete(if the record not present in source we have to delete in target Azure sql), can you please confirm is it…
asked 2025-04-17T17:03:20.8233333+00:00
SaiSekhar, MahasivaRavi (Philadelphia)
140
Reputation points
commented 2025-04-23T11:55:45.9633333+00:00
SaiSekhar, MahasivaRavi (Philadelphia)
140
Reputation points
1 answer
All accountid's in the contact-table are NULL when using Azure Synapse Link or a Synapse pipeline to parquet
I've tried two ways to copy data from our Dynamics 365 to a Storage Account on Azure: Azure Synapse Analytics: creating a pipeline that connects to Dynamics 365 and copies the data the a parquet file on a storage account. I tried fetching the data in…
asked 2025-02-17T20:05:44.3333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 97%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETD%3C/text%3E%3C/svg%3E)
Tim Derdelinckx
0
Reputation points
commented 2025-04-23T10:16:16.1733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(313.6, 73%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Nick Sabbe
0
Reputation points
0 answers
Synapse Analytics - ActionTimedOut
Hello Support Team, For the past three days, our processes have been encountering recurring "ActionTimedOut" errors. No recent code changes have been made, and there has been no significant increase in data processing volume. As you can see,…
asked 2025-04-23T07:09:48.65+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 34%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELR%3C/text%3E%3C/svg%3E)
LE RAY Jerome
0
Reputation points
commented 2025-04-23T08:16:04.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
15,315
Reputation points Microsoft External Staff
1 answer
New column in CosmosDB NoSQL does not show up in Analytical Store
When we add a new column to new documents in our CosmosDB NoSQL container, the new column does not show up in Analytical Store unless we add that new column to every other document in the container. This breaks not just queries for the new column, but…
asked 2024-10-21T23:04:10.2166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 35%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJH%3C/text%3E%3C/svg%3E)
Jon Hawkins
0
Reputation points
edited a comment 2025-04-22T22:23:51.4333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 96%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEO%3C/text%3E%3C/svg%3E)
Erik O'Leary
16
Reputation points
0 answers
Data Sync Delays with Microsoft Synapse
The process of exporting data from Dynamics to the new Data Lake (DL) via Dataverse and converting it to parquet format is taking around an hour, which is significantly slower than the existing 15–20 minute performance via export to datalake. This delay,…
asked 2025-04-11T16:29:51.0866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 45%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Steve Busi
0
Reputation points
commented 2025-04-22T20:42:43.94+00:00
Steve Busi
0
Reputation points
0 answers
Cold Start Query Performance
When running queries against the parquet format in the new DL, the initial "cold start" query takes significantly longer (e.g., 50 seconds) compared to subsequent queries (e.g., 10 seconds). This is double the time compared to the existing DL…
asked 2025-04-11T16:31:55.07+00:00
Steve Busi
0
Reputation points
commented 2025-04-22T20:40:20.91+00:00
Steve Busi
0
Reputation points