Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,310 questions
0 answers
Spark 3.4 not supporting CDM
Hello I have a problem. Currently, we run notebooks that are connected to Dataverse via an Azure Synapse link. Within these notebooks, data is read into CDM. However, Microsoft has stated that Spark Pool 3.4 does not support the CDM connector. What…
asked 2025-03-26T10:08:46.7833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 69%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHA%3C/text%3E%3C/svg%3E)
Harry Agyemang
5
Reputation points
commented 2025-04-28T10:26:11.39+00:00
Harry Agyemang
5
Reputation points
0 answers
How can I access the Synapse REST API from Spark notebooks in a Synapse workspace deployed with a private endpoint?
I'm trying to access the Synapse REST API from Spark notebooks in my Synapse workspace, which is deployed with a private endpoint and has public network access disabled. When I attempt to call the API from within a notebook, I get the following…
asked 2025-04-24T06:02:59.9733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 36%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKR%3C/text%3E%3C/svg%3E)
Kartik Rana
21
Reputation points
commented 2025-04-28T10:18:39.9166667+00:00
Kartik Rana
21
Reputation points
1 answer
Can't access external REST API using pyspark with Azure Synapse
I am not able to connect to external REST API with azure synapse. Sample code is as below from the notebook which is throwing a Failed to establish a new connection: [Errno 101] Network is unreachable error. Are there any settings that needs to be…
asked 2025-04-25T06:10:12.6833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 83%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMN%3C/text%3E%3C/svg%3E)
Mampare, Ngebuthu
0
Reputation points
edited a comment 2025-04-28T09:45:44.0266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
6,280
Reputation points Microsoft External Staff
1 answer
How can I optimize the upsert method in a Synapse Copy activity when transferring data from ADLS (using a SQL query) to MongoDB?
I’ve built a pipeline that copies data to MongoDB using an on-demand SQL query with upsert functionality. The MongoDB collection currently holds around 300 million records. My goal is to update approximately 50,000 records daily and insert around 700,000…
asked 2025-04-24T12:57:55.43+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 25%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
Maverick
0
Reputation points
commented 2025-04-28T09:44:33.7266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 50%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
Venkat Reddy Navari
1,270
Reputation points Microsoft External Staff
1 answer
Azure Synapse CI/CD Pipeline Spark Pool Parameterization Issue
A Synapse Project with three notebooks is set up, each utilizing a Spark cluster, e.g., sparkuksdev. The notebooks are part of a pipeline where the Spark cluster is parameterized through a SQL Data Warehouse's Parameters table using…
asked 2025-04-24T11:09:51.4433333+00:00
Debbie Edwards
521
Reputation points
commented 2025-04-28T09:42:44.7833333+00:00
Venkat Reddy Navari
1,270
Reputation points Microsoft External Staff
0 answers
Azure Synapse Workspace deployment fails due to Location 'westeurope' is not accepting creation of new Windows Azure SQL Database servers for the subscription
I am attempting to provision an Azure Synapse Workspace, but the deployment consistently fails with the following error: {"code":"ValidationFailed","message":"Workspace request validation failed, check error details…
asked 2025-04-28T09:04:56.6533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 75%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETA%3C/text%3E%3C/svg%3E)
Teyfik Aksoy
0
Reputation points
asked 2025-04-28T09:04:56.6533333+00:00
Teyfik Aksoy
0
Reputation points
1 answer
In Azure Synapse monitor Apache Spark Applications who or what is the submitter GUID?
In Synapse studio monitor Apache Spark Applications the same GUID - 2864980c-2195-47d6-bc42-d8b4ff8e3e11 - appears many times as the submitter and I cannot find out who or what this is. I have tried >az ad sp show --id…
asked 2025-04-14T13:28:23.2+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 87%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECR%3C/text%3E%3C/svg%3E)
Cooke, Richard (Allianz UK)
0
Reputation points
edited a comment 2025-04-28T09:01:49.9566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 92%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPK%3C/text%3E%3C/svg%3E)
Pritam Kabiraj
0
Reputation points Microsoft External Staff
0 answers
Create DB identity in synapse notebook
Hi, I want to create a Synapse notebook where can read data from Azure SQL server DB. So I need to connect the DB via jdbc and enter the user name and password like code below. Is there a way where I can store this username and password somewhere so that…
asked 2025-04-24T03:06:32.7566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 14.000000000000002%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
Lotus88
116
Reputation points
edited a comment 2025-04-28T06:48:15.01+00:00
Ganesh Gurram
6,280
Reputation points Microsoft External Staff
1 answer
There are substantial concurrent MappingDataflow executions which is causing failures due to throttling under subscription
I am getting a few errors. Firstly, when I trigger a pipeline, I get the error in the title: Operation on target file failed: There are substantial concurrent MappingDataflow executions which is causing failures due to throttling under subscription…
asked 2025-04-14T09:54:51.0533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 89%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EME%3C/text%3E%3C/svg%3E)
Matthew Ethan Tam
0
Reputation points
commented 2025-04-28T06:18:18.3+00:00
AnnuKumari-MSFT
34,456
Reputation points Microsoft Employee
0 answers
Synapse Analytics - ActionTimedOut
Hello Support Team, For the past three days, our processes have been encountering recurring "ActionTimedOut" errors. No recent code changes have been made, and there has been no significant increase in data processing volume. As you can see,…
asked 2025-04-23T07:09:48.65+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 34%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELR%3C/text%3E%3C/svg%3E)
LE RAY Jerome
0
Reputation points
commented 2025-04-28T05:53:58.68+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
15,320
Reputation points Microsoft External Staff
0 answers
Azure Synapse Link for Dataverse Spark issues
I have set up a Azure Synapse Linked Service for Dataverse and the Lake Database got auto created in the Synapse workspace. I can also run a SQL script on any of the Dataverse tables that I included in the Link by right clicking them and using the select…
asked 2025-04-25T23:35:21.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 27%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBD%3C/text%3E%3C/svg%3E)
Broomfield, Darrien
0
Reputation points
commented 2025-04-28T04:34:08.33+00:00
Ganesh Gurram
6,280
Reputation points Microsoft External Staff
1 answer
Issue with Creating Storage Event Trigger in Azure Synapse – Access Token Missing Required Claims
I have Owner access on: The Azure Storage Account The Azure Synapse Workspace My storage account is a general-purpose v2 account. The Synapse workspace has system-assigned managed identity enabled. I assigned the following roles to the Synapse…
asked 2025-04-24T08:25:26.6333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 22%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Anil Kumar Nagar
0
Reputation points
commented 2025-04-28T01:08:42.5533333+00:00
Ganesh Gurram
6,280
Reputation points Microsoft External Staff
0 answers
Error generated in Synapse when I query dataverse data in Spark Notebook
Hey there, I am having an issue querying the data from the Synapse Link for Dataverse in a Spark Notebook within Synapse. I am able to run a SQL query against the data (which appears in Synapse as a Lake Database) and it returns data. See below …
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,553 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,310 questions
.NET Machine learning
.NET Machine learning
.NET: Microsoft Technologies based on the .NET software framework.Machine learning: A type of artificial intelligence focused on enabling computers to use observed data to evolve new behaviors that have not been explicitly programmed.
160 questions
asked 2022-05-18T08:17:36.823+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 84%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERO%3C/text%3E%3C/svg%3E)
Robert O'Neill
6
Reputation points
commented 2025-04-26T05:44:01.75+00:00
Broomfield, Darrien
0
Reputation points
1 answer
One of the answers was accepted by the question author.
How to export Artifacts ARM Template for a Azure Synapse Workspace?
Is there a possible way to export an "Artifacts ARM Template" from a specific resource? I have been trying to implement an Azure devops CI/CD Pipeline that migrates artifacts (Pipeline, dataflows and datasets) between Synpase workspaces (i.e.…
asked 2025-04-21T16:58:39.59+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 91%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3END%3C/text%3E%3C/svg%3E)
Napsty Dev
20
Reputation points
edited a comment 2025-04-25T19:33:15.8633333+00:00
Napsty Dev
20
Reputation points
0 answers
I am unable to publish changes in my Synapse workspace
M workspace "simple-modern-analytics" is integrate with github. We've successfully published changes hundreds of times. Now we are getting an error "Error while publishing: Failed to getFolderInSingleRequest, folder: [workspace-name].…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 16%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJC%3C/text%3E%3C/svg%3E)
1 answer
Pipeline to wait until spark cluster is available
Hello, We have a very small node cluster and due to multiple spark notebooks running at the same time the notebooks or pipeline fails with 6002 timedout error. Some notebooks do not start at all and fails. Is there a way we can queue the notebooks or…
asked 2025-04-23T15:31:40.7533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 51%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKA%3C/text%3E%3C/svg%3E)
Kumar, Arun
336
Reputation points
answered 2025-04-25T17:19:13.1666667+00:00
Kumar, Arun
336
Reputation points
1 answer
Pipelines are failing is Synapse Environment without any proper error specified?
Issue Description: Pipelines are failing in the Synapse environment without any specific error message or detailed logs. The pipelines run as usual but fail silently, and no error or warning is provided that can help diagnose the root cause of the…
asked 2025-04-22T16:55:26.9233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 65%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERY%3C/text%3E%3C/svg%3E)
Rohith Yantrapalli
0
Reputation points
commented 2025-04-25T16:44:58.7133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
11,750
Reputation points Microsoft External Staff
1 answer
How to improve data flow sink performance
Hi, I am using Synapse for ETL. When I only updated 3 records, my sink processing time still need 7 mins. My target table contains about 6 millions records. I tried to optimize the sink by setting round robin and hash with 8 partitions, but I feel the…
asked 2025-04-16T06:30:56.5566667+00:00
Lotus88
116
Reputation points
commented 2025-04-25T10:54:26.3266667+00:00
Venkat Reddy Navari
1,270
Reputation points Microsoft External Staff
1 answer
Synapse Create Lake Database Table using a Parquet file from python
I have a Lake Database in Synapse - Not a SQL DatabaseTo date I have been creating tables manually in the workspace GUI choosing "Create external table from data lake" and selecting a parquet file from my Azure Storage.I want to use Python to…
asked 2025-03-23T18:04:12.6633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 27%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EW%3C/text%3E%3C/svg%3E)
Will
0
Reputation points
commented 2025-04-25T10:03:02.8066667+00:00
Venkat Reddy Navari
1,270
Reputation points Microsoft External Staff
1 answer
What are the best practices or recommendations for setting up Disaster Recovery (DR) for services hosted on Azure?
I'm looking for recommendations on how to set up disaster recovery (DR) for the following Azure services, which are deployed in both test and production resource groups. I’ve been asked to plan a comprehensive DR strategy (Particularly Synapse dedicated…
asked 2025-04-23T12:43:03.0066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 53%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVK%3C/text%3E%3C/svg%3E)
Vivek Komarla Bhaskar
956
Reputation points
commented 2025-04-25T09:18:15.05+00:00
Venkat Reddy Navari
1,270
Reputation points Microsoft External Staff