682 questions with Azure Synapse Analytics tags

Sort by: Updated
0 answers

Cannot access secrets from notebook call by pipeline

Hi, I have grant secrets permission to my Synapse workspace and my microsoft entra ID. I have notebooks that read the secrets created by me. I can run the notebook to access my secrets and it is working. However when I tried to run the notebook from…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-07-01T09:46:54.2833333+00:00
Lotus88 176 Reputation points
commented 2025-07-02T10:20:07.11+00:00
Lotus88 176 Reputation points
0 answers

Create DB identity in synapse notebook

Hi, I want to create a Synapse notebook where can read data from Azure SQL server DB. So I need to connect the DB via jdbc and enter the user name and password like code below. Is there a way where I can store this username and password somewhere so that…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-04-24T03:06:32.7566667+00:00
Lotus88 176 Reputation points
edited a comment 2025-07-02T05:42:43.75+00:00
Lotus88 176 Reputation points
0 answers

Issues with Synapse Workflows After Updating Self-hosted Integration Runtime

After updating the Self-hosted Integration Runtime to version 5.54.9267.1 from 5.44.xx on June 10th, the pipeline runs that previously completed in about 30 minutes are now taking several hours and worsening daily. The Integration Runtime operates…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-06-23T13:01:54.3233333+00:00
John Brynford-Jones 1 Reputation point
commented 2025-06-27T17:55:11.7266667+00:00
Chandra Boorla 14,585 Reputation points Microsoft External Staff Moderator
0 answers

How to resolve an error with too many vcores being requested?

I'm trying to run a simple script on a spark notebook. But I keep running into this error. Your Spark job requested 672 vcores. However, the workspace has a 12 core limit. The script in itself should not require that many vcores as it is only checking…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-06-09T16:58:20.6533333+00:00
Joseph Y. Lee 5 Reputation points
edited a comment 2025-06-23T06:40:00.13+00:00
Pritam Kabiraj 315 Reputation points Microsoft External Staff Moderator
0 answers

Not able to create an azure synaps analytics workspace in west europe

I'm trying to create an Azure Synapse Analytics workspace located in westeurope but when I try, I recieve the error: {"code":"ValidationFailed","message":"Workspace request validation failed, check error details for…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-05-30T12:05:20.93+00:00
Esther Kiel 0 Reputation points
commented 2025-06-09T07:10:15.7533333+00:00
Smaran Thoomu 24,260 Reputation points Microsoft External Staff Moderator
0 answers

There are not enough vcores resources available in the workspace while running the synapse dataflows

Hi Team, we are getting the below error dataflow while running in for each activity, Could you please help us to fix this **There are not enough resources available in the workspace, details: 'Your job requested 8 vcores. However, the workspace…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-04-15T11:09:50.6333333+00:00
commented 2025-06-05T04:22:37.6266667+00:00
Dileep Raj Narayan Thumula 330 Reputation points Microsoft External Staff Moderator
0 answers

Can't ask for bigger limit for Synapse workspaces on subscription

When I try requesting a bigger limit, I'm faced with the following: So, no value is ever gonna be accepted. Also, the limit I'm told I actually have when I try creating a new workspace is 20. Couldn't open an actual support request, because it…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-05-26T21:23:03.49+00:00
João Florido 0 Reputation points
commented 2025-06-02T13:33:55.9466667+00:00
Shraddha Pore 525 Reputation points Microsoft External Staff Moderator
0 answers

Unable to create a managed private endpoint in synapse (because it already exists)

If i try to create a managed endpoint in synapse i get the following error message: Failed to save AzureSynapseAnalytics695. Error: {"CorrelationId":null,"StatusCode":409,"Message":"Resource already exists:…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-05-23T10:12:20.8733333+00:00
Roman Rietmann 0 Reputation points
commented 2025-05-30T05:58:22.9666667+00:00
SSingh-MSFT 16,371 Reputation points Moderator
0 answers

com.microsoft.dataflow.broker.InvalidOperationException: Only one valid authentication should be used for linkedservice

Hello, I am trying to create a linked service in synapse connecting to Azure SQL Database. The test connection works successfully with credentials in Key Vault. But when linked service used in the dataflow, the pipeline fails with below…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-05-19T18:33:20.6466667+00:00
Kumar, Arun 336 Reputation points
commented 2025-05-20T21:30:25.5833333+00:00
Chandra Boorla 14,585 Reputation points Microsoft External Staff Moderator
0 answers

Lake Database table created Delta Location not populating schema for Table in Synapse UI

I have migrated Synapse from one subscription to another including all the data that is stored in Storage account ( Which also consist Delta tables ). While migrating all the things got migrated except Lake Database tables in synapse workspace. Now I am…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-05-15T12:49:06.53+00:00
Anish, Sukhathankar 5 Reputation points
commented 2025-05-20T07:23:07.07+00:00
Anish, Sukhathankar 5 Reputation points
0 answers

Change Feed in Synapse Link for SQL Server 2022 – Getting change data as CSV file without header and starting with 4 meta data columns

I'm using Azure Synapse Link with SQL Server 2022 (on-premises), and my change data is being stored in ADLS Gen2 as CSV files. I’ve observed that the first four columns in the CSV files contain metadata, but I couldn't find clear documentation explaining…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-05-13T09:30:08.7166667+00:00
Vignesh K 20 Reputation points
commented 2025-05-15T11:53:24.98+00:00
Vignesh K 20 Reputation points
0 answers

Deltalake change data feed on synapse notebook

Hi Team, Could you please help us we are looking to enable change data feed to existing delta file but it is not working, we have used below steps to enable. set spark.databricks.delta.properties.defaults.enableChangeDataFeed =…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-05-02T06:06:55.0233333+00:00
commented 2025-05-15T01:55:25.58+00:00
Shraddha Pore 525 Reputation points Microsoft External Staff Moderator
0 answers

When I'm trying to add a table in an existing synapse link, my new table in waiting for snapshot state for long, after I check the log, I got the error message.Error 22724 severity 16 Error occurred while exporting initial snapshot to landing zone.

When I'm trying to add a table in an existing synapse link, my new table in waiting for snapshot state for long, after I check the log, I got the error message. Error 22724 severity 16 Error occurred while exporting initial snapshot to landing zone.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-04-29T07:34:44.83+00:00
Vignesh K 20 Reputation points
commented 2025-05-12T06:00:52.0666667+00:00
AnnuKumari-MSFT 34,556 Reputation points Microsoft Employee Moderator
0 answers

errorCode": "2402 unable to load bulk data using COPY command to table

I am unable to execute COPY command to table . COPY INTO stg.Temp_FctSOSTracking_TPS_India FROM 'https://mfgdasynapseprod01sa.dfs.core.windows.net/landing/RSA/2024-02/2024-02-07/2024-02-07_00:00/fct_sos_details_*.csv' WITH ( FILE_TYPE='CSV', FIRSTROW=2,…

Azure SQL Database
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2024-02-07T16:14:10.38+00:00
GUPTA, Mayank 5 Reputation points
commented 2025-05-05T01:38:11.7333333+00:00
Shelly Chen 0 Reputation points
0 answers

How to fix an established connection was aborted by the server error while making SFTP connection using Azure Synapse?

Hello Team, While we are trying to connect SFTP using SSH public key authentication we are getting following error. Established connection was aborted by the server let me share the steps which we followed so far to connect the sftp. [1] Open PuttyGen…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,633 questions
asked 2023-02-22T19:41:13.66+00:00
Nikunj Patel 61 Reputation points
commented 2025-05-02T18:02:28.6566667+00:00
JKU 0 Reputation points
0 answers

How to create a incremental pipeline in dataflow need to delete records if that is not present in source

Hi Team, Could you please help us, we have to create incremental pipeline in synapse dataflow, we should have to handle insert, update and delete(if the record not present in source we have to delete in target Azure sql), can you please confirm is it…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-04-17T17:03:20.8233333+00:00
commented 2025-04-29T11:18:49.52+00:00
Singireddi JagadishKumar 0 Reputation points Microsoft External Staff Moderator
0 answers

Spark 3.4 not supporting CDM

Hello   I have a problem. Currently, we run notebooks that are connected to Dataverse via an Azure Synapse link. Within these notebooks, data is read into CDM. However, Microsoft has stated that Spark Pool 3.4 does not support the CDM connector.   What…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-03-26T10:08:46.7833333+00:00
Harry Agyemang 5 Reputation points
commented 2025-04-28T10:26:11.39+00:00
Harry Agyemang 5 Reputation points
0 answers

Error generated in Synapse when I query dataverse data in Spark Notebook

Hey there, I am having an issue querying the data from the Synapse Link for Dataverse in a Spark Notebook within Synapse. I am able to run a SQL query against the data (which appears in Synapse as a Lake Database) and it returns data. See below …

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,562 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
Developer technologies | .NET | .NET Machine Learning
asked 2022-05-18T08:17:36.823+00:00
Robert O'Neill 6 Reputation points
commented 2025-04-26T05:44:01.75+00:00
Broomfield, Darrien 0 Reputation points
0 answers

Unable to generate MSI token from Synapse while writing data to ServiceBus and EventHub

As per doc https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/microsoft-spark-utilities?pivots=programming-language-scala only the services for which Audience is available in list are allowed to generate a token using MsSparkUtils credential…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-03-21T09:02:50.2433333+00:00
Ravi Shankar Gupta 0 Reputation points Microsoft Employee
commented 2025-04-23T12:25:39.0933333+00:00
Dileep Raj Narayan Thumula 330 Reputation points Microsoft External Staff Moderator
0 answers

Error : External table 'habitant_gold' is not accessible because location does not exist or it is used by another process.

When I run my code : CREATE DATABASE datalake_gold_db; CREATE MASTER KEY ENCRYPTION BY PASSWORD ='Welcome@MS92'; CREATE DATABASE SCOPED CREDENTIAL MyCred WITH IDENTITY='SHARED ACCESS SIGNATURE', SECRET='SAS Token'; CREATE EXTERNAL DATA SOURCE…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
asked 2025-04-13T18:12:00.97+00:00
Mehdi SAADI 0 Reputation points
commented 2025-04-23T12:21:37.24+00:00
Dileep Raj Narayan Thumula 330 Reputation points Microsoft External Staff Moderator