12,190 questions with Azure Data Factory tags

Sort by: Updated
1 answer

ADF issue in running Scope activity

n Azure Data Factory, I have setup a Pipeline that runs a Scope script activity against Asimov VC to aggregate data. I am finding that the activity is failing whenever it runs for more than 3 hours+. The activity error I see is  Error…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-07-08T21:47:00.1+00:00
Balaji Krish 0 Reputation points Microsoft Employee
commented 2026-07-14T19:05:13.6733333+00:00
Pilladi Padma Sai Manisha 11,350 Reputation points Microsoft External Staff Moderator
2 answers

SSISDB catalog schema version remains fixed at 5 regardless of Managed Instance update policy/compatibility level changes

We're trying to determine whether the SSISDB catalog schema version can be advanced beyond its current value, and whether it's expected to track our Managed Instance's update policy. Steps taken: Original SSISDB catalog reported SCHEMA_VERSION = 5 (via…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-07-13T12:18:11.47+00:00
Jim Evans - DBA 0 Reputation points
answered 2026-07-14T09:22:01.1133333+00:00
Christos Panagiotidis 81 Reputation points
2 answers

Azure Data Factory SFTP Service error: SftpAuthenticationFailure

I'm using ADF Linked Service for SFTP connection to get data from an on-premises server. We're running SHIR on containers, and keys/passphrases are passed by key vault. Meet authentication failure when connect to Sftp server '' using 'SshPublicKey'…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-07-06T11:47:31.9+00:00
Apurva Pathak 920 Reputation points
commented 2026-07-13T16:34:07.28+00:00
Sina Salam 30,566 Reputation points Volunteer Moderator
2 answers

URGENT: Our sharepoint pipelines on ADF has been failing due to ACS authentication depreciation

Migration steps taken: **•** Received a new .pfx certificate; imported into Key Vault (base64-encoded secret), with the certificate password stored as a separate secret **•** Created a new **REST linked service** with authenticationType:…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-07-13T04:04:54.4833333+00:00
Santosh Sudhindira 0 Reputation points
commented 2026-07-13T09:19:06.27+00:00
Srikanth Chavithina 20 Reputation points Microsoft External Staff Moderator
2 answers

[URGENT] ADF WebActivity + SharePoint REST API app-only with Sites.Selected — "Unsupported app only token" even with valid AAD token

We have Azure Data Factory (not the fabric version) pipelines that read and write files from SharePoint Online document libraries. The pipelines use a WebActivity to manually obtain an AAD bearer token, then call the SharePoint REST API (/_api/) with…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-07-07T10:39:23.64+00:00
msuser50495 0 Reputation points
edited an answer 2026-07-10T23:04:49.5133333+00:00
Ganesh Chelluri 0 Reputation points Microsoft External Staff Moderator
3 answers One of the answers was accepted by the question author.

Using AME unable to create Linked Service for Azure SQL in Data Factory

Using AME unable to create Linked Service for Azure SQL in Data Factory. Any help would be greatly appreciated.  

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-24T01:44:55.94+00:00
Hemant 61 Reputation points Microsoft External Staff
accepted 2026-07-08T21:55:41.6566667+00:00
Hemant 61 Reputation points Microsoft External Staff
1 answer

Try the new Azure Data Factory security toggle

What steps to take regarding the notice "Azure Data Factory will retire support for the trusted services firewall exception that allows access to Azure Storage accounts and Azure Key Vault by using a managed identity. This change will take effect on…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-07-08T20:37:17.0833333+00:00
Perera, Marlon 0 Reputation points
answered 2026-07-08T21:29:06.53+00:00
Perera, Marlon 0 Reputation points
3 answers

Azure Data Factory resources and linked service disappeared. Error code 401 Failed to load resource in console.

Hi, I have been working on my data factory for about a day now. I added a linked service and configured around 6 pipelines to feed into my PostgreSQL database. This was working successfully and I was getting ready to publish. I log in later today and…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-29T20:59:30.9366667+00:00
Dylan Johns 0 Reputation points
commented 2026-07-07T18:32:26.17+00:00
Pilladi Padma Sai Manisha 11,350 Reputation points Microsoft External Staff Moderator
3 answers One of the answers was accepted by the question author.

Does the Azure Data Factory HubSpot connector v2.0 (Preview) support incremental load?

The HubSpot connector 1.0 supported incremental loads by means of passing a query. The new 2.0 connector does not support queries. Does that mean the new connector does not support incremental loads, or is there some other option? Fetching all of the…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-15T07:21:26.49+00:00
Tariq Riahi 20 Reputation points
accepted 2026-07-06T09:50:06.5333333+00:00
Tariq Riahi 20 Reputation points
3 answers

Azure Data Factory : Successfactors OAuth challenge

Hi, we currently ingest data from Successfactors in Azure Data Factory using REST API and Basic Authentification. As you may know, Basic Authentification will be depreciated in November 2026, we wish to turn to OAuth. After research we figured out there…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-26T07:51:44.8166667+00:00
RENOUD Thomas 0 Reputation points
recommended 2026-07-03T16:16:06.2+00:00
Manoj Kumar Boyini 18,765 Reputation points Microsoft External Staff Moderator
3 answers

Azure Data Factory linked services ClientIPNotAuthorized

Nothing changed on our pipeline/resources The linked services and publishing fail with Failed to get DataPlane access, reason: {"code":"ClientIPNotAuthorized","message":"Client IP not authorized to access the API.…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-29T22:38:59.2633333+00:00
Felix Fung 0 Reputation points Microsoft Employee
answered 2026-06-30T20:05:21.4366667+00:00
Felix Fung 0 Reputation points Microsoft Employee
4 answers

How to parametrize the referenceName for Linked Service Reference in the Datasets part for only 1 parquet dataset.

We currently have two Azure Data Factory instances: one for DEV and one for PROD. The issue is related to dataset parameterization. We have 37 Parquet datasets, but only one of them needs to reference a different linked service in the PROD environment.…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-23T13:58:41.3633333+00:00
Dina 0 Reputation points
commented 2026-06-27T07:09:55.3866667+00:00
Manoj Kumar Boyini 18,765 Reputation points Microsoft External Staff Moderator
2 answers One of the answers was accepted by the question author.

ADF pipeline getting triggered for old blob event

Hi All, I have a storage event trigger attached to a pipeline in ADF. On 4-jun-2026 between 4:00 and 4:05 PM UTC around 70k blob json files were written to the ADLS2 path monitored by storage event trigger. Due to this 10000 runs of pipeline started as…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-25T12:48:22.1166667+00:00
JigarkumarPatel-5373 46 Reputation points
accepted 2026-06-26T11:53:00.7533333+00:00
JigarkumarPatel-5373 46 Reputation points
2 answers

how to fix The template validation failed: 'The number of template actions limit exceeded: maximum '1500' and actual '1501'

Error in ADF: Operation on target For Each Tenant failed: Activity failed because an inner activity failed; Inner activity name: Execute StageTenantTablesInner, Error: ErrorCode=InvalidTemplate, ErrorMessage=The template validation failed: 'The number of…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-23T06:33:38.25+00:00
Goundla, Sai Karthik Goud 0 Reputation points
commented 2026-06-26T08:37:21.9633333+00:00
Manoj Kumar Boyini 18,765 Reputation points Microsoft External Staff Moderator
2 answers

How to send a JSON payload to Azure SQL (via Azure Data Factory)?

I am unable successfully send a JSON payload over to an Azure DB Stored Procedure as i am getting this error: Execution fail against sql server. Please contact SQL Server team if you need further support. Sql error number: 13609. Error Message: JSON…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2023-09-07T11:00:35.3866667+00:00
Kevin Perez 10 Reputation points
commented 2026-06-25T18:37:23.6333333+00:00
MikeBurek 0 Reputation points
3 answers

how connect to a source system - SAP database?

how connect to a source SAP database?This project involves the migration of the existing data warehouse environment from Qlik to Microsoft Azure. The initiative aims to modernize the analytics platform by leveraging Azure’s scalable storage, compute, and…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-04-15T09:42:26.0466667+00:00
Makama, ANTHEA (404) 0 Reputation points
commented 2026-06-25T09:25:19.95+00:00
sega 0 Reputation points
2 answers

getting error while creating crm dynamics 365 dataset in ADF

I am encountering the following error while trying to create a dataset using the Dynamics 365 connector in Azure Data Factory (ADF) although Linked service connection successful: "The HTTP request was forbidden with client authentication scheme…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-24T10:43:20.4666667+00:00
Khurshid, Danish 0 Reputation points
answered 2026-06-24T11:30:24.88+00:00
Manoj Kumar Boyini 18,765 Reputation points Microsoft External Staff Moderator
1 answer One of the answers was accepted by the question author.

Our production pipelines are failing due to shared error and need your assistance further.

Error Copy User data to SQL_copy1 Troubleshoot activity failures { "errorCode": "2200", "message": "ErrorCode=SqlFailedToConnect,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Cannot…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-23T09:22:05.1666667+00:00
Murli Kunagu (Tata Consultancy Services Limited) 20 Reputation points Microsoft External Staff
accepted 2026-06-24T02:48:52.85+00:00
Murli Kunagu (Tata Consultancy Services Limited) 20 Reputation points Microsoft External Staff
2 answers

ADF Pipeline Error: This resource references resources which could not be loaded.

I have an existing data factory. Trying to create a CICD pipeline from adf_publish branch via ADO. (Aiming for it to auto run with each successful pull request. Failing to export and publish the artifact. Error logs: BaseFileResourceProviderService:…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-06-23T17:43:35.6533333+00:00
Soniran, Dare 0 Reputation points
answered 2026-06-23T19:31:22.6033333+00:00
SAI JAGADEESH KUDIPUDI 3,640 Reputation points Microsoft External Staff Moderator
0 answers

Join: Microsoft Azure Q&A Champions Program

The Microsoft Azure Q&A Champions Program recognizes and empowers a global community of internal and external Azure experts who help customers succeed by providing high-quality, trusted answers on Microsoft Q&A. The program scales expert led…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,190 questions
asked 2026-05-25T23:37:24.3633333+00:00
Ashley Swearingen 0 Reputation points Microsoft External Staff Moderator
edited the question 2026-06-23T19:16:12.25+00:00
Ashley Swearingen 0 Reputation points Microsoft External Staff Moderator