Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
1 answer
One of the answers was accepted by the question author.
Managing conflicts regarding an Azure DevOps Repos for Data Factory
Hi, I'm working together with other colleagues to a Data Factory instance linked to an Azure DevOps Repos. In a such team working scenario sometimes it occurs a conflict about a pull request to merge a dev branch to main branch, conflict that could be…
Azure Data Factory
asked
2024-03-04T16:07:11.53+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 41%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
pmscorca
1,052
Reputation points
commented
2026-10-27T10:45:52.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 72%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJR%3C/text%3E%3C/svg%3E)
Jeroen Reinink (JR DevTools)
0
Reputation points
2 answers
Issues Azure data Factory SFTP connection ( SSH client)
HI MS Team, We encounter some issues with connecting a SFTP server of one of our financial counter parties. The issue is due to security level that they do not support rsa-sha but only the rsa-sha2 version for host key exchange. If i look at the error…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-09-07T10:30:11.16+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 85%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Shivata Tikoo
0
Reputation points
recommended
2026-10-27T01:18:48.9133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 57.99999999999999%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESY%3C/text%3E%3C/svg%3E)
SIVASANKAR YEDDULA
250
Reputation points Microsoft External Staff
Moderator
1 answer
Data Factory: Pipeline Start Delayed Without Apparent Reason — Intermittent Issue
Problem description I am experiencing an issue where my Azure Data Factory pipeline starts as expected, but the parallel data flow activities within the pipeline begin approximately 19 minutes later, which is unusual compared to other runs. This delay…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 88%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
1 answer
Azure credits for university educators beyond the $100 Azure for Students offer
I am a university educator using Azure for teaching and preparing practical exercises for my students. The $100 Azure for Students credit is insufficient for these activities. Are there additional Azure credits, academic grants, or sponsorship programs…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-10-12T18:25:56.18+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 74%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZA%3C/text%3E%3C/svg%3E)
ZARAS ANDREAS;ΖΑΡΑΣ ΑΝΔΡΕΑΣ
0
Reputation points
answered
2026-10-24T08:55:37.6566667+00:00
kagiyama yutaka
5,655
Reputation points
1 answer
Post/Put huge volume of data to an api endpoint through ADF
I have few output files in my unity catalog volumes/adls gen2. I want to push the data of these files as a body for api endpoint. However the data from my table is huge which goes upto 2 million rows, which is then converted to the below formatted json…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-10-16T07:34:29.6466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 6%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHK%3C/text%3E%3C/svg%3E)
Hrishikesh K M
0
Reputation points
answered
2026-10-23T03:55:39.4833333+00:00
Rukshan edirisinghe
2,205
Reputation points
2 answers
Error occurring during the SSH handshake with ADF
I am trying to find a solution on this error when I tried to copy data from a client who uses AWS for SFPT. I am using ADF (Azure Data Factory) to move data from this client's SFTP. Bottom is details of error…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-09-14T21:28:00.8833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 39%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJD%3C/text%3E%3C/svg%3E)
Justin Doh
1,000
Reputation points
commented
2026-10-20T08:50:30.0333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 51%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGC%3C/text%3E%3C/svg%3E)
Ganesh Chelluri
205
Reputation points Microsoft External Staff
Moderator
3 answers
Azure Data Factory Copy activities remain Queued for several hours intermittently
The pipeline had been running successfully for approximately one month. Recently, some Copy activities have remained in Queued status for several hours, while other activities in the same pipeline complete successfully. During the morning run, all…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-10-13T11:13:53.41+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 64%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EU%3C/text%3E%3C/svg%3E)
Usman
0
Reputation points
answered
2026-10-14T00:15:19.5966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 63%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMK%3C/text%3E%3C/svg%3E)
Maraiah Kasetti (Wipro Limited)
85
Reputation points Microsoft External Staff
Moderator
4 answers
"Incompatible data type mapping" only in Flowlet
I'm building a flowlet to transform data from a DL2 pool of Json files to a Parquet sink. First I built a traditional dataflow with the exact same transformation on DL2 datasets for 4 different countries. That works like a charm, but I would like to…
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,666 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2022-01-29T06:56:14.883+00:00
Axel B Andersen
186
Reputation points
commented
2026-10-13T20:55:56.5866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 86%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGJ%3C/text%3E%3C/svg%3E)
Gaston Jacinto
0
Reputation points
0 answers
How to pass arrays/complex types as Input Columns to Flowlets in Azure Data Factory
Hello everyone! I've got an issue when trying to pass arrays as Input Columns to Flowlets in ADF. I'll be attaching images of how Flowlet is called in the Dataflow and the actual error message I see when I try to Validate the Dataflow. Image 1: how…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-10-12T20:51:01.4766667+00:00
Gaston Jacinto
0
Reputation points
1 answer
How do you handle incremental loads in Azure Data Factory when different sources have different change patterns?
When building an Azure data platform, scheduled loads are fairly straightforward. Incremental loads can be more complicated when each source handles changes differently. For example, some systems may have a reliable timestamp or change indicator, while…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-10-01T14:56:52.1666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 69%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKR%3C/text%3E%3C/svg%3E)
Kavika Roy
0
Reputation points
edited an answer
2026-10-12T11:47:17.28+00:00
Amira Bedhiafi
43,046
Reputation points MVP
Volunteer Moderator
1 answer
ADF PostgreSQL Metadata Lookup fails with Npgsql connection pool exhausted despite only ~20 PostgreSQL connections
ADF PostgreSQL Metadata Lookup fails with Npgsql connection pool exhausted despite low PostgreSQL connections Post I’m troubleshooting an Azure Data Factory (ADF) pipeline where a PostgreSQL Metadata Lookup activity is failing with an Npgsql connection…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-10-05T06:49:21.4466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(32, 86%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJD%3C/text%3E%3C/svg%3E)
Jyothi Devanabanda
0
Reputation points
answered
2026-10-12T11:44:47.2233333+00:00
Amira Bedhiafi
43,046
Reputation points MVP
Volunteer Moderator
1 answer
Lookup Activity in ForEach in ADF using batchcount shows most activities with the same duration while they should all be different.
For one of my customers I have a pipeline that executes lots of stored procedures in an Azure SQL database using a ForEach with a batch count of 50. The only activity is a LookUp with a variable stored procedure (and parameters). I used to be able to…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-10-08T09:37:36.0166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 9%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAH%3C/text%3E%3C/svg%3E)
Aart Hopman
0
Reputation points
commented
2026-10-09T08:56:29.18+00:00
Aart Hopman
0
Reputation points
2 answers
To Deploy Azure ARM template which is greater than 4MB from one environment to another environment
Hi, I was trying to move a Azure data factory from DEV to QA using ARM template which has a size of 20 MB. When i try to export the ARM template from DEV data factory and try to import it in QA data factory using Azure custom deployment, It fails to…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2024-07-02T10:13:45.4633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 56.00000000000001%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Srihari B
20
Reputation points
answered
2026-10-09T02:10:30.9333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 48%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EUT%3C/text%3E%3C/svg%3E)
Umesh Tiwari
0
Reputation points
1 answer
SERVER NODE issue when connection to SAP HANA through Linked Service in Azure Data Factory
I cannot get past this error when connecting to SAP HANA cloud test system via SAP HANA linked service in Azure Data Factory: Error code 9603 Details ERROR [HY000] [SAP AG][LIBODBCHDB DLL][HDBODBC] General error;-10719 Connect failed (invalid…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-10-05T09:56:07.19+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 84%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAD%3C/text%3E%3C/svg%3E)
Azure Demo
0
Reputation points
answered
2026-10-05T10:04:02.3566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(9.6, 52%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Senthil kumar
2,915
Reputation points
2 answers
MS Fabric// Unable to connect with Lakehouse with Data gen2.
Unable to add the Lakehouse as Destination to DATAGen2 getting this error message. What can be the reason
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-09-22T06:45:43.2333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 89%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVN%3C/text%3E%3C/svg%3E)
Verma, Namrata
0
Reputation points
edited an answer
2026-10-03T12:38:05.65+00:00
CrazyKats
20,125
Reputation points
Volunteer Moderator
1 answer
ADF Copy Activity Issue - Copy sql data -> ADLS -> Snowflake
Hi, We got a data flow with Azure managed sql instance db as source. CDC is enabled (SQL Server CDC) in the source with Allow schema drift and Infer drifted column types. Sink is the ADLS gen 2 type as dataset and allow schema drift. There is derived…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-10-02T10:23:39.63+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 28.999999999999996%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Anil Kestur
5
Reputation points
answered
2026-10-02T11:16:04.46+00:00
Senthil kumar
2,915
Reputation points
1 answer
How to Ingest a table of xml data from on-prem DB2 warehouse to ASDLGen2 using ADF
Hi, I am using ADF to incrementally load a table containing 2 XML columns from on-prem DB2 warehouse to ADSLGen2 in parquet format. In Copy activity, I use DB2 connector at source and parquet dataset at sink; I also use Query with XMLSERIALIZE for these…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-09-16T14:13:37.9933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(249.60000000000002, 23%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENP%3C/text%3E%3C/svg%3E)
Nhan, Phuong
0
Reputation points
commented
2026-09-30T17:17:09.08+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 64%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHY%3C/text%3E%3C/svg%3E)
Himaja Y
560
Reputation points Microsoft External Staff
Moderator
1 answer
Restrict Non-Owners from changing the Collaboration branch of Azure Data Factory using Azure Policy
We would like to impose the following restrictions on Azure Data Factory. We were able to achieve the #1 feature using the below policy { "mode": "All", "policyRule": { "if": { …
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2023-02-22T05:51:14.7966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 97%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Praveen Sreeram
11
Reputation points
edited a comment
2026-09-28T11:35:32.23+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 22%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
JuliaMarvin
23,510
Reputation points
Volunteer Moderator
2 answers
Trying to filter csv file based on content
I have an existing ADF pipeline that is creating an error due to some bad (or not acceptable data) displayed on the csv file. Basically, instead of having proper header on the first row with certain amount columns (with data), csv file displays data like…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-07-22T21:45:14.68+00:00
Justin Doh
1,000
Reputation points
commented
2026-09-26T01:45:13.7933333+00:00
Justin Doh
1,000
Reputation points
1 answer
How can I implement incremental API ingestion from Odoo ERP to Azure Data Lake Storage Gen2 using Azure Data Factory?
I'm working for a project on an Azure data platform where Odoo ERP is the source and ADLS Gen2 is used as the data lake. The Odoo API contains manufacturing, inventory, procurement, and financial data that needs to be refreshed regularly for Power BI…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,214 questions
asked
2026-09-25T14:43:28.43+00:00
Kavika Roy
0
Reputation points
answered
2026-09-25T15:01:47.1033333+00:00
Allan Solomon Mejia
10,545
Reputation points