1,492 questions with Azure Data Lake Storage tags

Sort by: Updated
1 answer

Azure ML | Support for ADLS Gen2 Datastore with SAS Token Authentication

Hello, I’m currently working on an application where we’re connecting various data sources—such as file shares and ADLS Gen2—to Azure Machine Learning. While we can create file-share datastores with SAS token authentication, I noticed that this option…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,980 questions
asked 2024-11-15T08:37:53.9266667+00:00
Pål Hannus 0 Reputation points
edited a comment 2024-11-20T08:28:22.0366667+00:00
Pål Hannus 0 Reputation points
0 answers

Getting "storage_root does not specify a URI scheme." while setting ADLS path in default metastore.

Hi Team, I am facing while setting the ADLS path for the existing default metastore in account console page in azure. Below is the image showing i have already created the container and have enabled Hierarchical namespace as well. But getting below…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
asked 2024-11-18T11:01:44.11+00:00
Akancha 0 Reputation points
commented 2024-11-19T19:36:24.87+00:00
Keshavulu Dasari 1,805 Reputation points Microsoft Vendor
1 answer

Data Quality issue in the Purview

Hello Team, We have configured the ADLS2 as source and scan it. For the data Quality, we did the following the steps : Create the Governance domain and publish it. Create the Data Product and add the tables. In the Data Quality section, add the…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Microsoft Purview
Microsoft Purview
A Microsoft data governance service that helps manage and govern on-premises, multicloud, and software-as-a-service data. Previously known as Azure Purview.
1,241 questions
asked 2024-11-11T15:53:43.47+00:00
Dhaval Pujara 0 Reputation points
edited a comment 2024-11-19T01:03:02.2+00:00
Chandra Boorla 3,380 Reputation points Microsoft Vendor
2 answers

Failure happened on 'destination' side. ErrorCode=DeltaInvalidCharacterInColumnName.

This exception occurred when I use the pipeline of data factory to copy data from sql server to lakehouse. But I didn't find any problems with the raw data.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,902 questions
asked 2024-03-18T13:42:24.5+00:00
Xavier Yan 0 Reputation points
commented 2024-11-14T15:48:33.4933333+00:00
asktocedsq 20 Reputation points
1 answer One of the answers was accepted by the question author.

error code 2011

i am testing a pipeline, i introduced a repeated row in one of the files that i want to upload. i was expecting that the pipeline would have run anyway, uploading the correct files and not the incorrect one. actually, the entire pipeline did not work...…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,902 questions
asked 2024-11-11T16:28:19.3633333+00:00
Galvan, Elena (Bip Group) 20 Reputation points
commented 2024-11-14T07:31:57.2466667+00:00
Galvan, Elena (Bip Group) 20 Reputation points
1 answer

In MS fabric users able to view data but cannot download it

In MS Fabric, I want users to be able to view data in the workspace but not be able to download it. Please provide clear steps, along with links to verify the authenticity of the solution provided.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,902 questions
SharePoint
SharePoint
A group of Microsoft Products and technologies used for sharing and managing content, knowledge, and applications.
10,888 questions
Microsoft Fabric Training
Microsoft Fabric Training
Microsoft Fabric: A Microsoft unified data platform.Training: Instruction to develop new skills.
51 questions
asked 2024-11-11T11:51:04.92+00:00
Jain, Shreyans 0 Reputation points
commented 2024-11-13T18:06:00.42+00:00
Ganesh Gurram 1,510 Reputation points Microsoft Vendor
1 answer

Copy Files from sharepoint online site to azure datalake storage

Hello We are trying to setup the flow which will copy files from sharepoint online site to azure datalake storage. As per my understanding there are 2 options : Using ADF to pull the files as mentioned in the link below…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,902 questions
SharePoint
SharePoint
A group of Microsoft Products and technologies used for sharing and managing content, knowledge, and applications.
10,888 questions
asked 2024-11-06T22:12:34.5+00:00
Sourav 105 Reputation points
edited the question 2024-11-13T08:57:32.6533333+00:00
Givary-MSFT 33,391 Reputation points Microsoft Employee
1 answer

504.0 GatewayTimeout & Invoking Azure function failed with HttpStatusCode - 499.

We've developed an Azure Function in python that connect to a Blob Storage, reads files and writes into in Azure tables. During the process, using Azure Functions & it's running fine for small size files (Less than 100 MB). The problem is that, when…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,902 questions
asked 2024-11-12T09:44:30.44+00:00
syamprasad kari 0 Reputation points
answered 2024-11-12T13:21:53.56+00:00
Amira Bedhiafi 26,656 Reputation points
1 answer

Alternative Methods for Capturing Data Lake Size in Less Time

Need assistance in capturing the size of the data lake per environment (e.g., Dev, SIT, Prod). Currently, a PowerShell script is used to fetch details, generating a CSV file for each environment with the medallion, folder, subfolder, and size. The…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
PowerShell
PowerShell
A family of Microsoft task automation and configuration management frameworks consisting of a command-line shell and associated scripting language.
2,630 questions
asked 2024-11-11T15:45:58.24+00:00
Arianne Chung 0 Reputation points
answered 2024-11-11T19:39:38.5766667+00:00
Keshavulu Dasari 1,805 Reputation points Microsoft Vendor
2 answers One of the answers was accepted by the question author.

Transforming JSON files using data flow

Hello! I currently have about 60 json files inside a blob container which most of them have different fields and values. I have created a pipeline with a get metadata activity that points to the container, with the field list set to Child items. I have…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,902 questions
asked 2024-11-07T15:23:15.93+00:00
A K 45 Reputation points
commented 2024-11-11T19:05:56.01+00:00
Keshavulu Dasari 1,805 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

Data lake solutions

We are in the process of Data Lake and going further down the line we are really getting confused whether to go for delta lake , datalakehouse, or synapse analytics. The subtle  nuances making things not easier  such as  " A Data Lake House merges…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
asked 2024-11-08T17:01:54.56+00:00
azure_learner 420 Reputation points
accepted 2024-11-11T14:31:57.8033333+00:00
azure_learner 420 Reputation points
1 answer One of the answers was accepted by the question author.

Why function could not find file

Hi there, I built an Azure Function to process json data from external requests, and then saved the json to a local file, uploaded it to the Container through the storage client. It worked well locally, and once deployed to Azure, it would prompt that…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
5,150 questions
asked 2024-11-08T16:55:08.6266667+00:00
zmsoft 115 Reputation points
accepted 2024-11-09T00:02:47.1366667+00:00
zmsoft 115 Reputation points
2 answers One of the answers was accepted by the question author.

Data lake schema enforcement

Hello, In Data Lake data is processed or ingested as schema on read and that is data is read in it format that it comes from the source. But I read an article that says schema enforcement makes data lakes high-performance and data readable. Please…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
asked 2024-11-03T14:25:43.58+00:00
azure_learner 420 Reputation points
accepted 2024-11-08T17:03:00.9133333+00:00
azure_learner 420 Reputation points
4 answers

How to send a mail notification for a failed pipeline in Azure Synapse Analytics?

How can I send a notification email to a specific email address without using a logic app when one of my Synapse Analytics pipelines fails? I would like to include the error message in the email notification.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,940 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,010 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,902 questions
asked 2024-03-20T15:48:58.53+00:00
Aditya Singh 160 Reputation points
answered 2024-11-07T21:37:08.3566667+00:00
Manuel Dones 0 Reputation points
1 answer

How cn i restrict someone from not downloading any data from Fabric Lakehouse or Fabric warehouse

As a data admin, i want to control data access for a user in your Microsoft Fabric Warehouse. The goal is to allow this user, who has a Contributor role, to view data directly in the workspace without being able to download it as a file. This scenario…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Microsoft Fabric Training
Microsoft Fabric Training
Microsoft Fabric: A Microsoft unified data platform.Training: Instruction to develop new skills.
51 questions
asked 2024-11-05T05:24:13.73+00:00
Jain, Shreyans 0 Reputation points
commented 2024-11-07T18:46:34.07+00:00
Smaran Thoomu 17,360 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

Getting Issue while upload files on azure data lake storage

I have two application first application for frontend :- which is responsible to take file from local storage and send that files azure functions via API 2nd application for backend:- which is responsible to take file from form-data (multipart) and…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
asked 2024-11-07T06:28:01.47+00:00
Akshay Bahir 20 Reputation points
accepted 2024-11-07T11:36:48.01+00:00
Akshay Bahir 20 Reputation points
2 answers

Azure resource type - microsoft.datalakestore/accounts

Hi Team, There is a azure resource type - "microsoft.datalakestore/accounts" What exactly are these resources? How is it different from the storage accounts (microsoft.storage/storageaccounts)? Thanks. Regards, Nagesh CL

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
asked 2023-07-11T10:32:56.07+00:00
Nagesh CL 686 Reputation points
commented 2024-11-06T16:08:45.37+00:00
FlorisZ 1 Reputation point
1 answer

How cn i restrict someone from not downloading any data from Fabric Lakehouse or Fabric warehouse

As a data admin, i want to control data access for a user in your Microsoft Fabric Warehouse. The goal is to allow this user, who has a Contributor role, to view data directly in the workspace without being able to download it as a file. This scenario…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Microsoft Fabric Training
Microsoft Fabric Training
Microsoft Fabric: A Microsoft unified data platform.Training: Instruction to develop new skills.
51 questions
asked 2024-11-05T05:14:23.56+00:00
Jain, Shreyans 0 Reputation points
commented 2024-11-06T03:04:55.42+00:00
Hari Babu Vattepally 710 Reputation points Microsoft Vendor
1 answer

Copy activity failed because you have multiple concurrent copy activities runs writing to the same file

Hi All,   I am migrating 1000 of SQL tables from on premise SQL server to Azure Blob storage. I am using ADF for each and copy activity to do so. However, while processing the tables in parallel/concurrent, I am getting below error -     Failure happened…

Azure Database Migration service
Azure Files
Azure Files
An Azure service that offers file shares in the cloud.
1,309 questions
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,940 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,902 questions
asked 2024-10-24T11:00:55.3666667+00:00
Magarde, A (Ankit) 0 Reputation points
commented 2024-11-04T14:47:01.6266667+00:00
AnnuKumari-MSFT 33,476 Reputation points Microsoft Employee
3 answers One of the answers was accepted by the question author.

PATCH method not allowed for Storage Services REST API "Path - Update"

I try to use the Set Access Control option described in https://learn.microsoft.com/en-us/rest/api/storageservices/data-lake-storage-gen2?view=rest-storageservices-datalakestoragegen2-2019-12-12. Although I was able to run other methods (as Path - List)…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,492 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,940 questions
asked 2024-10-22T09:31:41.02+00:00
Scowl 25 Reputation points
accepted 2024-11-04T12:54:23.3333333+00:00
Scowl 25 Reputation points