Direct Lake overview

Direct Lake is a storage mode option for tables in a Power BI semantic model that's stored in a Microsoft Fabric workspace. It's optimized for large volumes of data that can be quickly loaded into memory from Delta tables stored in OneLake—the single store for all analytics data. Once loaded into memory, the semantic model enables high performance interactive analysis.

Direct Lake is ideal for semantic models connecting to large Fabric lakehouses, warehouses, and other sources with Delta tables, especially when replicating the entire data volume into an Import model is challenging or impossible. Direct Lake queries are, like Import mode, processed by the VertiPaq query engine, whereas DirectQuery federates queries to the underlying data source. This means Direct Lake queries, like Import mode, normally outperform DirectQuery.

However, a Direct Lake differs from an Import mode in an important way: a refresh operation for a Direct Lake semantic model is conceptually different to a refresh operation for an Import semantic model. Import mode replicates the data and creates an entire cached copy of the data for the semantic model, whereas a Direct Lake refresh copies only metadata (known as framing, described later in this article), which can take a few seconds to complete. The Direct Lake refresh is a low-cost operation that analyzes the metadata of the latest version of the Delta tables and is updated to reference the latest files in OneLake. In contrast, for an Import refresh produces a copy of the data, which can take considerable time and consume significant data source and capacity resources (memory and CPU). Direct Lake moves data preparation to OneLake and in doing so uses the full breadth of Fabric technologies for data prep, including Spark jobs, T-SQL DML statements, dataflows, pipelines, and more.

Direct Lake storage mode offers the following key benefits:

• Similar to Import mode, Direct Lake queries are processed by the VertiPaq engine, and thus delivers query performance comparable to Import mode without the management overhead of data refresh cycles to load the entire data volume.
• Uses existing Fabric investments by seamlessly integrating with large lakehouses, warehouses, and other Fabric sources with Delta tables. For example, Direct Lake is an ideal choice for the gold analytics layer in the medallion lakehouse architecture.
• Maximizes Return on Investment (ROI) because analyzed data volumes can exceed the capacity’s max memory limits, since only the data that’s needed to answer a query is loaded into memory.
• Minimizes data latencies by quickly and automatically synchronizing a semantic model with its sources, making new data available to business users without refresh schedules.

When should you use Direct Lake storage mode?

The primary use case for Direct Lake storage mode is typically for IT-driven analytics projects that use lake-centric architectures. In such scenarios, you have, or expect to accumulate, large volumes of data in OneLake. The fast loading of that data into memory, frequent and fast refresh operations, efficient use of capacity resources, and fast query performance are all important for this use case.

Direct Lake depends on data preparation being done in the data lake. Data preparation can be done by using various tools, such as Spark jobs for Fabric lakehouses, T-SQL DML statements for Fabric warehouses, dataflows, pipelines, and others, which helps ensure data preparation logic is performed upstream in the architecture to maximize reusability. However, if the semantic model author doesn't have the ability to modify the source item, for example if a self-service analyst doesn't have write permissions on a lakehouse that is managed by IT, then augmenting the model with Import storage mode tables might be a good choice, since Import mode supports data preparation by using Power Query, which is defined as part of semantic model.

Be sure to consider your current Fabric capacity license and the Fabric capacity guardrails when you consider Direct Lake storage mode. Also, factor in the considerations and limitations, which are described later in this article.

Key concepts and terminology

This article assumes familiarity with the following concepts:

• Users interact with visuals in Power BI reports, which generate DAX queries to the semantic model.
• Storage mode: The semantic model processes the DAX queries differently depending on the storage mode employed. For example:
  • Import and Direct Lake storage modes use the VertiPaq engine to process DAX queries and return results to the Power BI report and user.
  • DirectQuery, on the other hand, translates DAX queries to the query syntax of the data source (typically a form of SQL) and federates them to the underlying database. Source database query processors are often not geared to BI-style, aggregated queries and therefore result in slower performance and reduced user interactivity when compared to Import and Direct Lake modes.

Storage mode is a property of a table in the semantic model. When a semantic model includes tables with different storage modes, it's referred to as a composite model. For more information about storage modes, see Semantic model modes in the Power BI service.

• Direct Lake mode can use two different access methods:

  • Direct Lake on OneLake doesn't depend on SQL endpoints and can use data from any Fabric data source with Delta tables. Direct Lake on OneLake doesn't fall back to DirectQuery mode.

  Note

  Direct Lake on OneLake is currently in public preview.

  • Direct Lake on SQL endpoints uses the SQL endpoint of a Fabric lakehouse or warehouse for Delta table discovery and permission checks. Direct Lake on SQL endpoints can fall back to DirectQuery mode when it can’t load the data directly from a Delta table, such as when the data source is a SQL view or when the Warehouse uses SQL-based Row-level Security (RLS). Direct Lake on SQL endpoints is generally available and fully supported in production.

Comparison of storage modes

The following table compares Direct Lake storage mode to Import and DirectQuery storage modes.

¹ When using Direct Lake on SQL endpoints, you can't combine Direct Lake storage mode tables with DirectQuery or Dual storage mode tables in the same semantic model. However, you can use Power BI Desktop to create a composite model on a Direct Lake semantic model and then extend it with new tables (by using Import, DirectQuery, or Dual storage mode) or calculations. For more information, see Build a composite model on a semantic model.

² Requires a V2 embed token. If you're using a service principal, you must use a fixed identity cloud connection.

How Direct Lake works

Typically, queries sent to a Direct Lake semantic model are handled from an in-memory cache of the columns sourced from Delta tables. The underlying storage for a Delta table is one or more Parquet files in OneLake. Parquet files organize data by columns rather than rows. Semantic models load entire columns from Delta tables into memory as they're required by queries.

Direct Lake on OneLake isn't coupled with the SQL endpoint, offering tighter integration with OneLake features such as OneLake security and more efficient DAX query plans because, for example, checking for SQL based security isn't required. DirectQuery fallback isn't supported by Direct Lake on OneLake.

With Direct Lake on SQL endpoints, a DAX query might use DirectQuery fallback, which involves seamlessly switching to DirectQuery mode. DirectQuery fallback retrieves data directly from the SQL analytics endpoint of the lakehouse or the warehouse. For example, fallback occurs when SQL based security is detected in the SQL endpoint. In this case, a DirectQuery operation sends a query to the SQL analytics endpoint. Fallback operations might result in slower query performance.

The following sections describe Direct Lake concepts and features, including column loading, framing, automatic updates, and DirectQuery fallback.

Column loading (transcoding)

Direct Lake semantic models only load data from OneLake as and when columns are queried for the first time. The process of loading data on-demand from OneLake is known as transcoding.

When the semantic model receives a DAX (or Multidimensional Expressions—MDX) query, it first determines what columns are needed to produce a query result. Any column directly used by the query is needed, and also columns required by relationships and measures. Typically, the number of columns needed to produce a query result is significantly smaller than the number of columns defined in the semantic model.

Once it understands which columns are needed, the semantic model determines which columns are already in memory. If any columns needed for the query aren't in memory, the semantic model loads all data for those columns from OneLake. Loading column data is typically a fast operation, however it can depend on factors such as the cardinality of data stored in the columns.

Columns loaded into memory are then resident in memory. Future queries that involve only resident columns don't need to load any more columns into memory.

A column remains resident until there's reason for it to be removed (evicted) from memory. Reasons that columns might get removed include:

• The model or table was refreshed after a Delta table update at the source (see Framing in the next section).
• No query used the column for some time.
• Other memory management reasons, including memory pressure in the capacity due to other, concurrent operations.

Your choice of Fabric SKU determines the maximum available memory for each Direct Lake semantic model on the capacity. For more information about resource guardrails and maximum memory limits, see Fabric capacity guardrails and limitations later in this article.

Framing

Framing provides model owners with point-in-time control over what data is loaded into the semantic model. Framing is a Direct Lake operation triggered by a refresh of a semantic model, and in most cases takes only a few seconds to complete. That's because it's a low-cost operation where the semantic model analyzes the metadata of the latest version of the Delta Lake tables and is updated to reference the latest Parquet files in OneLake.

When framing occurs, resident table column segments and dictionaries might be evicted from memory if the underlying data has changed and the point in time of the refresh becomes the new baseline for all future transcoding events. From this point, Direct Lake queries only consider data in the Delta tables as of the time of the most recent framing operation. For that reason, Direct Lake tables are queried to return data based on the state of the Delta table at the point of the most recent framing operation. That time isn't necessarily the latest state of the Delta tables.

The semantic model analyzes the Delta log of each Delta table during framing to drop only the affected column segments and to reload newly added data during transcoding. An important optimization is that dictionaries will usually not be dropped when incremental framing takes effect, and new values are added to the existing dictionaries. This incremental framing approach helps to reduce the reload burden and benefits query performance. In the ideal case, when a Delta table received no updates, no reload is necessary for columns already resident in memory and queries show far less performance impact after framing because incremental framing essentially enables the semantic model to update substantial portions of the existing in-memory data in place.

The following diagram shows how Direct Lake framing operations work.

The diagram depicts the following processes and features.

Item	Description
	A semantic model exists in a Fabric workspace.
	Framing operations take place periodically, and they set the baseline for all future transcoding events. Framing operations can happen automatically, manually, on schedule, or programmatically.
	OneLake stores metadata and Parquet files, which are represented as Delta tables.
	The last framing operation includes Parquet files related to the Delta tables, and specifically the Parquet files that were added before the last framing operation.
	A later framing operation includes Parquet files added after the last framing operation.
	Resident columns in the Direct Lake semantic model might be evicted from memory, and the point in time of the refresh becomes the new baseline for all future transcoding events.
	Subsequent data modifications, represented by new Parquet files, aren't visible until the next framing operation occurs.

It's not always desirable to have data representing the latest state of any Delta table when a transcoding operation takes place. Consider that framing can help you provide consistent query results in environments where data in Delta tables is transient. Data can be transient for several reasons, such as when long-running extract, transform, and load (ETL) processes occur.

Refresh for a Direct Lake semantic model can be done manually, automatically, or programmatically. For more information, see Refresh Direct Lake semantic models.

Automatic updates

There's a semantic model-level setting to automatically update Direct Lake tables. It's enabled by default. It ensures that data changes in OneLake are automatically reflected in the Direct Lake semantic model. You should disable automatic updates when you want to control data changes by framing, which was explained in the previous section. For more information, see Manage Direct Lake semantic models.

DirectQuery fallback

When using Direct Lake on SQL endpoints, a query sent to a Direct Lake semantic model can fall back to DirectQuery mode in which case the table no longer operates in Direct Lake mode. It retrieves data directly from the SQL analytics endpoint of the lakehouse or warehouse. Such queries always return the latest data because they're not constrained to the point in time of the last framing operation.

When DirectQuery fallback occurs, a query no longer uses Direct Lake mode. A query can't leverage Direct Lake mode when the semantic model queries a view in the SQL analytics endpoint, or a table in the SQL analytics endpoint that enforces row-level security (RLS). Also, a query can't leverage Direct Lake mode when a Delta table exceeds the guardrails of the capacity.

You can control fallback of your Direct Lake semantic models by setting its DirectLakeBehavior property. This setting only applies to Direct Lake on SQL endpoints. Direct Lake on OneLake doesn't support DirectQuery fallback. For more information, see Set the Direct Lake behavior property.

Data security and access permissions

By default, Direct Lake uses single sign-on (SSO), which means that the identity that queries the semantic model (often a report user) must be authorized to access the data. Alternatively, you can bind a Direct Lake model to a sharable cloud connection (SCC) to provide a fixed identity and disable SSO. In this case, only the fixed identity requires read access to the data in the source.

Fabric item permissions

Direct Lake semantic models adhere to a layered security model. They perform permission checks to determine whether the identity attempting to access the data has the necessary data access permissions in the source data item and the semantic model. Permissions can be assigned directly or acquired implicitly using workspace roles in Microsoft Fabric.

It's important to know that Direct Lake on OneLake and Direct Lake on SQL endpoints perform permission checks differently.

• Direct Lake on OneLake requires Read and ReadAll permissions on the lakehouse/warehouse to access Delta tables.
• Direct Lake on SQL endpoints requires Read and ReadData permissions on the lakehouse/warehouse to access data from the SQL analytics endpoint.

Semantic model permissions

In addition to Fabric item permissions, you must also grant permissions to users so that they can use or manage the Direct Lake semantic model. In short, report consumers need Read permission, and report creators need additional Build permission. Semantic model permissions can be assigned directly or acquired implicitly using workspace roles. To manage the semantic model settings (for refresh and other configurations), you must be the semantic model owner.

Permission requirements

Consider the following scenarios and permission requirements.

For more information, see Semantic model permissions.

Fabric capacity requirements

Direct Lake semantic models require a Fabric capacity license. Also, there are capacity guardrails and limitations that apply to your Fabric capacity subscription (SKU), as presented in the following table.

¹ For Direct Lake semantic models, Max Memory represents the upper memory resource limit for how much data can be paged in. For this reason, it's not a guardrail because exceeding it doesn't result in a fallback to DirectQuery mode; however, it can have a performance impact if the amount of data is large enough to cause excessive paging in and out of the model data from the OneLake data.

If exceeded, the Max model size on disk/OneLake causes all queries to the semantic model to fall back to DirectQuery mode. All other guardrails presented in the table are evaluated per query. It's therefore important that you optimize your Delta tables and Direct Lake semantic model to avoid having to unnecessarily scale up to a higher Fabric SKU.

Additionally, Capacity unit and Max memory per query limits apply to Direct Lake semantic models. For more information, see Capacities and SKUs.

Considerations and limitations

Direct Lake semantic models present some considerations and limitations.

Related content

• Develop Direct Lake semantic models
• Manage Direct Lake semantic models
• Understand storage for Direct Lake semantic models
• Create a lakehouse for Direct Lake
• Analyze query processing for Direct Lake semantic models