Share via


Monitor materialized views in Databricks SQL

This article describes how to monitor and query refresh data about a materialized view in Databricks SQL.

View details in Catalog Explorer

The easiest way to view information about your materialized view is to view it in Catalog Explorer. Catalog Explorer also allows you to see refresh history, including logs, and to view the status of the current refresh.

There are a few properties of the materialized view that are not available in Catalog Explorer. For those properties, or to get the information programmatically, you can use the DESCRIBE EXTENDED command.

View materialized view details with DESCRIBE EXTENDED

You can view details about an materialized view programmatically by using the DESCRIBE EXTENDED command. This includes:

  • The status of the latest completed refresh.
  • The refresh schedule.
  • The columns of the materialized view.
  • The data size for the materialized view, in total bytes (not available in Catalog Explorer).
  • The storage location of the materialized view (not available in Catalog Explorer).
-- As table:
DESCRIBE TABLE EXTENDED sales;

-- As a single JSON object:
DESCRIBE TABLE EXTENDED sales AS JSON;

View the refresh history for a materialized view

To view the status of REFRESH operations on a materialized view, including current and past refreshes, query the DLT event log:

SELECT
  *
FROM
  event_log(TABLE(my_catalog.my_schema.sales))
WHERE
  event_type = "update_progress"
ORDER BY
  timestamp desc;

Replace <fully-qualified-table-name> with the fully qualified name of the materialized view, including the catalog and schema.

See What is the DLT event log?.

View the type of refresh that occurred for a materialized view

Some queries can be incrementally refreshed. If an incremental refresh cannot be performed, a full refresh is performed instead.

To see the refresh type for a materialized view, query the event log:

SELECT
  timestamp,
  message
FROM
  event_log(TABLE(my_catalog.my_schema.sales))
WHERE
  event_type = 'planning_information'
ORDER BY
  timestamp desc;

Sample output for this command:

    • timestamp
    • message
    • 2025-03-21T22:23:16.497+00:00
    • Flow 'sales' has been planned in DLT to be executed as ROW_BASED.

For details on refresh types, and how to optimize your refreshes, see Incremental refresh for materialized views.

Monitor runs using query history

You can use the query history page to access query details and query profiles that can help you identify poorly performing queries and bottlenecks in the DLT pipeline used to run your streaming table updates. For an overview of the kind of information available for query histories and query profiles, see Query history and Query profile.

Important

This feature is in Public Preview. Workspace admins can enable this feature from the Previews page. See Manage Azure Databricks Previews.

All statements related to materialized views appear in the query history. You can use the Statement drop-down filter to select any command and inspect the related queries. All CREATE statements are followed by a REFRESH statement that executes asynchronously on a DLT pipeline. The REFRESH statements typically include detailed query plans that provide insights into optimizing performance.

To access REFRESH statements in the query history UI, use the following steps:

  1. Click History Icon in the left sidebar to open the Query History UI.
  2. Select the REFRESH checkbox from the Statement drop-down filter.
  3. Click the name of the query statement to view summary details like the duration of the query and aggregated metrics.
  4. Click See query profile to open the query profile. For details about navigating the query profile, see Query profile.
  5. Optionally, use the links in the Query Source section to open the related query or pipeline.

See CREATE MATERIALIZED VIEW.

Troubleshoot a failed refresh

Note

To troubleshoot a refresh that is full when you believe it should be incremental, first check that you have row tracking enabled for any source Delta tables. For other details about incremental refresh, see Support for materialized view incremental refresh.

A refresh operation can fail for a number of different reasons. For example:

  • The owner of the materialized view may lose SELECT access to the source tables.
  • The source tables may have a schema change that is incompatible with the materialized view query definition.

To troubleshoot a failed materialized view refresh:

  1. Click Catalog icon Catalog in the sidebar.
  2. In the Catalog Explorer tree at the left, open the catalog and select the schema where your materialized view is located.
  3. Click the See refresh details link in the Overview tab.
  4. In the pipeline details page, from the Event log tab at the bottom, click Error to filter down to just errors in processing.
  5. Click an error to see details, including the exact error message.

This is a sample error message:

org.apache.spark.sql.AnalysisException: [AMBIGUOUS_REFERENCE] Reference `customer_name` is ambiguous, could be: ['p'.'customer_name', 'u'.'customer_name']. SQLSTATE: 42704