Edit

Share via


Manage data retention in a Log Analytics workspace

A Log Analytics workspace retains data in two states:

  • Analytics retention: In this state, data is available for monitoring, troubleshooting, and near-real-time analytics.
  • Long-term retention: In this low-cost state, data isn't available for table plan features, but can be accessed through search jobs.

This article explains how Log Analytics workspaces retain data and how to manage the data retention of tables in your workspace.

Analytics, long-term, and total retention

By default, all tables in a Log Analytics workspace retain data for 30 days, except for log tables with 90-day default retention. Tables with the Analytics plan make your data available for real-time queries during this Analytics retention period. All table plans can retrieve the stored data through queries or search jobs, and the data is available for visualizations, alerts, and other features and services, based on the table plan.

You can extend the analytics retention period of tables with the Analytics plan up to two years. Basic plan tables have a fixed period of 30 days for queries while Auxiliary plan tables can be queried for the total retention period. Both Basic and Auxiliary tables have additional considerations however. For more information, see Query data in Basic and Auxiliary tables.

Note

You can reduce the analytics retention period of Analytics tables to as little as four days using the API or CLI. However, since 31 days of analytics retention are included in the ingestion price, lowering the retention period below 31 days doesn't reduce costs.

To retain data in the same table beyond the default retention period, extend the table's total retention to up to 12 years. At the end of the analytics retention period, the data stays in the table for the remainder of the total retention period you configure. During this period - the long-term retention period - run a search job to retrieve the specific data you need from the table and make it available for interactive queries in a search results table.

Diagram that shows analytics and long-term retention in Azure Monitor Logs.

How retention modifications work

When you shorten a table's total retention, Azure Monitor Logs waits 30 days before removing the data, so you can revert the change and avoid data loss if you made an error in configuration.

When you increase total retention, the new retention period applies to all data that was already ingested into the table and wasn't yet removed.

When you change the long-term retention settings of a table with existing data, the change takes effect immediately.

Example:

  • You have an existing Analytics table with 180 days of analytics retention and no long-term retention.
  • You change the analytics retention to 90 days without changing the total retention period of 180 days.
  • Azure Monitor automatically treats the remaining 90 days of total retention as low-cost, long-term retention, so that data that's 90-180 days old isn't lost.

Permissions required

Action Permissions required
Configure default analytics retention for Analytics tables in a Log Analytics workspace Microsoft.OperationalInsights/workspaces/write and microsoft.operationalinsights/workspaces/tables/write permissions to the Log Analytics workspace, as provided by the Log Analytics Contributor built-in role, for example
Get retention setting by table for a Log Analytics workspace Microsoft.OperationalInsights/workspaces/tables/read permissions to the Log Analytics workspace, as provided by the Log Analytics Reader built-in role, for example

Configure the default analytics retention period of Analytics tables

The default retention period of Analytics tables in a Log Analytics workspace is 30 days. You can change the default analytics period of Analytics tables up to two years by modifying the workspace-level data retention setting. Basic and Auxiliary tables only have a total retention period, which is 30 days by default.

Changing the default workspace-level data retention setting automatically affects all Analytics tables to which the default setting still applies in your workspace. If you've already changed the analytics retention of a particular table, that table isn't affected when you change the workspace default data retention setting.

Important

Workspaces with 30-day retention might keep data for 31 days. If you need to retain data for 30 days only to comply with a privacy policy, configure the default workspace retention to 30 days using the API and update the immediatePurgeDataOn30Days workspace property to true. This operation is currently only supported using the Workspaces - Update API.

To set the default analytics retention period of Analytics tables within a Log Analytics workspace:

  1. From the Log Analytics workspaces menu in the Azure portal, select your workspace.

  2. In the Settings section, select Usage and estimated costs in the left pane.

  3. Select Data Retention at the top of the page.

    Screenshot that shows changing the workspace data retention setting.

  4. Move the slider to increase or decrease the number of days, and then select OK.

Configure table-level retention

By default, all tables with the Analytics data plan inherit the Log Analytics workspace's default retention setting and have no long-term retention. You can increase the analytics retention period of Analytics tables to up to 730 days at an extra cost.

To add long-term retention to a table with any data plan, set total retention to up to 12 years (4,383 days).

Note

Currently, you can set total retention to up to 12 years through the Azure portal and API. CLI and PowerShell are limited to seven years; support for 12 years will follow.

To modify the retention setting for a table in the Azure portal:

  1. From the Log Analytics workspaces menu, select Tables.

    The Tables screen lists all the tables in the workspace.

  2. Select the context menu for the table you want to configure and select Manage table.

    Screenshot that shows the Manage table button for one of the tables in a workspace.

  3. Configure the analytics retention and total retention settings in the Data retention settings section of the table configuration screen.

    Screenshot that shows the data retention settings on the table configuration screen.

Get retention settings by table

To view a table's retention settings in the Azure portal, from the Log Analytics workspaces menu, select Tables.

The Tables screen shows the analytics retention and total retention periods for all the tables in the workspace.

Screenshot that shows the Manage table button for one of the tables in a workspace.

What happens to data when you delete a table in a Log Analytics workspace?

A Log Analytics workspace can contain several types of tables. What happens when you delete the table is different for each:

Table type Data retention Recommendations Recovery
Azure table An Azure table holds logs from an Azure resource or data required by an Azure service or solution and can't be deleted. When you stop streaming data from the resource, service, or solution, data remains in the workspace until the end of the retention period defined for the table, and charged accordingly. To minimize charges, set table-level retention to four days in relevant tables before you disable a solution, Sentinel for example. Enable the solution. Data recovery is subjected to table retention.
Custom log table (table_CL) Custom log table holds logs from logs ingestion API, or HTTP data collector API (deprecated).
When you delete a table, the table name kept reserved for fourteen days and release after the period. Deleting a table in Analytics or Basic plan doesn't delete data in table, and after fourteen days, the retention inherits the workspace retention.
Deleting a table in Auxiliary plan, deletes data permanently after fourteen days, retention inherits the workspace retention, but retention charges remain and adheres to retention in tables.
To minimize charges, set table-level retention to four days before you delete the table. Analytics or Basic plans: Create the table with the same name and schema. Data recovery is subjected to table retention.
Auxiliary plan: Create the table with the same name and schema during the soft delete period.
Search results table (table_SRCH) Deletes the table and data immediately and permanently.
Restored table (table_RST) Deletes the hot cache provisioned for the restore, but source table data isn't deleted.

Log tables with 90-day default retention

By default, the Usage and AzureActivity tables keep data for at least 90 days at no charge. When you increase the workspace retention to more than 90 days, you also increase the retention of these tables. These tables are also free from data ingestion charges.

Tables related to Application Insights resources also keep data for 90 days at no charge. You can adjust the retention of each of these tables individually:

  • AppAvailabilityResults
  • AppBrowserTimings
  • AppDependencies
  • AppExceptions
  • AppEvents
  • AppMetrics
  • AppPageViews
  • AppPerformanceCounters
  • AppRequests
  • AppSystemEvents
  • AppTraces

Pricing model

The charge for adding analytics retention and long-term retention is calculated based on the volume of data you retain, in GB, and the number or days for which you retain the data. Log data that has _IsBillable == false isn't subject to ingestion or retention charges.

For more information, see Azure Monitor pricing.

Learn more about: