Authorize unattended access to Azure Databricks resources with a service principal using OAuth

Статья
2025-04-12

В этом разделе приведены инструкции и сведения об авторизации доступа к ресурсам Azure Databricks при автоматизации выполнения команд Azure Databricks CLI или вызове REST API Azure Databricks из кода, который будет выполняться в рамках ненаблюдаемого процесса.

Azure Databricks использует OAuth в качестве предпочтительного протокола для авторизации пользователей и проверки подлинности при взаимодействии с ресурсами Azure Databricks за пределами пользовательского интерфейса. Azure Databricks также предоставляет единое средство проверки подлинности клиента для автоматизации обновления маркеров доступа, созданных в рамках метода проверки подлинности OAuth. Это относится к субъектам-службам, а также к учетным записям пользователей, но необходимо настроить субъект-службу с соответствующими разрешениями и привилегиями для ресурсов Azure Databricks, к которым он должен обращаться в рамках своих операций.

Дополнительные сведения см. в статье Авторизация доступа к ресурсам Azure Databricks.

Каковы мои варианты авторизации и аутентификации при использовании служебного принципала Azure Databricks?

In this topic, authorization refers to the protocol (OAuth) used to negotiate access to specific Azure Databricks resources through delegation. Authentication refers to the mechanism by which credentials are represented, transmitted, and verified—which, in this case, are access tokens.

Azure Databricks uses OAuth 2.0-based authorization to enable access to Azure Databricks account and workspace resources from the command line or code on behalf of a service principal with the permissions to access those resources. После настройки субъекта-службы Azure Databricks и проверки его учетных данных во время выполнения команды CLI или вызова REST API, токен OAuth предоставляется инструменту или пакету SDK для осуществления аутентификации на основе токенов от имени субъекта-службы на постоянной основе. Маркер доступа OAuth имеет срок действия одного часа, после которого средство или пакет SDK, участвующий, сделает автоматическую фоновую попытку получить новый маркер, который также действителен в течение одного часа.

Azure Databricks поддерживает два способа авторизации доступа для субъекта-службы с помощью OAuth:

Mostly automatically, using the Databricks unified client authentication support. Use this simplified approach if you are using specific Azure Databricks SDKs (such as the Databricks Terraform SDK) and tools. Supported tools and SDKs are listed in Databricks unified client authentication. This approach is well-suited to automation or other unattended process scenarios.
Manually, by directly generating an OAuth code verifier/challenge pair and an authorization code, and using them to create the initial OAuth token you will provide in your configuration. Use this approach when you are not using an API supported by Databricks unified client authentication. В этом случае может потребоваться разработать собственный механизм для обработки обновления маркеров доступа, относящихся к стороннему инструменту или API, который вы используете. For more details, see: Manually generate and use access tokens for OAuth service principal authentication.

Перед началом работы необходимо настроить учетную запись службы Azure Databricks и назначить ей соответствующие разрешения для доступа к ресурсам, которые нужно использовать, когда ваш автоматизированный код или команды запрашивают их.

Prerequisite: Create a service principal

Account admins and workspace admins can create service principals. This step describes creating a service principal in a Azure Databricks workspace. For details on the Azure Databricks account console itself, see Manage service principals in your account.

You can also create an an Microsoft Entra ID managed service principal and add it to Azure Databricks. For more information, see Databricks and Microsoft Entra ID service principals.

As a workspace admin, log in to the Azure Databricks workspace.
Click your username in the top bar of the Azure Databricks workspace and select Settings.
Click on the Identity and access tab.
Next to Service principals, click Manage.
Click Add service principal.
Click the drop-down arrow in the search box and then click Add new.
Under Management, choose Databricks managed.
Enter a name for the service principal.
Click Add.

The service principal is added to both your workspace and the Azure Databricks account.

Step 1: Assign permissions to your service principal

Click the name of your service principal to open its details page.
On the Configurations tab, check the box next to each entitlement that you want your service principal to have for this workspace, and then click Update.
On the Permissions tab, grant access to any Azure Databricks users, service principals, and groups that you want to manage and use this service principal. See Manage roles on a service principal.

Шаг 2. Создайте секрет OAuth для служебного принципала

Прежде чем использовать OAuth для авторизации доступа к ресурсам Azure Databricks, необходимо сначала создать секрет OAuth, который можно использовать для создания маркеров доступа OAuth для проверки подлинности. A service principal can have up to five OAuth secrets.

OAuth secrets have a maximum lifetime of two years. Account admins and workspace admins can create an OAuth secret for a service principal.

On your service principal’s details page click the Secrets tab.
Under OAuth secrets, click Generate secret.
Set the secret's lifetime in days. OAuth secrets have a maximum lifetime of 730 days (two years).
Copy the displayed Secret and Client ID, and then click Done.

The secret will only be revealed once during creation. The client ID is the same as the service principal’s application ID.

Администраторы учетных записей также могут создать OAuth-секрет на странице с информацией о служебном принципале в консоли учетной записи.

As an account admin, log in to the account console.
На боковой панели щелкните " Управление пользователями".
On the Service principals tab, select your service principal.
Under OAuth secrets, click Generate secret.
Set the secret's lifetime in days. OAuth secrets have a maximum lifetime of 730 days (two years).
Copy the displayed Secret and Client ID, and then click Done.

Note

To enable the service principal to use clusters or SQL warehouses, you must give the service principal access to them. See Compute permissions or Manage a SQL warehouse.

Шаг 3. Использование авторизации OAuth

Чтобы использовать авторизацию OAuth с унифицированным средством аутентификации клиента, необходимо задать следующие связанные переменные окружения, поля .databrickscfg, поля Terraform или поля Config.

The Azure Databricks host, specified as https://accounts.azuredatabricks.net for account operations or the target per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net for workspace operations.
The Azure Databricks account ID, for Azure Databricks account operations.
The service principal client ID.
The service principal secret.

Чтобы выполнить аутентификацию субъекта службы OAuth, интегрируйте в код следующие компоненты в зависимости от используемого инструмента или пакета SDK.

Environment

Сведения об использовании переменных среды для определенного типа проверки подлинности Azure Databricks с помощью средства или пакета SDK см. в статье авторизация доступа к ресурсам Azure Databricks или документации по средству или пакету SDK. See also Environment variables and fields for unified client authentication and the Default methods for client unified authentication.

For account-level operations, set the following environment variables:

DATABRICKS_HOST, set to the Azure Databricks account console URL, https://accounts.azuredatabricks.net.
DATABRICKS_ACCOUNT_ID
DATABRICKS_CLIENT_ID
DATABRICKS_CLIENT_SECRET

Для операций на уровне рабочей областизадайте следующие переменные среды:

DATABRICKS_HOST, set to the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net.
DATABRICKS_CLIENT_ID
DATABRICKS_CLIENT_SECRET

Profile

Create or identify an Azure Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values. Сведения об использовании профиля с инструментом или пакетом SDK см. в статье Авторизация доступа к ресурсам Azure Databricks или документации по средству или пакету SDK. See also Environment variables and fields for unified client authentication and the Default methods for client unified authentication.

Для операций уровня учетной записи установите следующие значения в файле .databrickscfg. In this case, the Azure Databricks account console URL is https://accounts.azuredatabricks.net:

[<some-unique-configuration-profile-name>]
host          = <account-console-url>
account_id    = <account-id>
client_id     = <service-principal-client-id>
client_secret = <service-principal-secret>

For workspace-level operations, set the following values in your .databrickscfg file. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net:

[<some-unique-configuration-profile-name>]
host          = <workspace-url>
client_id     = <service-principal-client-id>
client_secret = <service-principal-secret>

CLI

For the Databricks CLI, do one of the following:

Задайте переменные среды, указанные в разделе "Среда".
Задайте значения в файле .databrickscfg, как указано в разделе "Профиль" этой статьи.

Environment variables always take precedence over values in your .databrickscfg file.

Connect

Note

OAuth service principal authentication is supported on the following Databricks Connect versions:

For Python, Databricks Connect for Databricks Runtime 14.0 and above.
For Scala, Databricks Connect for Databricks Runtime 13.3 LTS and above. The Databricks SDK for Java that is included with Databricks Connect for Databricks Runtime 13.3 LTS and above must be upgraded to Databricks SDK for Java 0.17.0 or above.

For Databricks Connect, you can do one of the following:

Set the values in your .databrickscfg file for Azure Databricks workspace-level operations as specified in this article’s “Profile” section. Also set the cluster_id environment variable in your profile to your per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net.
Задайте переменные среды для операций уровня рабочей области Azure Databricks , как указано в разделе "Среда". Also set the DATABRICKS_CLUSTER_ID environment variable to your per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net.

Значения в файле .databrickscfg всегда имеют приоритет над переменными среды.

To initialize the Databricks Connect client with these environment variables or values in your .databrickscfg file, see Compute configuration for Databricks Connect.

VS Code

For the Databricks extension for Visual Studio Code, do the following:

Set the values in your .databrickscfg file for Azure Databricks workspace-level operations as specified in this article’s “Profile” section.
In the Configuration pane of the Databricks extension for Visual Studio Code, click Configure Databricks.
In the Command Palette, for Databricks Host, enter your per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net, and then press Enter.
In the Command Palette, select your target profile’s name in the list for your URL.

Дополнительные сведения см. в разделе Настройка авторизации для расширения Databricks для Visual Studio Code.

Terraform

For account-level operations, for default authentication:

provider "databricks" {
  alias = "accounts"
}

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider). In this case, the Azure Databricks account console URL is https://accounts.azuredatabricks.net:

provider "databricks" {
  alias         = "accounts"
  host          = <retrieve-account-console-url>
  account_id    = <retrieve-account-id>
  client_id     = <retrieve-client-id>
  client_secret = <retrieve-client-secret>
}

For workspace-level operations, for default authentication:

provider "databricks" {
  alias = "workspace"
}

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider). In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net:

provider "databricks" {
  alias         = "workspace"
  host          = <retrieve-workspace-url>
  client_id     = <retrieve-client-id>
  client_secret = <retrieve-client-secret>
}

For more information about authenticating with the Databricks Terraform provider, see Authentication.

Python

For account-level operations, use the following for default authentication:

from databricks.sdk import AccountClient

a = AccountClient()
# ...

Для прямой конфигурации используйте следующую команду, заменив заполнители retrieve на собственную реализацию, чтобы получить значения из консоли или другого хранилища конфигурации, например, Azure KeyVault. In this case, the Azure Databricks account console URL is https://accounts.azuredatabricks.net:

from databricks.sdk import AccountClient

a = AccountClient(
  host          = retrieve_account_console_url(),
  account_id    = retrieve_account_id(),
  client_id     = retrieve_client_id(),
  client_secret = retrieve_client_secret()
)
# ...

For workspace-level operations, specifically default authentication:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
# ...

For direct configuration, replace the retrieve placeholders with your own implementation to retrieve the values from the console, or other configuration store, such as Azure KeyVault. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient(
  host          = retrieve_workspace_url(),
  client_id     = retrieve_client_id(),
  client_secret = retrieve_client_secret()
)
# ...

For more information about authenticating with Databricks tools and SDKs that use Python and implement Databricks client unified authentication, see:

Note

Расширение Databricks для Visual Studio Code использует Python, но еще не реализовало проверку подлинности сервисного принципала OAuth.

Java

For workspace-level operations using default authentication:

import com.databricks.sdk.WorkspaceClient;
// ...
WorkspaceClient w = new WorkspaceClient();
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console, or other configuration store, such as Azure KeyVault). In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net:

import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(retrieveWorkspaceUrl())
  .setClientId(retrieveClientId())
  .setClientSecret(retrieveClientSecret());
WorkspaceClient w = new WorkspaceClient(cfg);
// ...

For more information about authenticating with Databricks tools and SDKs that use Java and implement Databricks client unified authentication, see:

настройка клиента Databricks Connect для Scala (клиент Databricks Connect для Scala использует включенный пакет SDK Databricks для Java для проверки подлинности)
Authenticate the Databricks SDK for Java with your Azure Databricks account or workspace

Go

For account-level operations using default authentication:

import (
"github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient())
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console, or other configuration store, such as Azure KeyVault). In this case, the Azure Databricks account console URL is https://accounts.azuredatabricks.net:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{
    Host:         retrieveAccountConsoleUrl(),
    AccountId:    retrieveAccountId(),
    ClientId:     retrieveClientId(),
    ClientSecret: retrieveClientSecret(),
}))
// ...

For workspace-level operations using default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient())
// ...

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient(&databricks.Config{
  Host:         retrieveWorkspaceUrl(),
  ClientId:     retrieveClientId(),
  ClientSecret: retrieveClientSecret(),
}))
// ...

For more information about authenticating with Databricks tools and SDKs that use Go and that implement Databricks client unified authentication, see Authenticate the Databricks SDK for Go with your Azure Databricks account or workspace.

Manually generate and use access tokens for OAuth service principal authentication

Azure Databricks tools and SDKs that implement the Databricks client unified authentication standard will automatically generate, refresh, and use Azure Databricks OAuth access tokens on your behalf as needed for OAuth service principal authentication.

Databricks рекомендует использовать единую проверку подлинности клиента, однако если необходимо вручную создать, обновить или использовать маркеры доступа Azure Databricks OAuth, следуйте инструкциям в этом разделе.

Use the service principal’s client ID and OAuth secret to request an OAuth access token to authenticate to both account-level REST APIs and workspace-level REST APIs. The access token will expire in one hour. You must request a new OAuth access token after the expiration. The scope of the OAuth access token depends on the level that you create the token from. You can create a token at either the account level or the workspace level, as follows:

To call account-level and workspace-level REST APIs within accounts and workspaces that the service principal has access to, manually generate an access token at the account level.
To call REST APIs within only one of the workspaces that the service principal has access to, manually generate an access token at the workspace level for only that workspace.

вручную создайте маркер доступа на уровне учетной записи

An OAuth access token created from the account level can be used against Databricks REST APIs in the account, and in any workspaces the service principal has access to.

As an account admin, log in to the account console.
Click the down arrow next to your username in the upper right corner.
Copy your Account ID.
Construct the token endpoint URL by replacing <my-account-id> in the following URL with the account ID that you copied.
```
https://accounts.azuredatabricks.net/oidc/accounts/<my-account-id>/v1/token
```
Use a client such as curl to request an OAuth access token with the token endpoint URL, the service principal’s client ID (also known as an application ID), and the service principal’s OAuth secret you created. The all-apis scope requests an OAuth access token that can be used to access all Databricks REST APIs that the service principal has been granted access to.
- Replace <token-endpoint-URL> with the preceding token endpoint URL.
- Replace <client-id> with the service principal’s client ID, which is also known as an application ID.
- Replace <client-secret> with the service principal’s OAuth secret that you created.
```
export CLIENT_ID=<client-id>
export CLIENT_SECRET=<client-secret>

  curl --request POST \
  --url <token-endpoint-URL> \
  --user "$CLIENT_ID:$CLIENT_SECRET" \
  --data 'grant_type=client_credentials&scope=all-apis'
```
This generates a response similar to:
```
{
  "access_token": "eyJraWQiOiJkYTA4ZTVjZ…",
  "token_type": "Bearer",
  "expires_in": 3600
}
```
Copy the access_token from the response.

вручную создайте маркер доступа на уровне рабочей области

An OAuth access token created from the workspace level can only access REST APIs in that workspace, even if the service principal is an account admin or is a member of other workspaces.

Construct the token endpoint URL by replacing https://<databricks-instance> with the workspace URL of your Azure Databricks deployment:
```
https://<databricks-instance>/oidc/v1/token
```
Use a client such as curl to request an OAuth access token with the token endpoint URL, the service principal’s client ID (also known as an application ID), and the service principal’s OAuth secret you created. The all-apis scope requests an OAuth access token that can be used to access all Databricks REST APIs that the service principal has been granted access to within the workspace that you are requesting the token from.
- Replace <token-endpoint-URL> with the preceding token endpoint URL.
- Replace <client-id> with the service principal’s client ID, which is also known as an application ID.
- Replace <client-secret> with the service principal’s OAuth secret that you created.
```
export CLIENT_ID=<client-id>
export CLIENT_SECRET=<client-secret>

curl --request POST \
--url <token-endpoint-URL> \
--user "$CLIENT_ID:$CLIENT_SECRET" \
--data 'grant_type=client_credentials&scope=all-apis'
```
This generates a response similar to:
```
{
  "access_token": "eyJraWQiOiJkYTA4ZTVjZ…",
  "token_type": "Bearer",
  "expires_in": 3600
}
```
Copy the access_token from the response.

Call a Databricks REST API

You can use the OAuth access token to authenticate to Azure Databricks account-level REST APIs and workspace-level REST APIs. The service principal must have account admin privileges to call account-level REST APIs.

Включите токен доступа в заголовок авторизации, используя метод аутентификации Bearer. You can use this approach with curl or any client that you build.

Example account-level REST API request

В этом примере используется проверка подлинности Bearer для получения списка всех рабочих областей, связанных с учетной записью.

Replace <oauth-access-token> with the service principal’s OAuth access token that you copied in the previous step.
Replace <account-id> with your account ID.

export OAUTH_TOKEN=<oauth-access-token>

curl --request GET --header "Authorization: Bearer $OAUTH_TOKEN" \
'https://accounts.azuredatabricks.net/api/2.0/accounts/<account-id>/workspaces'

Example workspace-level REST API request

В этом примере используется проверка подлинности Bearer для перечисления всех доступных кластеров в указанной рабочей области.

Replace <oauth-access-token> with the service principal’s OAuth access token that you copied in the previous step.
Replace <workspace-URL> with your base workspace URL, which has the form similar to dbc-a1b2345c-d6e7.cloud.databricks.com.

export OAUTH_TOKEN=<oauth-access-token>

curl --request GET --header "Authorization: Bearer $OAUTH_TOKEN" \
'https://<workspace-URL>/api/2.0/clusters/list'

Additional resources

Service principals
Overview of the Databricks identity model
Additional information about authentication and access control

Поделиться через

Authorize unattended access to Azure Databricks resources with a service principal using OAuth

Каковы мои варианты авторизации и аутентификации при использовании служебного принципала Azure Databricks?

Prerequisite: Create a service principal

Step 1: Assign permissions to your service principal

Шаг 2. Создайте секрет OAuth для служебного принципала

Шаг 3. Использование авторизации OAuth

Environment

Profile

CLI

Connect

VS Code

Terraform

Python

Java

Go

Manually generate and use access tokens for OAuth service principal authentication

вручную создайте маркер доступа на уровне учетной записи

вручную создайте маркер доступа на уровне рабочей области

Call a Databricks REST API

Example account-level REST API request

Example workspace-level REST API request

Additional resources

Обратная связь

Дополнительные ресурсы