Secrets API (Preview)

The Secrets API, which is in Preview, allows you to create and manage secrets, secret scopes, and access permissions. To learn how to use the Secrets API, see Secrets (Preview).

You access and reference secrets in notebooks and jobs by using the DBUtils SecretUtils API, also in Preview.


Create Secret Scope

Endpoint HTTP Method
2.0/preview/secret/scopes/create POST

Creates a new secret scope.

The scope name must consist of alphanumeric characters, dashes, underscores, and periods, and may not exceed 128 characters. The maximum number of scopes in a workspace is 100.

Example request:

{
    "scope": "my-simple-databricks-scope",
    "initial_manage_acl": "creator_only"|"all_users"
}

If you specify creator_only, the initial ACL applied to the scope is MANAGE permission, assigned to the request issuer’s user id (discoverable from the API token). If you specify all_users, the initial ACL applied to the scope is MANAGE permission, assigned to the group “all users”.

Throws RESOURCE_ALREADY_EXISTS if a scope with the given name already exists. Throws RESOURCE_LIMIT_EXCEEDED if maximum number of scopes in the workspace is exceeded. Throws INVALID_PARAMETER_VALUE if the scope name is invalid.

Request Structure

Field Name Type Description
scope STRING Scope name requested by the user. Scope names are unique. This field is required.
initial_manage_acl InitialManageAcl The initial ACL rule applied to the scope. If not specified, will default to CREATOR_ONLY.

Delete Secret Scope

Endpoint HTTP Method
2.0/preview/secret/scopes/delete POST

Deletes a secret scope. Returns message that the scope was successfully deleted.

Example request:

{
    "scope": "my-secret-scope"
}

Throws RESOURCE_DOES_NOT_EXIST if the scope does not exist. Throws PERMISSION_DENIED if the user does not have permission to make this API call.

Request Structure

Field Name Type Description
scope STRING Name of the scope to delete. This field is required.

Response Structure

Field Name Type Description
deleted BOOL Indicates whether the scope was deleted successfully. This will be false if requested against a non-existent scope. This field is required.

List Secret Scopes

Endpoint HTTP Method
2.0/preview/secret/scopes/list GET

Lists all secret scopes available in the workspace.

Example response:

{
    "scopes": [{
        "name": "my-databricks-scope",
        "backend_type": "databricks"
    },{
        "name": "mount-points",
        "backend_type": "databricks"
    }]
}

Throws PERMISSION_DENIED if the user does not have permission to make this API call.

Response Structure

Field Name Type Description
scopes An array of SecretScope The available secret scopes.

Write Secret

Endpoint HTTP Method
2.0/preview/secret/secrets/write POST

Inserts a secret under the provided scope with the given name. If a secret already exists with the same name, this command overwrites the existing secret’s value. The server encrypts the secret using the secret scope’s encryption settings before storing it. You must have WRITE or MANAGE permission on the secret scope.

The secret key must consist of alphanumeric characters, dashes, underscores, and periods, and cannot exceed 128 characters. The maximum allowed secret value size is 128 KB. The maximum number of secrets in a given scope is 1000.

Example request:

{
    "scope": "my-databricks-scope",
    "key": "my-string-key",
    "string_value": "foobar"
}

The input fields “string_value” or “bytes_value” specify the type of the secret, which will determine the value returned when the secret value is requested. Exactly one must be specified.

Throws RESOURCE_DOES_NOT_EXIST if no such secret scope exists. Throws RESOURCE_LIMIT_EXCEEDED if maximum number of secrets in scope is exceeded. Throws INVALID_PARAMETER_VALUE if the key name or value length is invalid. Throws PERMISSION_DENIED if the user does not have permission to make this API call.

Request Structure

Field Name Type Description
string_value OR bytes_value STRING OR BYTES

If string_value, if specified, note that the value will be stored in UTF-8 (MB4) form.

If bytes_value, if specified, value will be stored as bytes.

scope STRING The name of the scope to which the secret will be associated with. This field is required.
key STRING A unique name to identify the secret. This field is required.

Delete Secret

Endpoint HTTP Method
2.0/preview/secret/secrets/delete POST

Deletes the secret stored in this secret scope. You must have WRITE or MANAGE permission on the secret scope. Returns if the secret was successfully deleted. This returns false if the secret does not exist.

Example request:

{
    "scope": "my-secret-scope",
    "key": "my-secret-key"
}

Throws RESOURCE_DOES_NOT_EXIST if no such secret scope exists. Throws PERMISSION_DENIED if the user does not have permission to make this API call.

Request Structure

Field Name Type Description
scope STRING The name of the scope that contains the secret to delete. This field is required.
key STRING Name of the secret to delete. This field is required.

Response Structure

Field Name Type Description
deleted BOOL Indicates whether the secret was deleted successfully. This will be false if requested against a non-existent secret. This field is required.

List Secrets

Endpoint HTTP Method
2.0/preview/secret/secrets/list GET

Lists the secret keys that are stored at this scope. This is a metadata-only operation; secret data cannot be retrieved using this API. Users need the READ permission to make this call.

Example response:

{
    "secrets": [
        {
            "key": "my-string-key"",
            "last_updated_timestamp": "1520467595000"
        },
        {
            "key": "my-byte-key",
            "last_updated_timestamp": "1520467595000"
        },
    ]
}

The lastUpdatedTimestamp returned is in milliseconds since epoch.

Throws RESOURCE_DOES_NOT_EXIST if no such secret scope exists. Throws PERMISSION_DENIED if the user does not have permission to make this API call.

Request Structure

Field Name Type Description
scope STRING The name of the scope whose secrets you want to list. This field is required.

Response Structure

Field Name Type Description
secrets An array of SecretMetadata Metadata information of all secrets contained within the given scope.

Write Secret ACL

Endpoint HTTP Method
2.0/preview/secret/acls/write POST

Creates or overwrites the ACL associated with the given principal (user or group) on the specified scope point. In general, a user or group will use the most powerful permission available to them, and permissions are ordered as follows:

  • MANAGE - Allowed to change ACLs, and read and write to this secret scope.
  • WRITE - Allowed to read and write to this secret scope.
  • READ - Allowed to read this secret scope and list what secrets are available.

Note that in general, secret values can only be read from within a command on a cluster (for example, through a notebook). There is no API to read the actual secret value material outside of a cluster. However, the user’s permission will be applied based on who is executing the command, and they must have at least READ permission.

Users must have the MANAGE permission to invoke this API.

Example request:

{
    "scope": "my-secret-scope",
    "principal": "data-scientists",
    "permission": "READ"
}

The principal is a user or group name corresponding to an existing Databricks principal to be granted or revoked access.

Throws RESOURCE_DOES_NOT_EXIST if no such secret scope exists. Throws RESOURCE_ALREADY_EXISTS if a permission for the principal already exists. Throws INVALID_PARAMETER_VALUE if the permission is invalid. Throws PERMISSION_DENIED if the user does not have permission to make this API call.

Request Structure

Field Name Type Description
scope STRING The name of the scope to apply permissions to. This field is required.
principal STRING The principal to which the permission is applied.
permission AclPermission The permission level applied to the principal.

Delete Secret ACL

Endpoint HTTP Method
2.0/preview/secret/acls/delete POST

Deletes the given ACL on the given scope.

Users must have the MANAGE permission to invoke this API.

Example request:

{
    "scope": "my-secret-scope",
    "principal": "data-scientists"
}

Throws RESOURCE_DOES_NOT_EXIST if no such secret scope or principal exists. Throws PERMISSION_DENIED if the user does not have permission to make this API call.

Request Structure

Field Name Type Description
scope STRING The name of the scope to remove permissions from. This field is required.
principal STRING The principal to remove an existing ACL from. This field is required.

Get Secret ACL

Endpoint HTTP Method
2.0/preview/secret/acls/get GET

Describes the details about the given ACL, such as the group and permission.

Users must have the MANAGE permission to invoke this API.

Example response:

{
    "principal": "data-scientists",
    "permission": "READ"
}

Throws RESOURCE_DOES_NOT_EXIST if no such secret scope exists. Throws PERMISSION_DENIED if the user does not have permission to make this API call.

Request Structure

Field Name Type Description
scope STRING The name of the scope to fetch ACL information from. This field is required.
principal STRING The principal to fetch ACL information for. This field is required.

Response Structure

Field Name Type Description
principal STRING The principal to which the permission is applied.
permission AclPermission The permission level applied to the principal.

List Secret ACLs

Endpoint HTTP Method
2.0/preview/secret/acls/list GET

Lists the ACLs set on the given scope.

Users must have the MANAGE permission to invoke this API.

Example response:

{
    "acls: [{
        "principal": "admins",
        "permission": "MANAGE"
    },{
        "principal": "data-scientists",
        "permission": "READ"
    }]
}

Throws RESOURCE_DOES_NOT_EXIST if no such secret scope exists. Throws PERMISSION_DENIED if the user does not have permission to make this API call.

Request Structure

Field Name Type Description
scope STRING The name of the scope to fetch ACL information from. This field is required.

Response Structure

Field Name Type Description
items An array of AclItem The associated ACLs rule applied to principals in the given scope.

Data Structures

AclItem

An item representing an ACL rule applied to the given principal (user or group) on the associated scope point.

Field Name Type Description
principal STRING The principal to which the permission is applied.
permission AclPermission The permission level applied to the principal.

SecretMetadata

The metadata about a secret. Returned when listing secrets. Does not contain the actual secret value.

Field Name Type Description
key STRING A unique name to identify the secret.
last_updated_timestamp INT64 The last updated timestamp (in milliseconds) for the secret.

SecretScope

An organizational resource for storing secrets. Secret scopes can be different types (Databricks-managed, Azure KeyVault backed, etc), and ACLs can be applied to control permissions for all secrets within a scope.

Field Name Type Description
name STRING A unique name to identify the secret scope.
backend_type ScopeBackendType The type of secret scope backend.

AclPermission

The ACL permission levels for Secret ACLs applied to secret scopes.

READ Allowed to perform read operations (get, list) on secrets in this scope.
WRITE Allowed to read and write secrets to this secret scope.
MANAGE Allowed to read/write ACLs, and read/write secrets to this secret scope.

InitialManageAcl

Initial ACL options when a scope is created.

CREATOR_ONLY Scope is created with MANAGE permissions for only the user who created the scope.
ALL_USERS Scope is created with MANAGE permissions for the all-users group.

ScopeBackendType

The types of secret scope backends in the Secret Manager. Azure KeyVault-backed secret scopes will be supported in a later release.

DATABRICKS A secret scope in which secrets are stored in Databrick managed storage and encrypted with a cloud-based specific encryption key.
AZURE_KEYVAULT A customer Azure KeyVault-backed secret scope. Reading secrets from this scope will directly read secrets from the customer vault. Only scope and secret ACL metadata are stored in Databricks.