Sometimes accessing data requires that you authenticate to external data sources, such as Azure Data Lake Store, through JDBC. Instead of directly typing your credentials into a notebook, use Azure Databricks secret management to store your credentials and reference them in notebooks and jobs. To manage secrets, you can use the Databricks CLI to access the Secrets API.
Secret management requires Databricks Runtime 4.0 or above and Databricks CLI 0.7.1 or above.
To set up secrets you:
- Create a secret scope.
- Add secrets to the scope.
- If you have the Premium SKU, assign access control to the secret scope.
This guide shows you how to perform these setup tasks and manage secrets using the Databricks CLI.
For more information, see:
- An end-to-end example of how to use secret management in your workflows: Secret Management Workflow Example.
- Reference information about the Databricks CLI: Secrets CLI.
- Reference information about the API: Secrets API.
- How to use Databricks Utilities to reference secrets in notebooks and jobs: Secrets utilities.
In this guide:
- Secret Scopes
- Secret Access Control
- Notebook Secret Redaction
- Secret Management Workflow Example