Metrics help you monitor the performance of Azure Databricks clusters.
You have these options:
You can configure an Azure Databricks cluster to send metrics to a Log Analytics workspace in Azure Monitor, the monitoring platform for Azure. For complete instructions, see Monitoring Azure Databricks.
If you have deployed the Azure Databricks workspace in your own virtual network and you have configured network security groups (NSG) to deny all outbound traffic that is not required by Azure Databricks, then you must configure an additional outbound rule for the “AzureMonitor” service tag.
You can install Datadog agents on cluster nodes to send Datadog metrics to your Datadog account. The following notebook demonstrates how to install a Datadog agent on a cluster using a cluster-scoped init script.
To install the Datadog agent on all clusters, use a global init script after testing the cluster-scoped init script.