How to Calculate the Number of Cores in a Cluster

If your organization has installed a metrics service on your cluster nodes, you can view the number of cores in an Azure Databricks cluster in the Workspace UI using the Metrics tab on the cluster details page.

You can also determine the number of cores available in a cluster programmatically, using Scala utility code:

  1. Use sc.statusTracker.getExecutorInfos.length to get the total number of nodes. The result includes the driver node, so subtract 1.

  2. Use java.lang.Runtime.getRuntime.availableProcessors to get the number of cores per node.

  3. Multiply both results (subtracting 1 from the total number of nodes) to get the total number of cores available:

    java.lang.Runtime.getRuntime.availableProcessors * (sc.statusTracker.getExecutorInfos.length -1)


This method works only if the driver and executors are of the same node type.