Common Issues in Library Installation and Uninstallation

This topic explains issues you may encounter while uninstalling or upgrading libraries in a Azure Databricks cluster.

Problem

  • Libraries installed via the API do not show up in the UI
  • Deleting a library in UI does not uninstall the library
  • The status of libraries in the UI may be duplicated with inconsistent statuses

Cause

The issue is seen when there are a large number of libraries in the cluster and a library installed through the cluster UI is deleted through REST API or vice versa. If the installation and uninstallation are done through the same mechanism then this issue won’t happen.

Solution

Follow the below steps to resolve the issues:

  1. Detach the library from the cluster.
  2. Move the library to the Trash folder.
  3. Delete the library from the Trash folder.
  4. If the library was installed using REST API, use the REST API to: list the libraries installed on the cluster and then remove the library.
  5. Restart the cluster.
  6. Re-add the library.
  7. Attach the library to the cluster.
  8. Detach and reattach the cluster from the notebook.