Exporting and Importing ML Models

After developing ML models, the next step is productionizing the trained models. A typical workflow of the productionization in Azure Databricks involves the steps:

  1. Export a trained model.
  2. Import the model into an external system.

Azure Databricks supports two methods to export and import models and full ML pipelines from Apache Spark: MLeap and Databricks ML Model Export.

MLeap, which Azure Databricks recommends, is a common serialization format and execution engine for machine learning pipelines. It supports serializing Apache Spark, scikit-learn, and TensorFlow pipelines into a bundle, so you can load and deploy your trained models to make predictions with new data. You can also use Databricks ML Model Export to export models and ML pipelines. These exported models and pipelines can be imported into other (Spark and non-Spark) platforms to do scoring and make predictions.

The older feature for exporting models, Databricks ML Model Export, is deprecated in Databricks Runtime 5.3 and is removed in Databricks Runtime 6.0.