Importing Models into Your Application

This section discusses the import and scoring parts of a ML Model Export workflow; see Exporting Apache Spark ML Models and Pipelines for the export part of the workflow.

To use models exported via Databricks ML Model Export, you call APIs in the library dbml-local. This library provides Scala and Java APIs for importing models and performing low latency scoring (a.k.a. prediction or inference).

Using saved models in Java applications

Suppose that you have exported a logistic regression pipeline and saved it under my_models/lr_pipeline. You can use ModelFactory to create a LocalModel from the saved model’s directory and perform scoring on new data.

// Load exported model
String modelPath = "my_models/lr_pipeline";
LocalModel model = ModelFactory.loadModel(modelPath);

// The model input is a standard JSON string.
// The input schema here is: [origLabel: Double, features: Vector].
String input =
  "{\"origLabel\":-1.0," +
  "\"features\":{\"type\":0,\"size\":13," +
  "\"indices\":[0,2,3,4,6,7,8,9,10,11,12]," +
  "\"values\":[74.0,2.0,120.0,269.0,2.0,121.0,1.0,0.2,1.0,1.0,3.0]}" +
  "}";

// The model output is also a standard JSON string, with the expected output fields.
String output = model.transform(input);

The input accepts the same JSON format produced by Apache Spark Datasets and DataFrames using the Dataset.toJSON method (see Dataset API docs). See the dbml-local API docs for more details.

Specifying the dbml-local library dependency in Maven

You specify the dbml-local library dependency to an application just like any other dependency, with a Maven coordinate. The following code snippet gives an example of including dbml-local in a Maven project pom.xml build file.

<!-- Add repository for dbml-local dependency -->
 <repositories>
   <repository>
     <snapshots>
       <enabled>false</enabled>
     </snapshots>
     <id>bintray-databricks-maven</id>
     <name>bintray</name>
     <url>https://dl.bintray.com/databricks/maven</url>
   </repository>
 </repositories>

 <dependencies>
   <!-- Main dependency for Model Scoring -->
   <dependency>
     <groupId>com.databricks</groupId>
     <artifactId>dbml-local</artifactId>
     <version>0.2.2-spark2.2</version>
   </dependency>
 </dependencies>

Downloading dbml-local JARs

The dbml-local JARs are available from bintray.

dbml-local license

The dbml-local library is published under the MIT license.

Example application

You can view a very simple example application that shows how to use the Databricks ML Model Export companion library dbml-local in the databricks-ml-examples Github repository. This demo includes Databricks notebooks for training and exporting MLlib models. These notebooks are paired with simple Java applications that show how to import models and make predictions.