Exporting and importing ML Models and Pipelines with Databricks ML Model Export

Databricks ML Model Export is used to export models and full ML pipelines from Apache Spark. These exported models and pipelines can be imported into other (Spark and non-Spark) platforms to do scoring and make predictions.

Model Export is targeted at low-latency, lightweight ML-powered applications. With Model Export, you can:

  • Use an existing model deployment system
  • Achieve very low latency (milliseconds)
  • Use ML models and pipelines in custom deployments

The scoring (a.k.a. inference) library takes JSON-encoded features.

{"id":5923937,  // any metadata
"features:": { // MLlib vector format: 0 for sparse vector, 1 for dense vector
   "type": 1,
   "values":[0.1, 1.3, 8.4, 4.2]}}

The result is also encoded in JSON.

{"id":5923937,
 "prediction": 1.0}