1

I am using MLlib 1.1.0 and struggling to find a way to save my model. Docs do not seem to support such as feature in this version. Any ideas?

zero323
  • 322,348
  • 103
  • 959
  • 935
user706838
  • 5,132
  • 14
  • 54
  • 78

1 Answers1

3

There is save model option like:

// Save and load model
model.save(sc, "myModelPath")
val sameModel = LogisticRegressionModel.load(sc, "myModelPath")

But I see it starting from v1.3. I m not sure if it will be still valid for 1.1 You can try this and upgrade if it does not work??

Dr VComas
  • 735
  • 7
  • 22
  • That's right. It starts from 1.3.0 but I cannot update unfortunately. I ended up with this `sc.parallelize(List(model)).saveAsObjectFile("model.mdl")` for now - although I had to increase akka frame size to 32 MB. Any alternative? – user706838 Aug 12 '15 at 13:45
  • Umm, you want to save it for later use in another spark app? – Dr VComas Aug 12 '15 at 15:42
  • yes exactly. what is the minimum effort / code required to implement `model.save()` myself? thanks! – user706838 Aug 14 '15 at 10:02
  • I am not so sure how hard it could be. You can have a look at the code in github and try. https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala – Dr VComas Aug 14 '15 at 13:50