1

There was Update method in Spark 1.3.1

https://spark.apache.org/docs/1.3.1/api/java/org/apache/spark/mllib/linalg/DenseMatrix.html

But in Spark 1.6.0, there is no Update method

https://spark.apache.org/docs/1.6.0/api/java/org/apache/spark/mllib/linalg/DenseMatrix.html

My idea is to store large set of elements in a distributed matrix and perform operation on that, How to update values in DenseMatrix ??

Any example or suggestions are welcome!

zero323
  • 322,348
  • 103
  • 959
  • 935
purplebee
  • 61
  • 6

1 Answers1

1
  1. org.apache.spark.mllib.linalg.DenseMatrix - is not a distributed data structure but a local one, implemented using plain Java arrays. If you're looking for a local data structures which can be efficiently updated consider using Breeze
  2. distributed matrices (mllib.linalg.distributed.DistributedMatrix) are, same as other distributed data structures in Spark, immutable.
zero323
  • 322,348
  • 103
  • 959
  • 935
  • Thanks for update, i checked docs: http://spark.apache.org/docs/latest/mllib-data-types.html#local-matrix , Lets say, if i use **DenseMatrix**, then how should i update its elements. – purplebee Feb 05 '16 at 12:01
  • You shouldn't :) Theoretically you can modify Java array directly but don't do it. – zero323 Feb 05 '16 at 12:13