Stack Overflow
Stack Overflow

Questions tagged [scala-breeze]

Breeze is a linear algebra library for Scala.

Breeze is a linear algebra library for Scala.

From the project's GitHub page:

Breeze is a library for numerical processing, machine learning, and natural language processing. Its primary focus is on being generic, clean, and powerful without sacrificing (much) efficiency. Breeze is the merger of the ScalaNLP and Scalala projects, because one of the original maintainers is unable to continue development.

197 questions
4
votes
2 answers

Spark parallel processing of grouped data

Initially, I had a lot of data. But using spark-SQL and especially groupBy it could be trimmed down to a manageable size. (fits in RAM of a single node) How can I perform functions (in parallel) on all the groups (distributed among my nodes)? How…
Georg Heiler
  • 16,916
  • 36
  • 162
  • 292
4
votes
1 answer

Matrix Operation in Spark MLlib in Java

This question is about MLlib (Spark 1.2.1+). What is the best way to manipulate local matrices (moderate size, under 100x100, so does not need to be distributed). For instance, after computing the SVD of a dataset, I need to perform some matrix…
Julien
  • 3,613
  • 2
  • 23
  • 25
4
votes
2 answers

Using Breeze from Java on Spark MLlib

While trying to use MLlib from Java, what is the correct way to use breeze Matrix operations? For e.g. multiplication in scala it ist simply "matrix * vector". How is the corresponding functionality expressed in Java? There are methods like…
Mechanee
  • 41
  • 3
4
votes
2 answers

How to build a large distributed [sparse] matrix in Apache Spark 1.0?

I have an RDD as such: byUserHour: org.apache.spark.rdd.RDD[(String, String, Int)] I would like to create a sparse matrix of the data for calculations like median, mean, etc. The RDD contains the row_id, column_id and value. I have two Arrays…
Climbs_lika_Spyder
  • 6,004
  • 3
  • 39
  • 53
3
votes
1 answer

scalac :Out of memory during building Breeze

I'm trying to build Breeze by myself using Intellij IDEA Here's my enviroment Intellij version:2018.2.1 Community OS:Windows 10 64-bit JDK version:1.8.0_181 scala SDK version:2.12 sbt version: sorry.. can't find it. At First, It kept showing the…
LtChang
  • 135
  • 12
3
votes
2 answers

How to find the mean of same cells in an array of Breeze Matrices in spark scala?

I have an Array[DenseMatrix[Double]] and i want to find the mean of the same cells. For example: Array[0]: +---+---+ | 1 | 2 | +---+---+ | 2 | 3 | +---+---+ Array[1]: +---+---+ | 1 | 1 | +---+---+ | 3 | 1 | …
mkey
  • 55
  • 8
3
votes
1 answer

scala-breeze/spark replace a row of a densematrix with another densevector

I have a breeze.linalg.DenseMatrix: breeze.linalg.DenseMatrix[Int] = 1 5 9 2 6 10 3 7 11 4 8 12 and a breeze.linalg.DenseVector: breeze.linalg.DenseVector[Int] = DenseVector(13, 14, 15) Slicing allows me to get a particular…
8u88y
  • 33
  • 1
  • 6
3
votes
1 answer

Create a Breeze DenseMatrix from a List of double arrays in scala

I have a structure List[Array[Double]] and I want to convert to a DenseMatrix. I have this solution but I think there might be a better way: val data = List[Array[Double]] val rows = data.length; val cols = data(0).length; val matrix =…
MLeiria
  • 633
  • 1
  • 9
  • 22
3
votes
1 answer

Can Spark and the ScalaNLP library Breeze be used together?

I'm developing a Scala-based extreme learning machine, in Apache Spark. My model has to be a Spark Estimator and use the Spark framework in order to fit into the machine learning pipeline. Does anyone know if Breeze can be used in tandem with Spark?…
LucieCBurgess
  • 759
  • 5
  • 12
  • 26
3
votes
2 answers

How to insert vector of ones to Matrix?

I've a vector and a matrix: 1 1 0 0 0 0 I want to prepend the vector to matrix to produce : 1 0 0 1 0 0 I have so far : val dv = DenseVector(1.0,1.0); val dm = DenseMatrix.zeros[Double](2,2) Reading the API :…
blue-sky
  • 51,962
  • 152
  • 427
  • 752
3
votes
0 answers

scalanlp breeze linear algebra: zip two vectors together

I'm used to working with lists. If I have two lasts A and B and a function that takes two doubles and returns a double, then I could do the following A.zip(B).map(x=>myfunction(x._1,x._2)) This is the way my brain is used to thinking about these…
Lindon
  • 1,292
  • 1
  • 10
  • 21
3
votes
0 answers

For loop over a DenseVector does not work

I am using a function returning a DenseVector but since yesterday I am not able to read its values as before. Here is the snippet: val clusters = stsc() for (i <- 0 until clusters.length) { print(clusters(i)) } It returns…
Armand Grillet
  • 3,229
  • 5
  • 30
  • 60
3
votes
2 answers

Implementing flatMap on aggregate monad

I'm looking to implement a version of Generator combinators (e.g. analogous to those in ScalaCheck or Haskell's QuickCheck), in which a Generator contains an instance of Rand, a monad representing a probability distribution (taken from the breeze…
NietzscheanAI
  • 966
  • 6
  • 16
3
votes
1 answer

How to initialize a specific random seed in scala breeze, say for Gaussian distribution?

How do I force the random number generator (if possible) with a specific seed in scala breeze, so say the following scala code always generate the same sequence seq? import breeze.stats.distributions._ val g = new Gaussian(0, 1) …
Carson Pun
  • 1,742
  • 2
  • 13
  • 20
3
votes
1 answer

Scala Breeze prefix scalar multiplication

When using Breeze do I have to import some implicit conversions to make prefix scalar multiplication work? val v = DenseVector(1.0, 2.0, 3.0) val r = 2.0 * v The above doesn't work, where as: val r = v * 2.0 Does work.
The Sockmonster
  • 189
  • 1
  • 8
1
2
3
…
13 14