Questions tagged [matrix-multiplication]

Questions related to matrix multiplication, especially implementation. Mathematical questions should consider the [linear-algebra] tag.

Matrix multiplication is usually the process of multiplying two (or more) matrices. This arises quite frequently in linear algebra contexts and is a particularly fundamental task in computing, especially in scientific computing.

To that end, a number of fundamental libraries, such as LAPACK, BLAS, ATLAS, and others have been developed. Because the growth of matrices affects the computational time, extensive effort has been made to optimize these packages for various computer architectures and various matrix sizes.

In scientific software for statistical computing and graphics, operator %*% performs matrix multiplication (see ?"%*%"), and interfaces BLAS routine dgemmm.


The product of the multiplication of two matrices a and b is the matrix c, where each element is the sum of the products of the i-th row of a and the j-th column of b.

c[i][j] += a[i][k] * b[k][j];

Example:

     (b) { 1,  2,  3,  4}
         { 5,  6,  7,  8}
(a)    ┌(c)──────────────
{1, 2} │ {11, 14, 17, 20}
{3, 4} │ {23, 30, 37, 44}
{5, 6} │ {35, 46, 57, 68}

Algorithm of the matrix multiplication:

// rows of 'a' matrix
int m = 3;
// columns of 'a' matrix
// and rows of 'b' matrix
int n = 2;
// columns of 'b' matrix
int p = 4;
// matrices 'a=m×n', 'b=n×p'
int[][] a = {{1, 2}, {3, 4}, {5, 6}},
        b = {{1, 2, 3, 4}, {5, 6, 7, 8}};
// resulting matrix 'c=m×p'
int[][] c = new int[m][p];
// iterate over the rows of the 'a' matrix
for (int i = 0; i < m; i++) {
    // iterate over the columns of the 'b' matrix
    for (int j = 0; j < p; j++) {
        // iterate over the columns of the 'a'
        // matrix, aka rows of the 'b' matrix
        for (int k = 0; k < n; k++) {
            // sum of the products of
            // the i-th row of 'a' and
            // the j-th column of 'b'
            c[i][j] += a[i][k] * b[k][j];
        }
    }
}
2901 questions
15
votes
3 answers

How to compute only the diagonal of a matrix product in Octave?

Is there a way in Octave to compute and store only the diagonal of a matrix product? Basically like doing: vector = diag(A*B); I don't care about any of the values of A*B except those on the diagonal. The matrix sizes are around 80k x 12 and 12 x…
Chris H
  • 6,433
  • 5
  • 33
  • 51
15
votes
3 answers

Matrix multiplication, solve Ax = b solve for x

So I was given a homework assignment that requires solving the coefficients of cubic splines. Now I clearly understand how to do the math on paper as well as with MatLab, I want to solve the problem with Python. Given an equation Ax = b where I know…
Scalahansolo
  • 2,615
  • 6
  • 26
  • 43
15
votes
1 answer

compute only diagonals of matrix multiplication in R

I need only the diagonal elements from a matrix multiplication: , in R. As Z is huge I want to avoid the full out multiplication.... Z <- matrix(c(1,1,1,2,3,4), ncol = 2) Z # [,1] [,2] #[1,] 1 2 #[2,] 1 3 #[3,] 1 4 X <-…
guyabel
  • 8,014
  • 6
  • 57
  • 86
14
votes
1 answer

Faster evaluation of matrix multiplication from right to left

I noticed that evaluating matrix operations in quadratic form from right to left is significantly faster than left to right in R, depending on how the parentheses are placed. Obviously they both perform the same amount of calculation. I am wondering…
Taotao Tan
  • 273
  • 1
  • 8
14
votes
8 answers

What is the best matrix multiplication algorithm?

What is the best matrix multiplication algorithm? What means 'the best'for me? It means the fastest and ready for todays machines. Please give links to pseudocode if you can.
guest
  • 1,696
  • 4
  • 20
  • 31
14
votes
3 answers

Row Sum of a dot product for huge matrix in python

I have 2 matrix 100kx200 and 200x100k if they were small matrix I would just use numpy dot product sum(a.dot(b), axis = 0) however the matrix is too big, and also I can't use loops is there a smart way for doing this?
Aya Abdelsalam
  • 502
  • 3
  • 8
  • 21
14
votes
1 answer

Difference between numpy.dot and a.dot(b)

Is there a difference between import numpy as np np.dot(a,b) and a.dot(b) internally? I wasn't able to find any documentation on the latter method.
McLawrence
  • 4,975
  • 7
  • 39
  • 51
14
votes
2 answers

numpy elementwise outer product

I want to do the element-wise outer product of two 2d arrays in numpy. A.shape = (100, 3) # A numpy ndarray B.shape = (100, 5) # A numpy ndarray C = element_wise_outer_product(A, B) # A function that does the trick C.shape = (100, 3, 5) # This…
14
votes
1 answer

Why is matrix multiplication faster with Repa than with hmatrix?

It's interesting that Data.Array.Repa is actually faster than hmatrix, which is unexpected since hmatrix is implemented using LAPACK. Is this because Repa uses the unboxed type? import Data.Array.Repa import Data.Array.Repa.Algorithms.Matrix main =…
kai
  • 235
  • 1
  • 8
14
votes
5 answers

Why is a naïve C++ matrix multiplication 100 times slower than BLAS?

I am taking a look at large matrix multiplication and ran the following experiment to form a baseline test: Randomly generate two 4096x4096 matrixes X, Y from std normal (0 mean, 1 stddev). Z = X*Y Sum elements of Z (to make sure they are accessed)…
Andrew Tomazos
  • 66,139
  • 40
  • 186
  • 319
14
votes
2 answers

Why is my Strassen's Matrix Multiplication slow?

I wrote two Matrix Multiplications programs in C++: Regular MM (source), and Strassen's MM (source), both of which operate on square matrices of sizes 2^k x 2^k(in other words, square matrices of even size). Results are just terrible. For 1024 x…
newprint
  • 6,936
  • 13
  • 67
  • 109
13
votes
8 answers

Speed up float 5x5 matrix * vector multiplication with SSE

I need to run a matrix-vector multiplication 240000 times per second. The matrix is 5x5 and is always the same, whereas the vector changes at each iteration. The data type is float. I was thinking of using some SSE (or similar) instructions. I am…
Enzo
  • 964
  • 1
  • 9
  • 20
13
votes
3 answers

How to write a matrix matrix product that can compete with Eigen?

Below is the C++ implementation comparing the time taken by Eigen and For Loop to perform matrix-matrix products. The For loop has been optimised to minimise cache misses. The for loop is faster than Eigen initially but then eventually becomes…
Adhvaitha
  • 251
  • 2
  • 9
12
votes
3 answers

Efficient multiplication of very large matrices in MATLAB

I don't have enough memory to simply create a diagonal D-by-D matrix, since D is large. I keep getting an 'out of memory' error. Instead of performing M x D x D operations in the first multiplication, I do M x D operations, but still my code takes…
matcheek
  • 4,887
  • 9
  • 42
  • 73
12
votes
3 answers

How can I multiply a vector and a matrix in tensorflow without reshaping?

This: import numpy as np a = np.array([1, 2, 1]) w = np.array([[.5, .6], [.7, .8], [.7, .8]]) print(np.dot(a, w)) # [ 2.6 3. ] # plain nice old matrix multiplication n x (n, m) -> m import tensorflow as tf a = tf.constant(a, dtype=tf.float64) w…
Mr_and_Mrs_D
  • 32,208
  • 39
  • 178
  • 361