Questions tagged [matrix-multiplication]

Questions related to matrix multiplication, especially implementation. Mathematical questions should consider the [linear-algebra] tag.

Matrix multiplication is usually the process of multiplying two (or more) matrices. This arises quite frequently in linear algebra contexts and is a particularly fundamental task in computing, especially in scientific computing.

To that end, a number of fundamental libraries, such as LAPACK, BLAS, ATLAS, and others have been developed. Because the growth of matrices affects the computational time, extensive effort has been made to optimize these packages for various computer architectures and various matrix sizes.

In scientific software for statistical computing and graphics, operator %*% performs matrix multiplication (see ?"%*%"), and interfaces BLAS routine dgemmm.


The product of the multiplication of two matrices a and b is the matrix c, where each element is the sum of the products of the i-th row of a and the j-th column of b.

c[i][j] += a[i][k] * b[k][j];

Example:

     (b) { 1,  2,  3,  4}
         { 5,  6,  7,  8}
(a)    ┌(c)──────────────
{1, 2} │ {11, 14, 17, 20}
{3, 4} │ {23, 30, 37, 44}
{5, 6} │ {35, 46, 57, 68}

Algorithm of the matrix multiplication:

// rows of 'a' matrix
int m = 3;
// columns of 'a' matrix
// and rows of 'b' matrix
int n = 2;
// columns of 'b' matrix
int p = 4;
// matrices 'a=m×n', 'b=n×p'
int[][] a = {{1, 2}, {3, 4}, {5, 6}},
        b = {{1, 2, 3, 4}, {5, 6, 7, 8}};
// resulting matrix 'c=m×p'
int[][] c = new int[m][p];
// iterate over the rows of the 'a' matrix
for (int i = 0; i < m; i++) {
    // iterate over the columns of the 'b' matrix
    for (int j = 0; j < p; j++) {
        // iterate over the columns of the 'a'
        // matrix, aka rows of the 'b' matrix
        for (int k = 0; k < n; k++) {
            // sum of the products of
            // the i-th row of 'a' and
            // the j-th column of 'b'
            c[i][j] += a[i][k] * b[k][j];
        }
    }
}
2901 questions
51
votes
3 answers

How to multiply two vector and get a matrix?

In numpy operation, I have two vectors, let's say vector A is 4X1, vector B is 1X5, if I do AXB, it should result a matrix of size 4X5. But I tried lot of times, doing many kinds of reshape and transpose, they all either raise error saying not…
Larry
  • 525
  • 1
  • 4
  • 5
41
votes
4 answers

numerically stable way to multiply log probability matrices in numpy

I need to take the matrix product of two NumPy matrices (or other 2d arrays) containing log probabilities. The naive way np.log(np.dot(np.exp(a), np.exp(b))) is not preferred for obvious reasons. Using from scipy.misc import logsumexp res =…
mart
  • 440
  • 4
  • 9
38
votes
5 answers

matrix multiplication algorithm time complexity

I came up with this algorithm for matrix multiplication. I read somewhere that matrix multiplication has a time complexity of o(n^2). But I think my this algorithm will give o(n^3). I don't know how to calculate time complexity of nested loops. So…
zedai
  • 523
  • 2
  • 6
  • 11
36
votes
2 answers

Efficient element-wise multiplication of a matrix and a vector in TensorFlow

What would be the most efficient way to multiply (element-wise) a 2D tensor (matrix): x11 x12 .. x1N ... xM1 xM2 .. xMN by a vertical vector: w1 ... wN to obtain a new matrix: x11*w1 x12*w2 ... x1N*wN ... xM1*w1 xM2*w2 ... xMN*wN To give some…
36
votes
12 answers

Matrix Multiplication in pure Python?

I'm trying to multiply two matrices together using pure Python. Input (X1 is a 3x3 and Xt is a 3x2): X1 = [[1.0016, 0.0, -16.0514], [0.0, 10000.0, -40000.0], [-16.0514, -40000.0, 160513.6437]] Xt = [(1.0, 1.0), (0.0, 0.25),…
Ammar
  • 399
  • 1
  • 4
  • 6
35
votes
5 answers

Matrix expression causes error "requires numeric/complex matrix/vector arguments"?

ma=diag(3)+t(da)%*%da R Code above, error message as follows: Error in t(da) %*% da : requires numeric/complex matrix/vector arguments da is a matrix, looks as following: V45 V46 V47 V48 V49 V50 V51…
user3505808
  • 405
  • 2
  • 5
  • 6
33
votes
3 answers

Why is this naive matrix multiplication faster than base R's?

In R, matrix multiplication is very optimized, i.e. is really just a call to BLAS/LAPACK. However, I'm surprised this very naive C++ code for matrix-vector multiplication seems reliably 30% faster. library(Rcpp) # Simple C++ code for matrix…
Cliff AB
  • 1,160
  • 8
  • 15
33
votes
10 answers

Matrix multiplication using arrays

I'm trying to make a simple matrix multiplication method using multidimensional arrays ([2][2]). I'm kinda new at this, and I just can't find what it is I'm doing wrong. I'd really appreciate any help in telling me what it is. I'd rather not use…
user2577854
31
votes
4 answers

How is a convolution calculated on an image with three (RGB) channels?

Say we have a single channel image (5x5) A = [ 1 2 3 4 5 6 7 8 9 2 1 4 5 6 3 4 5 6 7 4 3 4 5 6 2 ] And a filter K (2x2) K = [ 1 1 1 1 ] An example of applying convolution (let us take the first 2x2 from A) would…
Aragorn
  • 477
  • 1
  • 8
  • 12
28
votes
4 answers

Is there any fast method of matrix exponentiation?

Is there any faster method of matrix exponentiation to calculate Mn (where M is a matrix and n is an integer) than the simple divide and conquer algorithm?
Akashdeep Saluja
  • 2,959
  • 8
  • 30
  • 60
27
votes
2 answers

Multi-threaded integer matrix multiplication in NumPy/SciPy

Doing something like import numpy as np a = np.random.rand(10**4, 10**4) b = np.dot(a, a) uses multiple cores, and it runs nicely. The elements in a, though, are 64-bit floats (or 32-bit in 32-bit platforms?), and I'd like to multiply 8-bit integer…
étale-cohomology
  • 2,098
  • 2
  • 28
  • 33
26
votes
7 answers

How to speed up matrix multiplication in C++?

I'm performing matrix multiplication with this simple algorithm. To be more flexible I used objects for the matricies which contain dynamicly created arrays. Comparing this solution to my first one with static arrays it is 4 times slower. What can…
multiholle
  • 3,050
  • 8
  • 41
  • 60
26
votes
10 answers

Multiply a 3D matrix with a 2D matrix

Suppose I have an AxBxC matrix X and a BxD matrix Y. Is there a non-loop method by which I can multiply each of the C AxB matrices with Y?
Jacob
  • 34,255
  • 14
  • 110
  • 165
23
votes
5 answers

Efficient 4x4 matrix multiplication (C vs assembly)

I'm looking for a faster and trickier way to multiply two 4x4 matrices in C. My current research is focused on x86-64 assembly with SIMD extensions. So far, I've created a function witch is about 6x faster than a naive C implementation, which has…
Krzysztof Abramowicz
  • 1,556
  • 1
  • 12
  • 30
22
votes
1 answer

Fast sparse matrix multiplication

for class I have to write my own linear equation solver for sparse matrices. I am free to use any type of data structure for sparse matrices and I have to implement several solves, including conjuguate gradient. I was wondering if there is a famous…
lezebulon
  • 7,607
  • 11
  • 42
  • 73