Questions tagged [matrix-multiplication]

Questions related to matrix multiplication, especially implementation. Mathematical questions should consider the [linear-algebra] tag.

Matrix multiplication is usually the process of multiplying two (or more) matrices. This arises quite frequently in linear algebra contexts and is a particularly fundamental task in computing, especially in scientific computing.

To that end, a number of fundamental libraries, such as LAPACK, BLAS, ATLAS, and others have been developed. Because the growth of matrices affects the computational time, extensive effort has been made to optimize these packages for various computer architectures and various matrix sizes.

In scientific software for statistical computing and graphics, operator %*% performs matrix multiplication (see ?"%*%"), and interfaces BLAS routine dgemmm.


The product of the multiplication of two matrices a and b is the matrix c, where each element is the sum of the products of the i-th row of a and the j-th column of b.

c[i][j] += a[i][k] * b[k][j];

Example:

     (b) { 1,  2,  3,  4}
         { 5,  6,  7,  8}
(a)    ┌(c)──────────────
{1, 2} │ {11, 14, 17, 20}
{3, 4} │ {23, 30, 37, 44}
{5, 6} │ {35, 46, 57, 68}

Algorithm of the matrix multiplication:

// rows of 'a' matrix
int m = 3;
// columns of 'a' matrix
// and rows of 'b' matrix
int n = 2;
// columns of 'b' matrix
int p = 4;
// matrices 'a=m×n', 'b=n×p'
int[][] a = {{1, 2}, {3, 4}, {5, 6}},
        b = {{1, 2, 3, 4}, {5, 6, 7, 8}};
// resulting matrix 'c=m×p'
int[][] c = new int[m][p];
// iterate over the rows of the 'a' matrix
for (int i = 0; i < m; i++) {
    // iterate over the columns of the 'b' matrix
    for (int j = 0; j < p; j++) {
        // iterate over the columns of the 'a'
        // matrix, aka rows of the 'b' matrix
        for (int k = 0; k < n; k++) {
            // sum of the products of
            // the i-th row of 'a' and
            // the j-th column of 'b'
            c[i][j] += a[i][k] * b[k][j];
        }
    }
}
2901 questions
11
votes
5 answers

Is there a good double-precision small matrix SIMD library for x86?

I'm looking for a SIMD library focused small (4x4) matrix operations for graphics. There's lots of single precision ones out there, but I need to support both single and double precision. I've looked at Intel's IPP MX library, but I'd prefer…
Justicle
  • 14,761
  • 17
  • 70
  • 94
11
votes
3 answers

Why is the performance of these matrix multiplications so different?

I wrote two matrix classes in Java just to compare the performance of their matrix multiplications. One class (Mat1) stores a double[][] A member where row i of the matrix is A[i]. The other class (Mat2) stores A and T where T is the transpose of…
CromTheDestroyer
  • 3,616
  • 3
  • 20
  • 26
11
votes
1 answer

How to force tensorflow tensors to be symmetric?

I have a set of MxM symmetric matrix Variables in a graph whose values I'd like to optimize. Is there a way to enforce the symmetric condition? I've thought about adding a term to the loss function to enforce it, but this seems awkward and…
Mark Borgerding
  • 8,117
  • 4
  • 30
  • 51
11
votes
2 answers

The complexity of the multiplication of two lower triangular matrices

I know that the lower bound of the multiplication of two full matrices is Ω(n^2). Matrix multiplication I have been trying to prove that the lower bound of the multiplication of two lower triangular matrices using problem transformation method. My…
Alex Lin
  • 395
  • 3
  • 13
11
votes
2 answers

parallelizing matrix multiplication through threading and SIMD

I am trying to speed up matrix multiplication on multicore architecture. For this end, I try to use threads and SIMD at the same time. But my results are not good. I test speed up over sequential matrix multiplication: void sequentialMatMul(void*…
11
votes
1 answer

Find the single wrong element in matrix product?

Three N*N matrices A,B,C are given. C is the same as the product of A and B except that exactly one element is wrong. The naive algorithm to find it out requires N^3 time. Can we do faster than that?
user2249675
  • 464
  • 3
  • 13
11
votes
3 answers

Which is the best way to multiply a large and sparse matrix with its transpose?

I currently want to multiply a large sparse matrix(~1M x 200k) with its transpose. The values of the resulting matrix would be in float. I tried loading the matrix in scipy's sparse matrix and by multiplying each row of first matrix with the second…
11
votes
1 answer

element wise multiplication in r

Is there an inbuilt function or operator to do the following in R : ElementwiseMultiply <- function ( a_, b_ ) { c_ = a_ ; for ( i in 1:ncol(a_) ) { c_[,i] = ( a_[,i] * b_ ) ; } return ( c_ ); } For instance > a_ [,1] [,2] [1,] 1 …
Humble Debugger
  • 4,439
  • 11
  • 39
  • 56
11
votes
2 answers

What is the fastest way to quadratic form numpy array multiplication?

I have tried those two alternatives objective = lambda A, x : (np.dot(x.T ,np.dot(A, x)))[0,0] objective = lambda A, x : (np.matrix(x).T * np.matrix(A) * np.matrix(x))[0,0] With primary one I got 5 sec of running time with my algorithm With…
erogol
  • 13,156
  • 33
  • 101
  • 155
11
votes
2 answers

Improving Performance of Multiplication of Scipy Sparse Matrices

Given a Scipy CSC Sparse matrix "sm" with dimensions (170k x 170k) with 440 million non-null points and a sparse CSC vector "v" (170k x 1) with a few non-null points, is there anything that can be done to improve the performance of the…
Willian Fuks
  • 11,259
  • 10
  • 50
  • 74
10
votes
1 answer

Efficient Algorithms for Computing a matrix times its transpose

For a class, a question that was posed by my teacher was the algorithmic cost of multiplying a matrix times its transpose. With the standard 3 loop matrix multiplication algorithm, the efficiency is O(N^3), and I wonder if there was a way to…
Matt
  • 1,599
  • 3
  • 21
  • 33
10
votes
6 answers

How to implement fast image filters on iOS platform

I am working on iOS application where user can apply a certain set of photo filters. Each filter is basically set of Photoshop actions with a specific parameters. This actions are: Levels adjustment Brightness / Contrast Hue / Saturation Single and…
10
votes
4 answers

Faster Matrix Multiplication in C#

I have as small c# project that involves matrices. I am processing large amounts of data by splitting it into n-length chunks, treating the chucks as vectors, and multiplying by a Vandermonde** matrix. The problem is, depending on the conditions,…
Kyle Lahnakoski
  • 924
  • 10
  • 16
10
votes
1 answer

Large (0,1) matrix multiplication using bitwise AND and popcount instead of actual int or float multiplies?

For multiplying large binary matrices (10Kx20K), what I usually to do is to convert the matrices to float ones and perform float matrix multiplication as integer matrix multiplication is pretty slow (have a look at here). This time though, I'd need…
NULL
  • 759
  • 9
  • 18
10
votes
3 answers

Matrix/Tensor Triple Product?

An algorithm I'm working on requires computing, in a couple places, a type of matrix triple product. The operation takes three square matrices with identical dimensions, and produces a 3-index tensor. Labeling the operands A, B and C, the …
Robert T. McGibbon
  • 5,075
  • 3
  • 37
  • 45