Math.Net SVD Algorithm throws Errors for Large Matrices

Question

I’ve been using the intel MKL implementation of SVD() through Math.NET numerics (https://numerics.mathdotnet.com/). I’ve had success running the algorithm. However, for very large matrices (in this case 64,000 x 500), I get the following error:

Unhandled exception. System.OverflowException: Arithmetic operation resulted in an overflow. at MathNet.Numerics.LinearAlgebra.Storage.DenseColumnMajorMatrixStorage`1..ctor(Int32 rows, Int32 columns) at MathNet.Numerics.LinearAlgebra.Double.DenseMatrix..ctor(Int32 order) at MathNet.Numerics.LinearAlgebra.Double.Factorization.DenseSvd.Create(DenseMatrix matrix, Boolean computeVectors) at MathNet.Numerics.LinearAlgebra.Double.DenseMatrix.Svd(Boolean computeVectors) >

Has anyone encountered anything similar? I was wondering if this is an an issue with the Math.Net implementation or with the intel MKL when using with large matrices. Is there a fix anyone could suggest that could be implemented to allow the library to be run on larger matrices?

Source Code

using System;
using MathNet.Numerics.LinearAlgebra;
using System.Diagnostics;

namespace Copula
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Parsing matrix dimensions from input parameters.");
            //int n_rows = Int32.Parse(args[0]);
            //int n_columns = Int32.Parse(args[1]);
            int n_columns = 500;
            int n_rows = 64000;
            Console.WriteLine("Generating Random matrix wtih dimensions " + n_rows + " rows and " + n_columns + " columns.");
        var stop_watch = Stopwatch.StartNew();
        var time_series_matrix = GenerateRandomMatrix(n_rows, n_columns);
        stop_watch.Stop();
        Console.Write("Took " + stop_watch.Elapsed + "\n");

        Console.WriteLine("Performing SVD.");

        stop_watch = Stopwatch.StartNew();
        var ts_svd = time_series_matrix.Svd();
        stop_watch.Stop();
        Console.Write("Took " + stop_watch.Elapsed + "\n");

    }

    static Matrix<double> GenerateRandomMatrix(int n_rows, int n_columns)
    {
        Matrix<double> out_matrix = Matrix<double>.Build.Random(n_rows, n_columns);

        return out_matrix;
    }
   }
}

Project File

<Project Sdk="Microsoft.NET.Sdk">

 <PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.0</TargetFramework>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="MathNet.Numerics" Version="4.8.1" />
<PackageReference Include="MathNet.Numerics.MKL.Win-x64" Version="2.3.0" />
<PackageReference Include="Meta.Numerics" Version="4.0.7" />
</ItemGroup>

</Project>

I get the same error using this code: DenseMatrix denseMatrix = new DenseMatrix(64000); — HerbalMart, Dec 09 '19 at 20:57

score 0 · Answer 1 · answered Dec 10 '19 at 03:32

0

Do you see the same problem when running with much smaller cases? let's say 1Kx1K?

answered Dec 10 '19 at 03:32

Gennady.F

571
2
7

then the problem may be caused by the ILP64 support. Please try to rebuild your case with the MKL_ILP64 compilers option and link it against ILP64 libs. Please check the Intel MKL Linker Adviser to see how to do that. – Gennady.F Dec 12 '19 at 03:21
https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor – Gennady.F Dec 12 '19 at 03:23

Math.Net SVD Algorithm throws Errors for Large Matrices

1 Answers1