I am using python sparse module to compute a eigenvalue problem. It is a very big sparse matrix, which would end up with large memory requirement. But the strange thing is I am using a cluster with 256GB memory which should be definitely enough for my problem. But I get the not enough memory error as below. I am wondering if anyone would give me a hint how to work around this issue?
Not enough memory to perform factorization.
Traceback (most recent call last):
File "init_Re620eta1_40X_2Z_omega10.py", line 158, in <module>
exec_stabDiagBatchFFfollow_2D(geometry,baseFlowFolder,baseFlowVarb,baseFlowMethod,h,y_max_factor_EVP,Ny_EVP,Nz_EVP,x_p_stabDiag,x_p_orig,eigSolver,noEigs2solv,noEigs2save,SIGMA0,arnoldiTol,OmegaTol,disc_y,disc_z,y_i_factor_EVP,z_i_factor_EVP,periodicZ,BC_top,customComment,BETA,ALPHA_min,ALPHA_max,noALPHA,ALPHA_start,xp_start,u_0,nu_0,y_cut,ParallelFlowA,comm,rank,RESTART,saveJobStep,saveResultFormat)
File "/lustre/cray/ws8/ws/iagyonwu-Re620eta1/omega10DNS_wTS_icoPerbFoam/LST_functions_linstab2D_Temperal_mpi_multiTracking4.py", line 3893, in exec_stabDiagBatchFFfollow_2D
OMEGA, eigVecs = sp.linalg.eigs(L0, k=noEigs2solv, sigma=SIGMA, v0=options_v0, tol=arnoldiTol)
File "/opt/python/3.6.1.1/lib/python3.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py", line 1288, in eigs
symmetric=False, tol=tol)
File "/opt/python/3.6.1.1/lib/python3.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py", line 1046, in get_OPinv_matvec
return SpLuInv(A.tocsc()).matvec
File "/opt/python/3.6.1.1/lib/python3.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py", line 907, in __init__
self.M_lu = splu(M)
File "/opt/python/3.6.1.1/lib/python3.6/site-packages/scipy/sparse/linalg/dsolve/linsolve.py", line 267, in splu
ilu=False, options=_options)
MemoryError
The typical size of the matirx is 50000x50000, and here followed the structure of L0:
