17

I have a 30000x14000 sparse matrix in MATLAB (version 7), which I need to use in another program. Calling save won't write this as ASCII (not supported). Calling full() on this monster results in an Out of Memory error.
How do I export it?

AlessioX
  • 3,167
  • 6
  • 24
  • 40
Midhat
  • 17,454
  • 22
  • 87
  • 114

8 Answers8

29

You can use find to get index & value vectors:

[i,j,val] = find(data)
data_dump = [i,j,val]

You can recreate data from data_dump with spconvert, which is meant to "Import from sparse matrix external format" (so I guess it's a good export format):

data = spconvert( data_dump )

You can save to ascii with:

save -ascii data.txt data_dump

But this dumps indices as double, you can write it out more nicely with fopen/fprintf/fclose:

fid = fopen('data.txt','w')
fprintf( fid,'%d %d %f\n', transpose(data_dump) )
fclose(fid)

Hope this helps.

Joel
  • 22,598
  • 6
  • 69
  • 93
Matthieu
  • 628
  • 5
  • 8
  • 4
    The data_dump during the fprintf command should be transposed before using according the docs (http://www.mathworks.com/help/matlab/ref/fprintf.html). This happens because the data is written in collumn-order. – Alan CN Apr 14 '14 at 19:18
8

Save the sparse matrix as a .mat file. Then, in the other program, use a suitable library to read the .mat file.

For instance, if the other program is written in Python, you can use the scipy.io.mio.loadmat function, which supports sparse arrays and gives you a sparse numpy matrix.

Vebjorn Ljosa
  • 17,438
  • 13
  • 70
  • 88
3

I saved it as text using Java within MATLAB. MATLAB Code:


pw=java.io.PrintWriter(java.io.FileWriter('c:\\retail.txt'));
line=num2str(0:size(data,2)-1);
pw.println(line);
for index=1:length(data)
    disp(index);
    line=num2str(full(data(index,:)));
    pw.println(line);
end
pw.flush();
pw.close();

Here data is an extremely large sparse matrix.

Midhat
  • 17,454
  • 22
  • 87
  • 114
  • 1
    I'm confused - why did you use java.io instead of MATLAB's built-in fopen and fprintf? – SCFrench Dec 11 '09 at 23:56
  • probably because i knew better java than matlab, and it was a throw away code, so it didnt need to be beautiful. it just needed to work correctly :) – Midhat Dec 15 '09 at 16:57
2

Did you try partitioning it ?

I mean try calling full() on the 1000 first rows (or 5000) and then repeat the process if it works.

Veynom
  • 4,079
  • 2
  • 19
  • 24
  • yes thats always an option, it will probably take a lot of time writing ascii chunks and merging them later – Midhat Oct 20 '08 at 09:35
2

Use the find function to get the indices of non-zero elements...

idcs = find(data);
vals = data(idcs);
...save the index vector and value vector in whatever format you want...

If you want, you can use ind2sub to convert the linear indices to row, column subscripts.

If you need to recreate a sparse matrix in matlab from subscripts + values, use spconvert.

Mr Fooz
  • 109,094
  • 6
  • 73
  • 101
1

dlmwrite - Write matrix to ASCII-delimited file Syntax

dlmwrite(filename, M)

dlmwrite(filename, M, 'D')

dlmwrite(filename, M, 'D', R, C)

dlmwrite(filename, M, 'attrib1', value1, 'attrib2', value2, ...)

dlmwrite(filename, M, '-append')

dlmwrite(filename, M, '-append', attribute-value list)

ehsan
  • 11
  • 1
0

If this is pretty much a one time deal, then I would just iterate through the matrix and write the matrix to an ASCII file by brute force, or else use @Veynom's suggestion and call full() on a subset of rows. It may take a while, but it will probably be done faster than it might take to learn how to read in a .mat file outside of the MATLAB environment.

If this is something you need to do on a recurring basis, then I would take @Vebjorn's advice and use a library to read the .mat file.

Community
  • 1
  • 1
Scottie T
  • 11,729
  • 10
  • 45
  • 59
0

Use this script: msm_to_mm.m, writes an MATLAB sparse matrix to an MatrixMarket file.

And This thread may also be useful.

Code42
  • 2,292
  • 1
  • 17
  • 22