0

I am having time-trouble reading my ".csv" file into Matlab. Using the following part from this forum i found out the number of rows is roughly 12 Million, the number of columns is 7. 

function count = countLines(fname)
fh = fopen(fname, 'rt');
assert(fh ~= -1, 'Could not read: %s', fname);
x = onCleanup(@() fclose(fh));
count = 0;
while ~feof(fh)
count = count + sum( fread( fh, 16384, 'char' ) == char(10) );
end
end

I tried the dlmread command to read it into Matlab which takes about 120 seconds:

tic
fname='test1_csv0.csv';
N1=countLines(fname)-1; 
N2=6;
blockmodel=dlmread('test1_csv0.csv',',',[1 0 N1 N2]);
toc

I found this post on fast Matlab reading

Fastest Matlab file reading?

But to be honest, as I am not a programmer, I do not know how to adapt the code to my problem and which would be the fastest option. I would like to keep the solution in Matlab, as this is the only programming-language I slightly understand. I would appreciate any help to get Matlab to read my matrix faster, do you have any experience whether "textscan" might work better then "dlmread" ?

Community
  • 1
  • 1
KiW
  • 593
  • 3
  • 20
  • `dlmread` is basically a wrapper for `textscan`. I therefore would not be surprised if you can make `textscan` work faster, since you can omit a lot of the checks performed by `dlmread`, provided you know your data beforehand. For csv files there's `csvread` as well (which also is a wrapper for `textscan`) – Adriaan Jun 08 '16 at 08:48
  • could you help me set up a short script for textscan ? ... i am not really sure how to use it ( the mathworks explanation is a bit confusing for me :/ ) – KiW Jun 08 '16 at 09:17
  • `fileID = fopen('datafile.csv'); C = textscan(fileID,'%f %f %f %f %f %f %f'); fclose(fileID); out = cell2mat(C);` should do the trick. You might as well use `csvread`, as that's written especially for CSVs and actually returns a matrix, so you don't have to manually convert it using `cell2mat` – Adriaan Jun 08 '16 at 09:31
  • trying to run your script i just get the following error : Error in textscan (line 3) C = textscan(fileID,'%f %f %f %f %f %f %f'); any ideas on that ? – KiW Jun 08 '16 at 10:24

0 Answers0