0

I would like your help to see whether there is a solution to the following "out of memory" problem in Matlab.

I have two big matrices in Matlab

  • A of size ax2 with a=7*10^4

  • B of size bx2 with b=7*10^4

I need to do a scatterplot of C against D obtained by doing

Ctemp=zeros(a,b);
for i=1:a
    for j=1:b
        Ctemp(i,j)=A(i,1)+B(j,1);
    end
end
C=reshape(Ctemp, a*b,1);
C(C>=2 | C<=-2)=[];

Dtemp=zeros(a,b);
for i=1:a
    for j=1:b
        Dtemp(i,j)=A(i,2)+B(j,2);
    end
end
D=reshape(Dtemp, a*b,1);
D(C>=2 | C<=-2)=[];

The problem is that Matlab gives out of memory error when I attempt to construct Ctemp and Dtemp. Is there any way to circumvent this issue or what I am trying to do is unfeasible?

A naive approach could be

C=[];
D=[];

for i=1:a
    for j=1:b
        if A(i,1)+B(j,1)<=2 && A(i,1)+B(j,1)>=-2 
           C=[C; A(i,1)+B(j,1)];
           D=[D; A(i,2)+B(j,2)];
        end
    end
end

But I don't like this approach: it has a double loop that takes forever; it does not preallocate C,D.

This seems to work better, I don't know why

C=cell(a,1);
D=cell(a,1);


for i=1:a
    tempC=A(i,1)+B(:,1);
    tempD=A(i,2)+B(:,2);
    del=(A(i,1)+B(:,1)>2 | A(i,1)+B(:,1)<-2);
    tempC(del)=[];
    tempD(del)=[];
    C{i}=tempC;
    D{i}=tempD;
end
TEX
  • 2,249
  • 20
  • 43
  • 1
    You may want to use tall arrays, or manipulate data in mat files without loading them (see `matfile` in doc). – marsei Oct 07 '18 at 20:08
  • Thanks. I've never used that. Could you post an answer if you've time? Thanks – TEX Oct 07 '18 at 20:11
  • You can search for answers on mat file - it gives for example this one https://stackoverflow.com/a/16744050/1904719 . Basically create your variable, save it in mat, and access it for operations. – marsei Oct 07 '18 at 20:31
  • You’re trying to create a 39 Gb array. No matter how you compute it, it’ll likely not fit in memory. Are you sure this is what you need to do? – Cris Luengo Oct 08 '18 at 01:35
  • @CrisLuengo The problem is that I need to do a scatter plot of `C` on `D`. How can I do this without "creating" `C` and `D`? – TEX Oct 08 '18 at 06:38

0 Answers0