1

I have a large table (x) to covert to matrix (y).

I used two different commands.

x <- reshape(y, direction="wide", v.names="column1", 
             timevar="column2", idvar="column3")

or

x <- cast(x, column1~column2)

After waiting for several hours, an error message appears.

 Error: cannot allocate vector of size 87 KB

In addition: There were 50 or more warnings (use warning() to see the first 50)

when I type warnings(), I found it is complainting 

Reached total allocation of 1535 Mb:
see help(memory.size)

In a second trial, I type "memory.size(TRUE) first, a message comes up: 

[1] 12.88

Then, I do the analysis all over again. However, the same error message appears.

I am using a desktop PC with windows 2000. I am using R2.12.0

Could you mind to let me know if it is possible to get rid of this problem?

Could you mind to let me know when I firstly type memory.size(TRUE) "12.88" comes up, does it mean that it all the 12G physical are RAM readily used by R in the second trial?

Gavin Simpson
  • 170,508
  • 25
  • 396
  • 453
Sally
  • 13
  • 3
  • Can you edit your Q to supply output from `memory.limit()` and `sessionInfo()` as those should help clarify the situation. – Gavin Simpson Apr 19 '11 at 07:40
  • no, you've done the `memory.size()` call wrong. Don't include *anything* in the parentheses. Can you run *exactly* this: `memory.size()` and paste the result into your Question. Also, I asked you for the output from `sessionInfo()` can you run that exactly like that and paste the output into your Q as well. That is the way to find out about your system. – Gavin Simpson Apr 19 '11 at 10:49

2 Answers2

1

On most windows systems, memory usage is very limited (in R).

You want to look at the results of memory.limit instead of memory.size, and then also use memory.limit to alter it.

In the best case scenario, you may crank this up to maybe 4 Gb, which may still be too little for your purpose (depends on the details of your OS - apparently, your machine has enough RAM, so physical memory is not the issue here (yet)).

See rw-FAQ for more info.

Nick Sabbe
  • 11,684
  • 1
  • 43
  • 57
  • In my window's cmd. I typed mem, following results comes up. 655360 bytes total conventional memory, 655360 bytes available to MS-DOS, 633632 largest executable program size, 1048576 bytes total contiguous extended memory, 0 bytes available contiguous extended memrory, 941056 bytes available XMS memory, MS-DOC resident in High memory Area. – Sally Apr 19 '11 at 09:19
  • Does it means that I can use up to 6G - 0.5 G RAM in R? Or Up to 4G RAM in max? – Sally Apr 19 '11 at 09:24
  • I tried to type "memory.limit(4000)" in R, it comes up [1] 4000. Then, I type "memory.size(), it comes up "[1] 730.56". It seems to me that I cannot use up to 4G RAM. More interesting is when I run the cast command above, Error mesage of "cannot allocate vector of size 128.0 Mb" was found. It seems to me that R cannot use up to 730 bytes or RAM. Why different number of RAM was found? – Sally Apr 19 '11 at 09:29
  • Admittedly, the help for the memory commands is somewhat confusing, but here's how I understand it:memory.size tells about how much memory could be allocated (max=TRUE) or is allocated (FALSE), regardless of the limit imposed by memory.limit (i.e.: is there room in the system's memory to increase the limit). The cannot allocate ... size is not of relevance: if you've used up 3800/4000, you cannot allocate 250 anymore. – Nick Sabbe Apr 19 '11 at 10:38
  • @Sally if you are running 32-bit R then the most RAM you could access would be 4GB but Windows might not allow that. If Windows is 32-bit then you can't access anything higher than 4GB over all applications and the OS. Windows will retain some of the memory for itself, so IIRC the max RAM you could possibly access on Windows with 32-bit R is ~3.5GB. That is why I asked to see the output from `sessionInfo()` as that will tell us what kind of system you are running on, and not what you tell us you are running on. – Gavin Simpson Apr 19 '11 at 10:54
0

Are you sure you have 12GB of RAM or that you are running a 64-bit version of Windows 2000? If you read ?memory.size you'll see that the value is in bytes not GB.

The main message states that you are running out of physical memory. The amount mentioned in the warnings of 1535MB is indicative of a machine with 2GB physical RAM or an OS that can only access that amount or allocate such an amount to a single process. If you really have 12GB physical RAM then you'll need to be running a 64-bit version of R on a 64-bit version of Windows on a 64-bit processor. Do all of these apply? It looks like for example that you might be running on a machine with 2GB of RAM or using a 32-bit version of R.

The problem boils down to exhausting the available RAM that R can access. Solve that or cut your problem down into smaller chunks and see if you can reshape each smaller chunk and then stick them together.

Gavin Simpson
  • 170,508
  • 25
  • 396
  • 453
  • I just type memory.size() in my computer. I found it is [1] 562.54. Does it means that i\ is only using 562 Bytes – Sally Apr 19 '11 at 08:48
  • @Sally that is what the help page claims – Gavin Simpson Apr 19 '11 at 09:08
  • I have added more details specifications of my PC in the comments in the answer 0 below, could you mind to further comments if I should call up "memory.limit(4000)" in window with the commmand states below? – Sally Apr 19 '11 at 09:31