50

I want to make a function that imports data in different numbers of batches depending on how much RAM is available on someone's system. But how can I find the amount of available RAM in R? I can use memory.size() but that only works for Windows.

Ben Bolker
  • 211,554
  • 25
  • 370
  • 453
Sacha Epskamp
  • 46,463
  • 20
  • 113
  • 131
  • 3
    You are not going to implement one method to do this. Your going to have to detect the platform then use different methods. – Security Hound Jun 23 '11 at 16:16
  • *"If you have to ask for the price, you cannot afford it."* This looks the wrong approach as memory and resource management is handled by the operating system. As Ramhound said, your approach will become platform-dependent. – Dirk Eddelbuettel Jun 23 '11 at 16:24
  • 3
    The idea is to get a crude value that works somewhat as a default. For example, 1 batch with 12gb, 2 with 6gb, etcetera. Platform can be found with `Sys.info()` so platform dependent approach should be possible right? – Sacha Epskamp Jun 23 '11 at 16:30
  • Warning message: 'memory.size()' is Windows-specific – Enrique Pérez Herrero Jul 11 '23 at 16:30

4 Answers4

49

Given the warnings concerning platform-dependency discussed in the earlier comment, you could for example parse /proc/meminfo on Linux:

$ grep MemFree /proc/meminfo 
MemFree:          573660 kB
$ awk '/MemFree/ {print $2}' /proc/meminfo 
565464

You could try the second approach via system(..., intern=TRUE), or even via a pipe connection.

Edit some 5+ years later: In R, and just following what the previous paragraph hinted at:

R> memfree <- as.numeric(system("awk '/MemFree/ {print $2}' /proc/meminfo", 
+                               intern=TRUE))
R> memfree
[1] 3342480
R>
Dirk Eddelbuettel
  • 360,940
  • 56
  • 644
  • 725
  • 6
    I recommend to use `gc()` before calling this function to get the true amount of available memory. – F. Privé May 01 '17 at 09:00
  • 3
    Is the output unit bytes? R in windows usually specifies – Scransom Sep 14 '17 at 07:39
  • 19
    for those that prefer copy-pasteable code: `as.numeric(system("awk '/MemFree/ {print $2}' /proc/meminfo", intern=TRUE))` – MichaelChirico Dec 11 '17 at 02:18
  • This is not allowed by CRAN. – xiaodai Jan 27 '20 at 01:38
  • @xiaodai do you know why? I have heard cran doesn't like `system()` calls, but I don't believe there's any complete ban. I think cran libraries need to work across all of windows/osx/unix/linux, perhaps this command stumbles on some platform (I don't know that for sure, just trying to guess why it would be denied). – stevec Apr 17 '20 at 07:28
  • 1
    I don't know why but my package got taken down cos I used it. So don't use it in a package. – xiaodai Apr 17 '20 at 09:52
  • @F.Privé How does `gc()` be useful in this context? Should the info be gathered from `Ncells` or `Vcells` output? – Prradep Oct 29 '20 at 10:26
  • 1
    @Prradep It is useful because it gets rid of all the things that can be cleaned up to free some memory. – F. Privé Oct 29 '20 at 14:57
  • 1
    Yes, I understand that. How to infer about the RAM availability from this output? – Prradep Oct 30 '20 at 12:07
  • 1
    Given that in the question it is asked about the available RAM, wouldn't `MemAvailable` make more sense than `MemFree`? (cache would not be free, but it is available) – zeehio Dec 29 '21 at 06:59
14

I would recommend using memuse::Sys.meminfo().

F. Privé
  • 11,423
  • 2
  • 27
  • 78
10

Now you can do that with benchmarkme::get_ram function.

https://cran.r-project.org/web/packages/benchmarkme/benchmarkme.pdf

user5029763
  • 1,903
  • 1
  • 15
  • 23
1

Adding to the solutions already presented, I wrote a base R workaround that avoids package dependencies (didn't want to add to the current dependencies in the package):

available_memory <- function()
{

  # Get operating system
  OS <- tolower(Sys.info()["sysname"])

  # Branch based on OS
  if(OS == "windows"){ # Windows

    # System information
    system_info <- system("systeminfo", intern = TRUE)

    # Get available memory
    value <- system_info[
      grep("Available Physical Memory", system_info)
    ]

    # Remove extraneous information
    value <- gsub("Available Physical Memory: ", "", value)
    value <- gsub("\\,", "", value)
    
    # Convert to bytes
    value_split <- unlist(strsplit(value, split = " "))
    
    # Check for second value
    bytes <- as.numeric(value_split[1]) * switch(
      value_split[2],
      "KB" = 1e03,
      "MB" = 1e06,
      "GB" = 1e09
    )

  }else if(OS == "linux"){ # Linux
    
    # Split system information
    info_split <- strsplit(system("free", intern = TRUE), split = " ")
    
    # Remove "Mem:" and "Swap:"
    info_split <- lapply(info_split, function(x){gsub("Mem:", "", x)})
    info_split <- lapply(info_split, function(x){gsub("Swap:", "", x)})
    
    # Get actual values
    info_split <- lapply(info_split, function(x){x[x != ""]})
    
    # Bind values
    info_split <- do.call(rbind, info_split[1:2])
    
    # Get free values
    bytes <- as.numeric(info_split[2, info_split[1,] == "free"])
    
  }else{ # Mac
    
    # System information
    system_info <- system("top -l 1 -s 0 | grep PhysMem", intern = TRUE)

    # Get everything after comma
    unused <- gsub(" .*,", "", system_info)
    
    # Get values only
    value <- gsub("PhysMem: ", "", unused)
    value <- gsub(" unused.", "", value)
    
    # Check for bytes
    if(grepl("M", value)){
      bytes <- as.numeric(gsub("M", "", value)) * 1e06
    }else if(grepl("G", value)){
      bytes <- as.numeric(gsub("G", "", value)) * 1e09
    }else if(grepl("K", value)){
      bytes <- as.numeric(gsub("K", "", value)) * 1e03
    }
    
  }
  
  # Return bytes
  return(bytes)

}

It returns available memory in bytes and generally works (in my testing)

Alexander Christensen
  • 348
  • 1
  • 10