32

R has some tools for memory profiling, like Rprofmem(), Rprof() with option "memory.profiling=TRUE" and tracemem(). The last one can only be used on objects, and hence is useful to follow how many times an object is copied, but doesn't give an overview on a function basis. Rprofmem should be able to do that, but the output of even the simplest function call like lm() gives over 500 lines of log. I tried to figure out what Rprof("somefile.log",memory.profile=T) actually does, but I don't think I really get it.

The last I could find was this message of Thomas Lumley, saying that, and I quote :

I do not yet have tools to summarize the output.

This was in 2006. Any chance there are options for some nice summaries now, based on either Rprofmem(), the mysterious output of Rprof() with memory.profile set TRUE or any other tool?

MichaelChirico
  • 33,841
  • 14
  • 113
  • 198
Joris Meys
  • 106,551
  • 31
  • 221
  • 263
  • 8
    Already looking forward to your RprofmemSummary package :) – Dirk Eddelbuettel Mar 03 '11 at 18:38
  • @Dirk I reckon that's a "Good luck, poor lad..." :) – Joris Meys Mar 03 '11 at 18:43
  • I'd to add this capability to profr. Hoping to find an interested student one day. It could be a good google summer of code project if you wanted to write it up. I'd be happy to co-mentor. – hadley Mar 05 '11 at 16:16
  • @hadley : I currently have no time to start with this, but if you didn't find an interested student by the summer, you can take me up on the challenge. Thx for the offer. – Joris Meys Mar 06 '11 at 22:48
  • Hi everyone, I just read this..is there any advance since Mar 2011 ? – nassimhddd Aug 17 '12 at 12:43
  • I didn't see any progress, but I haven't been checking either. Maybe a good thing to pick up when I finished the rest of the projects I still have to do... – Joris Meys Aug 17 '12 at 16:41
  • from http://developer.r-project.org/memory-profiling.html Rprof has an option memory.profiling. ... This is available only on Unix at the moment. – Andrew Hill Jan 29 '17 at 23:00

2 Answers2

7

profvis looks like the the solution to this question.

It generates an interactive .html file (using htmlwidgets) showing the profiling of your code.

The introduction vignette is a good guide on its capability.

Taking directly from the introduction, you would use it like this:

devtools::install_github("rstudio/profvis")
library(profvis)

# Generate data
times <- 4e5
cols <- 150
data <- as.data.frame(x = matrix(rnorm(times * cols, mean = 5), ncol = cols))
data <- cbind(id = paste0("g", seq_len(times)), data)
profvis({
    data1 <- data   # Store in another variable for this run

    # Get column means
    means <- apply(data1[, names(data1) != "id"], 2, mean)

    # Subtract mean from each column
    for (i in seq_along(means)) {
        data1[, names(data1) != "id"][, i] <- data1[, names(data1) != "id"][, i] - means[i]
    }
}, height = "400px")

Which gives

enter image description here

SymbolixAU
  • 25,502
  • 4
  • 67
  • 139
2

Check out profr -- it seems like exactly what you're looking for.

J. Taylor
  • 4,567
  • 3
  • 35
  • 55