10

Background:
This is the "microbenchmark" package for R: https://cran.r-project.org/web/packages/microbenchmark/index.html

The first line in the reference manual says that it is built for "Accurate Timing Functions".

One problem with this is the intrinsic computer-time vs. computer-memory trade-off. Some solutions are memory intensive, but CPU fast. Some are CPU intensive, but have a very small memory footprint.

Question:
How do I simultaneously, and with good resolution, benchmark/microbenchmark not only the execution time, but the memory use during execution in R?

EngrStudent
  • 1,924
  • 31
  • 46
  • I like the question but feel it's off-topic for SO ... – r2evans Feb 22 '18 at 00:25
  • What is the right library for my application is off-topic? I'm looking for what should be a software tool commonly used by R-programmers. – EngrStudent Feb 22 '18 at 00:26
  • 1
    Yes: *"asking us to recommend or find a book, tool, software library"*, https://stackoverflow.com/help/on-topic – r2evans Feb 22 '18 at 00:28
  • 2
    The [profvis](https://rstudio.github.io/profvis/) package does some memory profiling, but I don't think it's as easy to compare multiple solutions on their memory usage. – Marius Feb 22 '18 at 00:35
  • @r2evans - The piece of text that defines whether "recommend" is on vs. off is this "...as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it." Inspection will show that I gave a clear problem, and instead of inviting "opinion" or "spam", I am asking for action therefore the criterion you used does not apply. – EngrStudent Feb 22 '18 at 02:20
  • 1
    I'm happy to see that, which is why I said I like the question (hoping it might be an exception to the rule). Thanks for persevering, I'd like to see a good answer (as I don't have one offhand). (And great point about clarifying the on/off topic-ness, not certain I had that part as well remembered.) – r2evans Feb 22 '18 at 04:53
  • @Marius - it looks like profvis does both time and memory! Nice. – EngrStudent Feb 22 '18 at 13:13
  • 2
    EngrStudent, if you find a way to do simpler top-level comparison of memory use, similar to `microbenchmark`, I suggest (request?) you provide up a quick self-answer. If you don't have the time, perhaps you could suggest a [`profvis` issue](https://github.com/rstudio/profvis/issues) suggesting the use-case, I know I'd "+1" it. – r2evans Feb 22 '18 at 16:34
  • @Marius - It looks like the "profvis" uses this profiler: https://stat.ethz.ch/R-manual/R-devel/library/utils/html/Rprof.html – EngrStudent Feb 22 '18 at 18:31
  • @r2evans - I'm very much up for that. :) – EngrStudent Feb 22 '18 at 18:32

1 Answers1

13

Better late than never: You can use bench::mark() to measure both time and memory usage of code (and some more variables).

I.e., (taken from the help page for ?mark)

library(bench)

dat <- data.frame(x = runif(100, 1, 1000), y=runif(10, 1, 1000))
mark(
  dat[dat$x > 500, ],
  dat[which(dat$x > 500), ],
  subset(dat, x > 500)
)
#> # A tibble: 3 x 6
#>   expression                     min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr>                <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 dat[dat$x > 500, ]          21.7µs   23.6µs    40663.    4.15KB     89.7
#> 2 dat[which(dat$x > 500), ]   22.2µs   24.1µs    40228.    2.77KB     92.7
#> 3 subset(dat, x > 500)          36µs   39.2µs    23867.   20.12KB     86.2

Created on 2020-03-02 by the reprex package (v0.3.0)

David
  • 9,216
  • 4
  • 45
  • 78