0

I have a series of R scripts for doing the multiple steps of data analysis that I require. Some of these take a very long time and create really large objects. I've noticed that if I just source all of them in a row (via a main.R script), the processing for later steps takes much longer than if I source one script, save what I need, and restart R for the next step (loading the data I need).

I was wondering if there was a way, via Rscript or a Bash script perhaps, that I could carry this out. There would need to be objects that persist for the first 2 scripts (which load my external data and create the objects that will be used for all further steps). I suppose I could also just save those and load them in further scripts.

(I would also like to pass a number of named arguments to this script, which I think I can find on other SO posts and can use something like optparse.)

So, the script would look something like this, I think:

#! /bin/bash
Rscript 01_load.R  # Objects would persist, ideally
Rscript 02_create_graphs.R  # Objects would persist, ideally
Rscript 03_random_graphs.R  # contains code to save objects
#exit R
Rscript 04_permutation_analysis.R  # would have to contain code to load data
#exit

And so on. Is there a solution to this? I'm using R 3.2.2 on 64-bit CentOS 6. Thanks.

Chris Watson
  • 1,347
  • 1
  • 9
  • 24

1 Answers1

0

Chris,

it sounds you should do some manual housekeeping between (or within) your steps by using gc() and maybe also rm(). For more details see help(gc) and help(rm).

So instead of exit R and restart it again you could do:

rm(list = ls())
gc()

But please note: rm(list = ls()) would throw away all your objects. Better you create a suitable list of objects you really want to throw away and pass this list to rm().

leo
  • 141
  • 3
  • Thanks for your response. I should have mentioned that I have tried using `gc()`. While it helps in terms of memory management, it doesn't seem to get things as fast as running from a clean session. Although I haven't done extensive testing on this. – Chris Watson Jan 17 '16 at 19:37