0

I am trying to create an R package. I have a prebuilt r data object(model.RData) saved in data folder of my R package . But its size is huge ~30 GB. While installing the package I got the below error:

Error in utils::tar(filepath, pkgname, compression = "gzip", compression_level = 9L,  : 
  file size is limited to 8GB

I tried adding datalist using the below command but it returns NULL 

tools::add_datalist("/Package/data",force = TRUE)

I have seen a couple of techniques being proposed like distributing datasets as .Rda images but not sure how to go about it. Please suggest a technique on dealing with such huge datasets when creating R packages.

Vishnu
  • 110
  • 2
  • 10
  • 1
    I think in a package your limited to 10MB. What you could do is putting a link to a repo where to download this data. But do you really need 30GB of precomputed data? – F. Privé Aug 26 '17 at 18:46
  • @F.Privé: I do not need the 30 gb of data to be loaded every time the package function is called. Could you please elaborate on your comment? – Vishnu Aug 27 '17 at 02:13
  • This sounds like a [XY problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). Before asking "How to create R package with a 30GB dataset?", it may be good to ask "How do I reference a 30GB dataset in my R package?" or even "Do I need a 30GB dataset in my R package, rather than anywhere else in my computer?" If you have *compelling* reasons for including it in the package, do elaborate so that we understand your situation better. Else F. Privé's suggestion regarding providing a link for the data is worth considering. – Z.Lin Aug 27 '17 at 09:37
  • @Z.Lin: Apologies if I did not frame the question correctly. I have computed the dataset by creating and saving it as a model(.Rdata). To be clear I do not want the R object to be included in my package but only reference it. I have created simple r packages but they had smaller datasets. I would like to know if there is any efficient method to reference the R model object. – Vishnu Aug 27 '17 at 16:27

0 Answers0