3

I'm using knitr to make a document that uses the terra package to draw maps. Here's my minimal .Rmd file:

```{r init, echo=FALSE}
library(knitr)
opts_chunk$set(cache=TRUE)
```

```{r maps}
library(terra)
r = rast(matrix(1:12,3,4))
plot(r)
```

```{r test2}
print(r)
plot(r)
```

First run (via rmarkdown::render(...)), this works, creates a test_cache folder. Running a second time (with no changes) it also runs fine. If I make a minor change to chunk 2 (eg add a comment) and run I get:

Quitting from lines 14-17 (cache.Rmd) 
Error in .External(list(name = "CppMethod__invoke_notvoid", address = <pointer: (nil)>,  : 
  NULL value passed as symbol address

I've also had this from another Rmd file, probably related:

Quitting from lines 110-115 (work.Rmd) 
Error in x@ptr$readStart() : external pointer is not valid

clearing the cache or running with cache=FALSE then works, but then what's the point of the cache.

I think its because r is some sort of reference class which exists via some memory allocated by Rcpp, and knitr is only caching a reference, so when it tries to read the cached version it gets a reference to memory which doesn't have the object that was created to go with the reference there. So it fails.

FWIW a terra raster object looks like this:

> str(r)
Formal class 'SpatRaster' [package "terra"] with 1 slot
  ..@ ptr:Reference class 'Rcpp_SpatRaster' [package "terra"] with 20 fields
  .. ..$ depth    : num 0
  .. ..$ extent   :Reference class 'Rcpp_SpatExtent' [package "terra"] with 2 fields

and

> r@ptr
C++ object <0x55ce6fdf2bd0> of class 'SpatRaster' <0x55ce5a6750b0>

Is there a way to make the knitr cache work with these objects? I know I could exclude just these from the cache but 90% of my document is working with these sorts of objects and that's the reason I want to use the cache to speed things up. But then every time I get this error I have to stop, clear the cache, start again, and I don't know if that time is worth the speedup I get with the cache.

R 4.1.1 with

> packageVersion("knitr")
[1] ‘1.34’
> packageVersion("rmarkdown")
[1] ‘2.11’
Spacedman
  • 92,590
  • 12
  • 140
  • 224
  • 3
    Yes, I think it's a known Rcpp limitation that the objects are not at a 'known and fixed' memory location so reuse (i.e. caching) is tricky / impossible. – Dirk Eddelbuettel Jun 16 '22 at 15:15
  • Weirdly I couldn't find any mention of this in the docs or elsewhere. Oh well. Maybe I can't google properly. Or its solved with "quarto" (just checked: its not solved with quarto). – Spacedman Jun 16 '22 at 15:28
  • 1
    I raised an issue about this with the knitr developers: https://github.com/yihui/knitr/issues/2176 – Robert Hijmans Sep 27 '22 at 18:18
  • Just thinking out loud, but maybe you could use the memoise package instead of the cache option? – Josep Pueyo Aug 23 '23 at 18:30

1 Answers1

1

terra does some work to allow serialization, for example:

library(terra)
f <- system.file("ex/elev.tif", package="terra")
r <- rast(f)

saveRDS(r, "test.rds")
readRDS("test.rds")
[1] "This is a PackedSpatRaster object. Use 'terra::rast()' to unpack it"

readRDS("test.rds") |> rast()
#class       : SpatRaster 
#dimensions  : 90, 95, 1  (nrow, ncol, nlyr)
#resolution  : 0.008333333, 0.008333333  (x, y)
#extent      : 5.741667, 6.533333, 49.44167, 50.19167  (xmin, xmax, ymin, ymax)
#coord. ref. : lon/lat WGS 84 (EPSG:4326) 
#source      : memory 
#name        : elevation 
#min value   :       141 
#max value   :       547

But it seems that knitr caches with "save" to an RData file; in that case, the raw object is stored, which won't be valid when reloaded.

I think it may be possible to work around this with some clever use of hook functions; but that would be so involved that it would defeat the purpose of caching.

Robert Hijmans
  • 40,301
  • 4
  • 55
  • 63