Rmarkdown: long vectors not supported yet

Question

I am writing an Rmarkdown document that involves the following code chunk

library(wordvectors)
glove.model <- read.vectors(
   filename="glove.840B.300d.txt", # 5.6 GB text file
   vectors=300, binary=FALSE)

where the txt file is the result of unzipping http://nlp.stanford.edu/data/glove.840B.300d.zip. The code runs fine as I write this and when I run it with 'run current chunk', but when I try to knit it, I get one of the two following error messages:

Error in lazyLoadDBinsertvalue(data, datafile, ascii, compress, envhook) : cannot allocate buffer calls: ... lazyLoadDBinsertvalue -> -> lazyLoadDBinsertvalue
Error in lazyLoadDBinsertvalue(data, datafile, ascii, compress, envhook) : long vectors not supported yet: connections.c6011 calls: ... lazyLoadDBinsertvalue -> -> lazyLoadDBinsertvalue

Since the code runs fine per se and only doesn't work when knitted, is there any setting or something I'd need to tweak? Here's my session info:

R version 4.0.1 (2020-06-06)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Linux Mint 19.3

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] compiler_4.0.1  htmltools_0.4.0 tools_4.0.1     yaml_2.2.1      Rcpp_1.0.4.6    rmarkdown_2.2  
 [7] knitr_1.28      xfun_0.14       digest_0.6.25   rlang_0.4.6     evaluate_0.14

Any thoughts?

Did you read [this answer](https://stackoverflow.com/questions/39417003/long-vectors-not-supported-yet-error-in-rmd-but-not-in-r-script)? — J_F, Jun 14 '20 at 07:30
Darn, I had not found that one. I had come across the usual discussions of limits of vector length and matrices (like [here](https://stackoverflow.com/questions/24335692/large-matrices-in-r-long-vectors-not-supported-yet) but not the one you mention, which I _should_ have found of course. Thanks a lot! — user1657021, Jun 15 '20 at 13:00
It is also mentioned in this book now: https://bookdown.org/yihui/rmarkdown-cookbook/cache-lazy.html — Yihui Xie, Jun 15 '20 at 14:21

Rmarkdown: long vectors not supported yet

0 Answers0