DuckDB won't compile on free AWS EC2 instance. Are precompiled packages the solution?

Question

I'm trying to set up a shiny server on the free tier AWS EC2 to test my app but I can't get all the packages compiled and installed.

e.g. duckdb

in the terminal connected to my instance I paste:

sudo su - -c "R -e \"install.packages(c('duckdb'), repos='http://cran.rstudio.com/')\""

It just hands:

> install.packages('duckdb',INSTALL_opts = '--no-lock')
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/duckdb_0.2.4.tar.gz'
Content type 'application/x-gzip' length 1964806 bytes (1.9 MB)
==================================================
downloaded 1.9 MB

* installing *source* package ‘duckdb’ ...
** package ‘duckdb’ successfully unpacked and MD5 sums checked
staged installation is only possible with locking
** using non-staged installation
** libs
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -DDUCKDB_DISABLE_PRINT -Iduckdb/src/include -Iduckdb/third_party/fmt/include -Iduckdb/third_party/re2 -Iduckdb/third_party/miniz -Iduckdb/third_party/utf8proc/include -Iduckdb/third_party/utf8proc -Iduckdb/third_party/libpg_query/include -Iduckdb/third_party/libpg_query -Iduckdb/third_party/concurrentqueue -Iduckdb/extension/parquet/include -Iduckdb/third_party/parquet -Iduckdb/third_party/snappy -Iduckdb/third_party/thrift -Iduckdb/third_party/zstd -Iduckdb -DBUILD_PARQUET_EXTENSION -DDUCKDB_BUILD_LIBRARY    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-jbaK_j/r-base-3.6.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c duckdbr.cpp -o duckdbr.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -DDUCKDB_DISABLE_PRINT -Iduckdb/src/include -Iduckdb/third_party/fmt/include -Iduckdb/third_party/re2 -Iduckdb/third_party/miniz -Iduckdb/third_party/utf8proc/include -Iduckdb/third_party/utf8proc -Iduckdb/third_party/libpg_query/include -Iduckdb/third_party/libpg_query -Iduckdb/third_party/concurrentqueue -Iduckdb/extension/parquet/include -Iduckdb/third_party/parquet -Iduckdb/third_party/snappy -Iduckdb/third_party/thrift -Iduckdb/third_party/zstd -Iduckdb -DBUILD_PARQUET_EXTENSION -DDUCKDB_BUILD_LIBRARY    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-jbaK_j/r-base-3.6.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c duckdb/extension/parquet/column_reader.cpp -o duckdb/extension/parquet/column_reader.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -DDUCKDB_DISABLE_PRINT -Iduckdb/src/include -Iduckdb/third_party/fmt/include -Iduckdb/third_party/re2 -Iduckdb/third_party/miniz -Iduckdb/third_party/utf8proc/include -Iduckdb/third_party/utf8proc -Iduckdb/third_party/libpg_query/include -Iduckdb/third_party/libpg_query -Iduckdb/third_party/concurrentqueue -Iduckdb/extension/parquet/include -Iduckdb/third_party/parquet -Iduckdb/third_party/snappy -Iduckdb/third_party/thrift -Iduckdb/third_party/zstd -Iduckdb -DBUILD_PARQUET_EXTENSION -DDUCKDB_BUILD_LIBRARY    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-jbaK_j/r-base-3.6.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c duckdb/extension/parquet/parquet-extension.cpp -o duckdb/extension/parquet/parquet-extension.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -DDUCKDB_DISABLE_PRINT -Iduckdb/src/include -Iduckdb/third_party/fmt/include -Iduckdb/third_party/re2 -Iduckdb/third_party/miniz -Iduckdb/third_party/utf8proc/include -Iduckdb/third_party/utf8proc -Iduckdb/third_party/libpg_query/include -Iduckdb/third_party/libpg_query -Iduckdb/third_party/concurrentqueue -Iduckdb/extension/parquet/include -Iduckdb/third_party/parquet -Iduckdb/third_party/snappy -Iduckdb/third_party/thrift -Iduckdb/third_party/zstd -Iduckdb -DBUILD_PARQUET_EXTENSION -DDUCKDB_BUILD_LIBRARY    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-jbaK_j/r-base-3.6.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c duckdb/extension/parquet/parquet_reader.cpp -o duckdb/extension/parquet/parquet_reader.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -DDUCKDB_DISABLE_PRINT -Iduckdb/src/include -Iduckdb/third_party/fmt/include -Iduckdb/third_party/re2 -Iduckdb/third_party/miniz -Iduckdb/third_party/utf8proc/include -Iduckdb/third_party/utf8proc -Iduckdb/third_party/libpg_query/include -Iduckdb/third_party/libpg_query -Iduckdb/third_party/concurrentqueue -Iduckdb/extension/parquet/include -Iduckdb/third_party/parquet -Iduckdb/third_party/snappy -Iduckdb/third_party/thrift -Iduckdb/third_party/zstd -Iduckdb -DBUILD_PARQUET_EXTENSION -DDUCKDB_BUILD_LIBRARY    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-jbaK_j/r-base-3.6.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c duckdb/extension/parquet/parquet_statistics.cpp -o duckdb/extension/parquet/parquet_statistics.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -DDUCKDB_DISABLE_PRINT -Iduckdb/src/include -Iduckdb/third_party/fmt/include -Iduckdb/third_party/re2 -Iduckdb/third_party/miniz -Iduckdb/third_party/utf8proc/include -Iduckdb/third_party/utf8proc -Iduckdb/third_party/libpg_query/include -Iduckdb/third_party/libpg_query -Iduckdb/third_party/concurrentqueue -Iduckdb/extension/parquet/include -Iduckdb/third_party/parquet -Iduckdb/third_party/snappy -Iduckdb/third_party/thrift -Iduckdb/third_party/zstd -Iduckdb -DBUILD_PARQUET_EXTENSION -DDUCKDB_BUILD_LIBRARY    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-jbaK_j/r-base-3.6.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c duckdb/extension/parquet/parquet_timestamp.cpp -o duckdb/extension/parquet/parquet_timestamp.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -DDUCKDB_DISABLE_PRINT -Iduckdb/src/include -Iduckdb/third_party/fmt/include -Iduckdb/third_party/re2 -Iduckdb/third_party/miniz -Iduckdb/third_party/utf8proc/include -Iduckdb/third_party/utf8proc -Iduckdb/third_party/libpg_query/include -Iduckdb/third_party/libpg_query -Iduckdb/third_party/concurrentqueue -Iduckdb/extension/parquet/include -Iduckdb/third_party/parquet -Iduckdb/third_party/snappy -Iduckdb/third_party/thrift -Iduckdb/third_party/zstd -Iduckdb -DBUILD_PARQUET_EXTENSION -DDUCKDB_BUILD_LIBRARY    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-jbaK_j/r-base-3.6.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c duckdb/extension/parquet/parquet_writer.cpp -o duckdb/extension/parquet/parquet_writer.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -DDUCKDB_DISABLE_PRINT -Iduckdb/src/include -Iduckdb/third_party/fmt/include -Iduckdb/third_party/re2 -Iduckdb/third_party/miniz -Iduckdb/third_party/utf8proc/include -Iduckdb/third_party/utf8proc -Iduckdb/third_party/libpg_query/include -Iduckdb/third_party/libpg_query -Iduckdb/third_party/concurrentqueue -Iduckdb/extension/parquet/include -Iduckdb/third_party/parquet -Iduckdb/third_party/snappy -Iduckdb/third_party/thrift -Iduckdb/third_party/zstd -Iduckdb -DBUILD_PARQUET_EXTENSION -DDUCKDB_BUILD_LIBRARY    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-jbaK_j/r-base-3.6.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c duckdb/amalgamation-1.cpp -o duckdb/amalgamation-1.o

I've read that it could be because it runs out of RAM, but I'm not sure how to get around this. Has anyone had this problem before? Are pre-compiled packages the solution?

Hannes Mühleisen · Accepted Answer · 2021-03-09T10:11:11.497

1

This is indeed due to the lack of RAM on the free tier VM. Binary packages would indeed solve this. But will see whether we can do something about that as well.

edited Mar 09 '21 at 10:11

answered Mar 09 '21 at 10:01

Hannes Mühleisen

2,542
11
13

1

Thanks @Hannes I have upgraded the VM RAM to 4GB and it now compiles and runs fine. Thank you for all your hard work with DuckDB. – HCAI Mar 09 '21 at 10:52
1

I had a look and compiling on the micro instance seems indeed a bit hopeless. – Hannes Mühleisen Mar 09 '21 at 20:13

DuckDB won't compile on free AWS EC2 instance. Are precompiled packages the solution?

1 Answers1