0

I've been having trouble saving lightgbm boosters after running a model with the bonsai and the tidymodels package (as in this StackOverflow thread How to save Tidymodels Lightgbm model for reuse - however, I never managed to get it to work).

Therefore, reading about it (and as it says on the update in the abovementioned StackOverflow thread) led me to find out that the development version of the lightgbm package (version 4.0 I believe) supports using saveRDS and readRDS directly for their lightgbm models.

The thing is, I don't know how to install the development version of lightgbm. Reading this thread I can see that James, who's one of the maintainers of the lightgbm package, wrote this (https://github.com/tidymodels/bonsai/issues/34):

"If you already have a local installation of LightGBM, you can re-install from the latest development version by running the following:"

REPO_DIR="${HOME}/repos"
cd "${REPO_DIR}/lgb-dev"
git checkout master
git pull origin master
git submodule update --recursive

sh build-cran-package.sh \
    --no-build-vignettes

R CMD INSTALL \
    --with-keep.source \
    ./lightgbm_*.tar.gz

My problem is that I do not understand this code. Is it supposed to be run in R? (this does not work for me). Or in the command line? Which language is this?

If anyone can help me out regarding installation of the dev version of the lightgbm package, I would be really grateful

Andy
  • 109
  • 5
  • This is all on the command line. First you have to navigate to anywhere you want on you computer with `cd some_path`, then clone the repo with `git clone the_url.git` and then `cd` to the `lgb-dev` folder in that repo, then copy and paste the rest. – dcsuka Sep 25 '22 at 00:22

1 Answers1

3

The code you've provided is the preferred way to install the development version of {lightgbm} (the R package for LightGBM).

That can be run from any shell that understands Unix commands, as long as you have git and R installed. If you're on a Mac, you could for example use the Terminal application. If you're on a Windows machine, you could use Git for Windows.

Unfortunately, {lightgbm} cannot be installed from its git repo using R-only tools like remotes::install_github(). That is intentional, as described in https://github.com/tidymodels/bonsai/issues/34#issuecomment-1187589193.

If you are not comfortable using git and the command line, the following R-only solution could be used to install the latest development version of {lightgbm}.

library(httr)
library(jsonlite)

TEMP_DIR <- "tmp-lightgbm"

if (!dir.exists(TEMP_DIR)) {
    dir.create(TEMP_DIR)
}
builds_response <- httr::RETRY(
    verb = "GET"
    , url = "https://dev.azure.com/lightgbm-ci/lightgbm-ci/_apis/build/builds?branchName=refs/heads/master&resultFilter=succeeded&queryOrder=finishTimeDescending&%24top=1&api-version=7.1-preview.7"
)
build_id <- jsonlite::fromJSON(
    txt = httr::content(builds_response, as = "text")
    , simplifyDataFrame = FALSE
)[["value"]][[1L]][["id"]]
artifact_url <- paste0(
    "https://dev.azure.com/lightgbm-ci/lightgbm-ci/_apis/build/builds/"
    , build_id
    , "/artifacts?artifactName=R-package&api-version=7.1-preview.5&%24format=zip"
)
download.file(
    url = artifact_url
    , destfile = file.path(TEMP_DIR, "lightgbm-r.zip")
)
unzip(
    zipfile = file.path(TEMP_DIR, "lightgbm-r.zip")
    , exdir = file.path(TEMP_DIR, "extracted")
)
artifacts_dir <- file.path(TEMP_DIR, "extracted", "R-package")
tarball_path <- list.files(
    path = artifacts_dir
    , pattern = "lightgbm-[0-9.]+-r-cran\\.tar\\.gz"
    , full.names = TRUE
)[[1L]]
new_tarball_path <- file.path(artifacts_dir, "lightgbm.tar.gz")
file.rename(tarball_path, new_tarball_path)

install.packages(new_tarball_path, repos = NULL, type = "source")
James Lamb
  • 1,662
  • 12
  • 16
  • Hi James. Thank you so much for the detailed answer! Sorry for my late response, I got caught up with another project. I am unfortunately not so comfortable with git atm, so I tried your r-solution as far as installation goes. However, I received this error when trying to unzip - so after running: unzip( zipfile = file.path(TEMP_DIR, "lightgbm-r.zip") , exdir = file.path(TEMP_DIR, "extracted") ) I get Warning message: In unzip(zipfile = file.path(TEMP_DIR, "lightgbm-r.zip"), exdir = file.path(TEMP_DIR, : internal error in 'unz' code – Andy Oct 03 '22 at 09:20
  • That error makes it sound like maybe you do not have `gunzip.exe` or similar installed in your environment. You could try installing RTools (https://cran.r-project.org/bin/windows/Rtools/rtools42/rtools.html) to get that and other tools use to build R packages from source. – James Lamb Oct 03 '22 at 16:19