0

I'm interested in learning tidymodels and have tried to apply it to some exercises in Appied Predictive Modeling. This is Exercise 6.2. I would like to specify a Partial Least Squares (PLS) model to the permeability data set.

I have the following code that works all the way up to the tune grid. I've modeled my analysis off of Julia Silge's - Lasso regression with tidymodels and The Office found here.

You can see my script and the tune_grid error message below.

library(tidymodels)
library(tidyverse)
library(skimr)
library(plsmod)
library(caret)

library(AppliedPredictiveModeling)
data(permeability)


dim(fingerprints)
fingerprints <- fingerprints[, -nearZeroVar(fingerprints)]
dim(fingerprints)
df <- cbind(fingerprints, permeability)
df <- as_tibble(df)

perm_split <- initial_split(df)
perm_train <- training(perm_split)
perm_test <- testing(perm_split)

perm_rec<- recipe(permeability ~ ., data=perm_train) %>% 
  step_center(all_numeric(),-all_outcomes()) %>% 
  step_scale(all_numeric(),-all_outcomes()) 
  
perm_prep <- perm_rec %>% 
  prep()


perm_prep


pls_spec <- pls(num_comp = 4) %>% 
  set_mode("regression") %>% 
  set_engine("mixOmics")

wf <- workflow() %>% 
  add_recipe(perm_prep) 
  
pls_fit <- wf %>% 
  add_model(pls_spec) %>% 
  fit(data=perm_train)

pls_fit %>%  
  pull_workflow_fit() %>% 
  tidy()
    

set.seed(123)
perm_folds <-  vfold_cv(perm_train, v=10)

pls_tune_spec <- pls(num_comp = tune()) %>% 
  set_mode("regression") %>% 
  set_engine("mixOmics")

comp_grid <- expand.grid(num_comp = seq(from = 1, to = 20, by = 1))

doParallel::registerDoParallel()

set.seed(4763)

pls_grid <- tune_grid(
  wf %>% add_model(pls_tune_spec),
  resamples = perm_folds,
  grid = comp_grid
)

At this point I'm getting the following error: All models failed in tune_grid(). See the .notes column.

Two questions:

  1. Why is my tune grid failing and how can I fix it?
  2. How does one see the .note column.
Mutuelinvestor
  • 3,384
  • 10
  • 44
  • 75

1 Answers1

3

I am guessing that you may be using a Windows computer, because we currently have a bug in the CRAN version of tune for parallel processing on Windows. Try either:

  • training sequentially without parallel processing, or
  • installing the development version of tune where we have fixed this bug, via devtools::install_github("tidymodels/tune")

You should see results like this:

library(tidymodels)
library(plsmod)
library(AppliedPredictiveModeling)
data(permeability)

df <- cbind(fingerprints, permeability)
df <- as_tibble(df)

set.seed(123)
perm_split <- initial_split(df)
perm_train <- training(perm_split)
perm_test <- testing(perm_split)

set.seed(234)
perm_folds <-  vfold_cv(perm_train, v=10)

perm_rec <- recipe(permeability ~ ., data = perm_train) %>%
  step_nzv(all_predictors()) %>%
  step_center(all_numeric(), -all_outcomes()) %>% 
  step_scale(all_numeric(), -all_outcomes())

pls_spec <- pls(num_comp = tune()) %>% 
  set_mode("regression") %>% 
  set_engine("mixOmics")

comp_grid <- tibble(num_comp = seq(from = 1, to = 20, by = 5))

doParallel::registerDoParallel()

workflow() %>% 
  add_recipe(perm_rec) %>%
  add_model(pls_spec) %>%
  tune_grid(
    resamples = perm_folds,
    grid = comp_grid
  )
#> 
#> Attaching package: 'rlang'
#> The following objects are masked from 'package:purrr':
#> 
#>     %@%, as_function, flatten, flatten_chr, flatten_dbl, flatten_int,
#>     flatten_lgl, flatten_raw, invoke, list_along, modify, prepend,
#>     splice
#> 
#> Attaching package: 'vctrs'
#> The following object is masked from 'package:tibble':
#> 
#>     data_frame
#> The following object is masked from 'package:dplyr':
#> 
#>     data_frame
#> Loading required package: MASS
#> 
#> Attaching package: 'MASS'
#> The following object is masked from 'package:dplyr':
#> 
#>     select
#> Loading required package: lattice
#> 
#> Loaded mixOmics 6.12.2
#> Thank you for using mixOmics!
#> Tutorials: http://mixomics.org
#> Bookdown vignette: https://mixomicsteam.github.io/Bookdown
#> Questions, issues: Follow the prompts at http://mixomics.org/contact-us
#> Cite us:  citation('mixOmics')
#> 
#> Attaching package: 'mixOmics'
#> The following object is masked from 'package:plsmod':
#> 
#>     pls
#> The following object is masked from 'package:tune':
#> 
#>     tune
#> The following object is masked from 'package:purrr':
#> 
#>     map
#> # Tuning results
#> # 10-fold cross-validation 
#> # A tibble: 10 x 4
#>    splits           id     .metrics         .notes          
#>    <list>           <chr>  <list>           <list>          
#>  1 <split [111/13]> Fold01 <tibble [8 × 5]> <tibble [0 × 1]>
#>  2 <split [111/13]> Fold02 <tibble [8 × 5]> <tibble [0 × 1]>
#>  3 <split [111/13]> Fold03 <tibble [8 × 5]> <tibble [0 × 1]>
#>  4 <split [111/13]> Fold04 <tibble [8 × 5]> <tibble [0 × 1]>
#>  5 <split [112/12]> Fold05 <tibble [8 × 5]> <tibble [0 × 1]>
#>  6 <split [112/12]> Fold06 <tibble [8 × 5]> <tibble [0 × 1]>
#>  7 <split [112/12]> Fold07 <tibble [8 × 5]> <tibble [0 × 1]>
#>  8 <split [112/12]> Fold08 <tibble [8 × 5]> <tibble [0 × 1]>
#>  9 <split [112/12]> Fold09 <tibble [8 × 5]> <tibble [0 × 1]>
#> 10 <split [112/12]> Fold10 <tibble [8 × 5]> <tibble [0 × 1]>

Created on 2020-11-12 by the reprex package (v0.3.0.9001)

If you have an object like pls_grid with notes, you should be able to get to the column via pls_grid$.notes, or to see the first example via pls_grid$.notes[[1]].

Julia Silge
  • 10,848
  • 2
  • 40
  • 48