0

I've successfully built a pipeline for hierarchial time-series forecasting using the fable package. For now I've used multicore function from the future pacakge (in Databricks). And, for ~1k time series the runtime isn't too bad. Below is the code I have for that.

plan(multicore, workers = 14)
model_multicore_log_exts <- data_for_modeling %>%
    model(
          prophetML_additive_lin = prophet(log(dependent_variable) ~ exogenous_var1 + exogenous_var2 + exogenous_var3 + growth("linear") + season("year", type = "additive")),
          prophetML_multiplicative_lin = prophet(log(dependent_variable) ~ exogenous_var1 + exogenous_var2 + exogenous_var3 + growth("linear") + season("year", type = "multiplicative")),
          nn = NNETAR(log(dependent_variable) ~ trend() + season() + exogenous_var1 + exogenous_var2 + exogenous_var3, n_networks = 50)
          )

But, I have the need for increasing that number substantially.

So, I'm wondering if there is a way to distribute with sparkr. I cannot find anything in the fable CRAN documentation nor anything with a google search, so I am asking here.

Any suggestions/advice appreciated.

Thank you, Brian

Isaiah
  • 2,091
  • 3
  • 19
  • 28
Brian Head
  • 57
  • 4
  • I am not aware of fable package; what are you trying to parallelize here? Multiple model forms on same data? What did you imply by "1k time series"? – Vivek Atal Jan 05 '23 at 04:25

0 Answers0