Use parallel processes for API Call use future

Question

My shiny app displays 10 different phovotvoltaic systems. For every pv-system the app does the exact same calculations. I use modules in shiny to reduce the amount of lines. The process is like follwing:

The data is loaded from an API (Module)
The data from the api is recalculated (Module)
The data is graphically processed with infoboxes, ggplot etc.

Currently the Code gets executed and the waiting time for loading the API data is very long. I want to reduce the waiting time for the enduser by using the future package.

Currently i dont understand how to implement the future() into my shiny modules.

For example:

API api <- function(id,df, api_siteid, api_key, startmonth, startdate, url,db_location){ some code }

  api(
    id = "bb55",
    df = read.csv("Anlagen/Barbarastrasse_55-1/bb55.csv"),
    api_siteid =  "8784240",
    api_key = "EY0HXOAH1Y6fgMY912RGGPUGJD5C2G",
    startmonth = "apr2029",
    startdate <- "20200401",
    url = "https://monitoringapi.solaredge.com/site/",
    db_location = "Anlagen/barbstreet_55-1/bb55.csv"
  )
bb55_act <- read.csv("Anlagen/Barbarastrasse_55-1/bb55.csv")

Module Server

ib_prod_month_pkWp_UI  <- function(id){
  valueBoxOutput(NS(id,"ib_prod_pkwp_month"))
}

ib_prod_month_pkWp_server <- function(id,df, kwp_size){
  moduleServer(id, function(input, output, session){ 
    output$ib_prod_pkwp_month <- renderInfoBox({
      result <- df %>%
        mutate(timestamp = as.POSIXct(timestamp, format = "%Y-%m-%d %H:%M:%S")) %>%
        filter(year(timestamp) == year(Sys.Date()) &
                 month(timestamp) == month(Sys.Date())) %>%
        summarise(kwh = sum(prod) / as.numeric(kwp_size))
      valueBox(
        subtitle = "Prod. diesen Monat p. kWp",
        prettyNum(
          result$kwh,
          big.mark = ".",
          decimal.mark = "," ,
          digits = 4,
          scientific = FALSE
        )
      )
      
    })
  })
}

ib_prod_today_pkWp_server("bb55_1_ib_prod_pkwp_today", df = bb55_act, kwp_size = bb55_size)

The result of api() is to write the new data into a csv file. After that the csv file gets loaded into bb55_act = df.

This process is repeated for each pv-system. I would like to parallelize the API call beacuse this is the predominant time consuming part of the code.

Thanks for your help!

I changed some letters in the code, its not the real api code — jogabonito, Aug 09 '21 at 18:31

Gregor de Cillia · Answer 1 · 2021-08-01T15:01:39.313

As far as I can tell, the usage of {future} will only affect the performance if several users are using the app at the same time. Since you are saving the API responses in the local file system, I'd suggest you use callr::r_bg() to run the API requests in a background job and grab the responses using shiny::reactiveFileReader().

See the following example where I made some simplifications

api_args <- list(
  list(id = "first_resource", destfile = "~/api_files/destfile1.csv"),
  list(id = "second_resource", destfile = "~/api_files/destfile2.csv")
)

make_api_calls <- function() {
  lapply(api_args, function(x) {
    callr::r_bg(api, x)
  })
}

mod_valuebox_server <- function(id) {
  moduleServer(id, function(input, output, session) {
    df <- reactiveFileReader(api_args[[1]]$destfile)
    result <- df %>%
      summarise(kwh = sum(prod) / as.numeric(kwp_size))
    output$balueBox <- valueBox(result$kwh)
  })
}

Simplifications

I assume that the API function has one argument defining the resource to be grabbed and one argument for the download location
The processing takes place in the "valuebox-module"
The "valuebox-module" only renders the results of the first API response

Use parallel processes for API Call use future

1 Answers1