Alternatives to using `DT:replaceData()` when `server=FALSE` on shiny application deployed via kubernetes

Question

For various reasons I want to be able to use a proxied data table and replaceData while client side processing is being used i.e. DT::renderDataTable(..., server = FALSE).

Context

I have a shiny application/dashboard that communicates to a database and presents information to a user. The user is able to fill out a form in the application which will be added to the database and then the shiny app updates the data by making a query to the database to fetch the new information.

The application is currently being deployed via kubernetes using a LoadBalancer with the intention to use multiple replicas to scale up the application as needed. The application is not being run through shinyproxy.

Caveats

Currently, when the application is being run by a single replica (process) the application will behave perfectly fine and is able to use server=TRUE. However when I increase the number of processes/replicas to run, the data is not able to be presented to users unless server=FALSE is specified in renderDataTable. For a currently unknown reason but I suspect it might be due to the sessions not being sticky to IPs

While the code is able to function fine when server = TRUE if I want to allow multiple users to application they all cannot share a single process as the application will become very slow once multiple connections are made. As a result I likely need to use server=FALSE so each user is able to see the data at the cost of a very important functional detail (replaceData stops working). The product owner of the application is insistent that this behaviour remains intact as the data present is often large and requires some column sorting and paging to find a piece of information you want to look at. And when submitting a form, if I do not use replaceData and reconstruct the table from scratch the users previous table state is lost.

So while I could tear down the datatable and regenerate it within an observeEvent

observeEvent(input$button, {
    ...
    output$table = renderDataTable({DT::datatable(df(), selection = 'single', callback = 
    JS("$.fn.dataTable.ext.errMode = 'none';"))}, server = FALSE)
    ...
})

this would provide a solution that would yield unfavourable behaviour even though it will update the table accordingly.

Repoducible Example

This will create an application with a button and a table. Select a row on the table and then click the button. The expected behaviour would be that the table updates with 'new_content' on the row that is selected. This will only work when server=TRUE, nothing will happen when server=FALSE.

library(shiny)
library(DT)
data(iris)

server <- function(input, output, session) {
  iris$new_col = ''
  df = reactive({iris})
  output$table = renderDataTable({
      DT::datatable(df(), selection = 'single', 
        callback = JS("$.fn.dataTable.ext.errMode = 'none';"))}, server = FALSE) # When TRUE code works fine,,,
  proxy = dataTableProxy('table')

  observeEvent(input$button, {
    # This line would be replacing the write to a db
    iris[input$table_rows_selected, 'new_col'] <- 'changed'
    # This line would be replacing the query to the db to reflect changes the user (and potentially other users have made between loading the data previously.
    df <- reactive({iris})
    proxy %>% replaceData(df(), rownames = TRUE, resetPaging = FALSE)
  })
}
    
ui <- fluidPage(
  actionButton('button', 'Press Me'),
  DT::DTOutput('table') 
)

shinyApp(ui, server)

I have done a fairly extensive search on SO and this was the closest question I could find: DT Editing in Shiny application with client-side processing (server = F) throws JSON Error however this isn't actually answered and provides an answer of "it just does not work".

kubernetes.yaml (only look if you are a wizard)

I am including the yaml file incase there are some kubernetes boffins that know how to specifically solve the above issue with some clever trickery. The described problem might stem from sessions being swapped between replicas thus the data gets miscommunicated but I am honestly not the best at kubernetes... If that is the case and I would then be able to use server=TRUE within the shiny application then this would also solve the problem.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-appname
spec:
  replicas: 5
  selector:
    matchLabels:
      app: appname
  template:
    metadata:
      labels:
        app: appname
    spec:
      containers:
      - name: appname 
        securityContext:
            privileged: false
        image: appname:latest
        ports: 
        - name: http
          containerPort: 3838
---
apiVersion: v1
kind: Service
metadata:
  name: servive-appname
spec:
  ports:
  - name: http
    port: 3838
    protocol: TCP
    targetPort: 3838
  selector:
    app: appname
  type: LoadBalancer
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-appname
  annotations:
    nginx.org/websocket-services: "service-appname"
spec:
  tls:
  - hosts:
    - appname.url.com
  rules:
  - host: appname.url.com
    http:
      paths:
      - path: /
        backend:
          serviceName: service-appname
          servicePort: 3838

I don't think you'll be able to overcome this restriction. See the note in `?dataTableProxy`: `reloadData() only works for tables in the server-side processing mode`. `replaceData` uses `reloadData`. I'd rather focus on why your version using `server = TRUE` fails when multiple users connect. — ismirsehregal, Nov 24 '21 at 14:36
@ismirsehregal I am aware of what the documentation says - my question is still otherwise asking for an alternative to replaceData. — TJGorrie, Nov 24 '21 at 15:03
Sure, I thought so. I don't know if it meets the context, but would it be possible for you to use a [editable datatable](https://www.rstudio.com/blog/dt-0-4/) for the user inputs and save to the db once the user is done? This is possible with a client side table. — ismirsehregal, Nov 24 '21 at 15:56

ismirsehregal · Accepted Answer · 2021-12-04T07:28:27.333

Here is a client-side approach, building up on @jpdugo17's answer and @TJGorrie's initial example, using the stateSave option to maintain the table state on re-rendering. selectPage and updateSearch can be used along with dataTableProxy - the state of input$table_state$order needs to be passed as an option:

library(shiny)
library(DT)
data(iris)

iris$new_col <- ''

server <- function(input, output, session) {
  
  DF = reactiveValues(iris = iris)
  
  output$table <- DT::renderDataTable(expr = {
    if (is.null(isolate(input$table_state))) {
      DT::datatable(
        DF$iris,
        selection = 'single',
        callback = JS("$.fn.dataTable.ext.errMode = 'none';"),
        options = list(stateSave = TRUE)
      )
    } else {
      # print(isolate(input$table_state$order))
      DT::datatable(
        DF$iris,
        selection = 'single',
        callback = JS("$.fn.dataTable.ext.errMode = 'none';"),
        options = list(
          stateSave = TRUE,
          order = isolate(input$table_state$order),
          paging = TRUE,
          pageLength = isolate(input$table_state$length)
        )
      )
    }
  }, server = FALSE)
  
  proxy <- dataTableProxy('table')
  
  observeEvent(input$button, {
    DF$iris[input$table_rows_selected, c('new_col')] <- 'changed!'
  })

  observeEvent(DF$iris, {
    updateSearch(proxy, keywords = list(global = input$table_state$search$search, columns = NULL)) # see input$table_state$columns if needed
    selectPage(proxy, page = input$table_state$start/input$table_state$length+1)
  }, ignoreInit = TRUE, priority = -1)
}

ui <- fluidPage(
  actionButton('button', 'Press Me'),
  DT::DTOutput('table') 
)

shinyApp(ui, server)

Here is a related article.

the use of `selectPage` was very clever! However, it seems that updateSearch() should execute first than `SelectPage()` because pages change when the table is filtered. Please take a look. You can find this in line 42 — jpdugo17, Dec 03 '21 at 23:14

score 0 · Answer 2 · answered Dec 03 '21 at 02:56

We can try to use reactiveValues combined with the information of input$table_rows_selected. The server argument is equal to FALSE as requested.

library(shiny)
library(DT)
data(iris)

server <- function(input, output, session) {
  iris$new_col = ''
  df = reactiveValues(iris = iris)
  
  
  
  output$table = renderDataTable({
    DT::datatable(df$iris, selection = 'single', 
                  callback = JS("$.fn.dataTable.ext.errMode = 'none';"))}, server = FALSE) # When TRUE code works fine,,,
  
  
  
  observeEvent(input$button, {
    
    # This line would be replacing the write to a db
    df$iris[input$table_rows_selected, c('new_col')] <- 'changed!'
    
  })
}

ui <- fluidPage(
  actionButton('button', 'Press Me'),
  DT::DTOutput('table') 
)

shinyApp(ui, server)

This however, re-renders the table and by default resets the column sorting and paging position, which @TJGorrie mentioned to be undesired. — ismirsehregal, Dec 03 '21 at 07:18

score 0 · Answer 3 · answered Dec 03 '21 at 17:13

You can achieve session affinity using cookies if you are using kubernetes/ingress-nginx.

https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/

but from your yaml, you are using nginx.org's kubernetes-ingress, then you can read

https://github.com/nginxinc/kubernetes-ingress/blob/master/examples/session-persistence/README.md

but it's supported only in NGINX Plus.

Alternatives to using `DT:replaceData()` when `server=FALSE` on shiny application deployed via kubernetes

Context

Caveats

Repoducible Example

kubernetes.yaml (only look if you are a wizard)

3 Answers3