3

I'm wondering if it's possible to substitute expressions into the i portion of data.table. I've set up a deeply nested list that contains preset parameters to perform subsetting depending on which sub-group a user has selected in a shiny app. I'm suspecting that it is not possible but am hoping to get confirmation one way or the other.

If it's not possible, I welcome any ideas for alternative solutions.

Basic example

library(data.table)

dt = data.table(
  y = rep(c('a','b'),5),
  x = 1:10
)

exp1 = "x > 5"

dt[y == "a" & substitute(exp1)]
# Error: operations are possible only for numeric, logical or complex types
Jamie
  • 1,793
  • 6
  • 16
  • 2
    Don't use eval parse. Use new env argument in data.table, which was created exactly to solve problems like yours. You can read more about it in new vignette https://rdatatable.gitlab.io/data.table/articles/datatable-programming.html – jangorecki Apr 29 '23 at 04:43
  • @jangorecki the new env argument is currently only available in the dev version. The CRAN version of 1.14.8 hasn't had this feature yet. – Hieu Nguyen Apr 29 '23 at 05:39
  • Yes I know, just run data.table::update_dev_pkg() – jangorecki Apr 29 '23 at 10:16
  • @jangorecki - I'm aware of the new env arg and wish that was a viable option. However, the shiny app I'm working on is part a larger group of apps all managed by the same `renv` set up. So I'm quite cautious towards using a dev version of any package in production. – Jamie May 01 '23 at 14:56

2 Answers2

4

My approach is like this:

library(data.table)
exp1 = "x > 5" |> str2lang()
(ll <- substitute(
    dt[y == "a" & e1],
    list(
        e1 = exp1
    )
))
eval(ll)

I think this is a good approach because it maintains idiomatic data.table's look. For example, if you want dt[y == "a" & x > 5, .(mm = length(x)), by = "y"]:

(ll <- substitute(
    dt[y == "a" & e1, .(mm = f1(e2)), by = e3], # Change right here and maintain data.table's look
    list(
        e1 = exp1,
        # e1 = "x > 5" |> str2lang(),
        e2 = quote(x), # Allow more flexibilities
        e3 = "y", # Allow more flexibilities
        f1 = quote(length) # Allow more flexibilities
    )
))
eval(ll)

It is also similar to the substitute2 and env approaches proposed in developmental version of data.table.

Hieu Nguyen
  • 492
  • 3
  • 8
  • 1
    Yes, env arg just wraps i, j, by into substitute calls – jangorecki Apr 29 '23 at 12:08
  • Awesome, thanks for this! In your second example, could you explain the need to add `quote` for `e2` and `f1`. Or refer me to some documentation around that? – Jamie May 01 '23 at 15:25
  • 1
    @Jamie Sorry for the late reply. Without `quote` in `e2`, it will become `e2 = x`. This will become the case of `e1` (i.e., find this variable `x` somewhere and substitute the **value** of variable `x` to `e2`). Try `x <- 3; (ll <- substitute(dt[y == "a" & e1, .(mm = f1(e2)), by = e3], list(e1 = "x > 5" |> str2lang(), e2 = x, e3 = "y", f1 = quote(length))))`, the result is `dt[y == "a" & x > 5, .(mm = length(3)), by = "y"]` – Hieu Nguyen May 20 '23 at 12:31
  • Alternatively, You can use `e2 = as.name("x")` instead of `e2 = quote(x)`. In this case, it helps in ease of interpretation is that we want the name/symbol `x` and **not the value of** `x`. – Hieu Nguyen May 20 '23 at 12:41
  • The canonical resources for this stuff are [R Manuals-The R language definition-Computing on the language](https://cran.r-project.org/doc/manuals/r-release/R-lang.html#Computing-on-the-language) and `help(quote)`, `help(expression)`, `help(parse)`... – Hieu Nguyen May 20 '23 at 13:14
0

It is possible using eval(parse(..)) but there are some concerns related to it.

library(data.table)

exp1 = "x > 5"
dt[y == "a" & eval(parse(text = exp1))]

#   y x
#1: a 7
#2: a 9
Ronak Shah
  • 377,200
  • 20
  • 156
  • 213
  • does `eval(str2lang(expr1))` also have concerns? or even `eval(str2expression(expr1))` – Onyambu Apr 29 '23 at 01:26
  • Hmm...I haven't used them so I am not sure. – Ronak Shah Apr 29 '23 at 01:50
  • @Onyambu the `eval(parse(..))`, `eval(str2lang(expr1))` and `eval(str2expression(expr1))` all require that the expressions are built by **string manipulation** (concatenate, replace...), then parsed. The authors of `data.table` has quite a [good summary](https://rdatatable.gitlab.io/data.table/articles/datatable-programming.html#use-of-parse-eval) about its problems. – Hieu Nguyen Apr 29 '23 at 06:28