Wrapping dplyr filter in function results in "Error: Result must have length 4803, not 3"

Question

I'm learning R for data analysis and using this Kaggle dataset. Following the movie recommendation script works, but when I try to generalize a dplyr code by making it a function I get an error:

I've tried troubleshooting some. It looks like the code stops at the filter and mutate functions.

The following works and gives the expected output.

genres <- df %>%
  filter(nchar(genres)>2) %>%
  mutate(
    separated = lapply(genres, fromJSON)
  ) %>%
  unnest(separated, .name_repair = "unique") %>%
  select(id, title, keyword = name) %>%
  mutate_if(is.character, factor)

Wrapping that code in a function results in an error message:

make_df <- function(list_df){
  df %>%
  filter(nchar(list_df)>2) %>%
  mutate(
    separated = lapply(list_df, fromJSON)
  ) %>%
  unnest(separated, .name_repair = "unique") %>%
  select(id, title, keyword = name) %>%
  mutate_if(is.character, factor)
}

Expected results:

> head(genres)
# A tibble: 6 x 3
#      id title                                    keyword        
#   <dbl> <fct>                                    <fct>          
# 1 19995 Avatar                                   Action         
# 2 19995 Avatar                                   Adventure      
# 3 19995 Avatar                                   Fantasy        
# 4 19995 Avatar                                   Science Fiction
# 5   285 Pirates of the Caribbean: At World's End Adventure      
# 6   285 Pirates of the Caribbean: At World's End Fantasy

Actual results:

> make_df(genres)
#  Error: Result must have length 4803, not 3 
# --- Traceback ---
# 12. stop(structure(list(message = "Result must have length 4803, not 3", 
#     call = NULL, cppstack = NULL), class = c("Rcpp::exception", 
#     "C++Error", "error", "condition"))) 
# 11. filter_impl(.data, quo) 
# 10. filter.tbl_df(., nchar(list_df) > 2) 
# 9. filter(., nchar(list_df) > 2) 
# 8. function_list[[i]](value) 
# 7. freduce(value, `_function_list`) 
# 6. `_fseq`(`_lhs`) 
# 5. eval(quote(`_fseq`(`_lhs`)), env, env) 
# 4. eval(quote(`_fseq`(`_lhs`)), env, env) 
# 3. withVisible(eval(quote(`_fseq`(`_lhs`)), env, env)) 
# 2. df %>% filter(nchar(list_df) > 2) %>% mutate(separated = lapply(list_df, 
#     fromJSON)) %>% unnest(separated, .name_repair = "unique") %>% 
#     select(id, title, keyword = name) %>% mutate_if(is.character, 
#     factor) 
# 1. make_df(genres)

Actual results without filter line:

> make_df(genres)
#  Error: Argument 'txt' must be a JSON string, URL or file. 
# 15. base::stop(..., call. = FALSE) 
# 14. stop("Argument 'txt' must be a JSON string, URL or file.") 
# 13. FUN(X[[i]], ...) 
# 12. lapply(list_df, fromJSON) 
# 11. mutate_impl(.data, dots, caller_env()) 
# 10. mutate.tbl_df(., separated = lapply(list_df, fromJSON)) 
# 9. mutate(., separated = lapply(list_df, fromJSON)) 
# 8. function_list[[i]](value) 
# 7. freduce(value, `_function_list`) 
# 6. `_fseq`(`_lhs`) 
# 5. eval(quote(`_fseq`(`_lhs`)), env, env) 
# 4. eval(quote(`_fseq`(`_lhs`)), env, env) 
# 3. withVisible(eval(quote(`_fseq`(`_lhs`)), env, env)) 
# 2. df %>% mutate(separated = lapply(list_df, fromJSON)) %>%  unnest(separated, 
#      .name_repair = "unique") %>% select(id, title, keyword = name) %>% 
#      mutate_if(is.character, factor) 
# 1. make_df(genres)

score 2 · Accepted Answer · answered Nov 11 '19 at 15:16

Your problem is actually connected to tidy programming. I am referring to rlang syntax

You just need to add the {{ }} in your code. Than it runs perfectly.

You can solve this with:

make_df <- function(list_df){
  df %>%
    filter(nchar({{ list_df }})>2) %>%
    mutate(
      separated = lapply({{ list_df }}, fromJSON)
    ) %>%
    unnest(separated, .name_repair = "unique") %>%
    select(id, title, keyword = name) %>%
    mutate_if(is.character, factor)
}

Than you can execute (make sure you delete the genres object in your environment, otherwise you give him a tibble with your former result instead of a string):

make_df(genres)

Output is:

# A tibble: 12,160 x 3
       id title                                    keyword        
    <dbl> <fct>                                    <fct>          
 1  19995 Avatar                                   Action         
 2  19995 Avatar                                   Adventure      
 3  19995 Avatar                                   Fantasy        
 4  19995 Avatar                                   Science Fiction
 5    285 Pirates of the Caribbean: At World's End Adventure      
 6    285 Pirates of the Caribbean: At World's End Fantasy        
 7    285 Pirates of the Caribbean: At World's End Action         
 8 206647 Spectre                                  Action         
 9 206647 Spectre                                  Adventure      
10 206647 Spectre                                  Crime          
# ... with 12,150 more rows

Works as expected, thank you! So if I understand correctly the {{ }} lets dplyr know that you are referring to something inside a dataframe, and not a global variable? — sc4s2cg, Nov 11 '19 at 15:52

score 1 · Answer 2 · answered Nov 11 '19 at 15:16

The problem is that you cannot use character strings to identify variables inside filter and mutate. The easiest way to solve your problem is to use filter_at and mutate_at:

make_df <- function(list_df){
  df %>%
  filter_at(vars(list_df), any_vars(nchar(.) > 2)) %>%
  mutate_at(vars(list_df), list(seperated = ~lapply(.x, fromJSON)) %>%
  unnest(separated, .name_repair = "unique") %>%
  select(id, title, keyword = name) %>%
  mutate_if(is.character, factor)
}

Alternatively, you could work with quasiquotations as described in this this question.

Wrapping dplyr filter in function results in "Error: Result must have length 4803, not 3"

2 Answers2

Linked