5

I'm Using Spark 2.0.2 in combination with sparklyr 0.5.4-9004 on RStudio, in a windows server.

Every once in a while, when I try to collect, read or write data from the spark server, I'm getting the following error:

Error in UseMethod("invoke") : 
  no applicable method for 'invoke' applied to an object of class "character"

I'm not sure if it's a server issue, or some problem with sparklyr.

So far, the only way I have to solve this, is restarting the sparklyr connection with Spark.

Here is the simple code I'm using:

sc <- spark_connect(master = "spark://####:7077")

tibble_file_parquet <- spark_read_parquet(
  sc, "file_parquet", path, overwrite = TRUE)

result <- tibble_file_parquet %>% count(x, y)

collect(result)
zero323
  • 322,348
  • 103
  • 959
  • 935
Igor
  • 913
  • 1
  • 8
  • 18
  • I get invoke method errors when my cluster is out of memory. But I'm on a shared cluster with different users so unable to do a detailed root cause analysis. But yes, resetting the sparklyr connection is the only solution I've found as well – DivyaJyoti Rajdev Apr 10 '18 at 19:28
  • How did you solve this error ? I am getting the same and I have tried everything but in vain – Yogesh Kumar May 21 '18 at 07:35

0 Answers0