0

What's the difference between the sdf_register and the copy_to command in sparklyr?

When do you use each command?

Ark
  • 93
  • 2
  • 6

1 Answers1

1

sdf_register is a Spark specific function which registers SparkDataFrame in the metastore It doesn't involve data copying and is used for Spark interop.

copy_to copies local data to Spark. If used with SparkDataFrame it will collect and put it back to Spark.

user7839293
  • 11
  • 1