0

I'm trying to write an unit test for a merge operation on a DeltaTable that I have. So far the approach has been to save the contents of a DataFrame to a local file so that it can then be read via DeltaTable.forPath(...)

Seq(MyData(column_1 = "column_1_value", column_2 = "column_2_value").toDF()
   .write.format("Delta").save(location)

val table = DeltaTable.forPath(location)

//execute code to be tested (includes a merge)

val resultOfMerge = table.toDF

//verify results of merge are correct

The problem is that takes too long, +20 seconds, sometimes +30 seconds. Is there a way I can construct a DeltaTable with sample data and avoid the time consuming write and read operations to disk?

edu
  • 428
  • 2
  • 10

0 Answers0