Snowflake Table from Databricks using Python/Scala

Question

I want to create a table and load data into it in Snowflake from Databricks using Python/Scala.

Below is my code snippet. I'm getting the below error. How can I create the table first if not exists in Databricks notebook using Python or Scala and then load the data?

If so, what functions do I need to use? Below gives me an error.

df1.write.format("snowflake").options(sfOptions).option("dbtable","TEST_TABLE")
         .mode(SaveMode.Append)

Without an error message, this question is missing a [mcve]. I will vote to close for now. — halfer, May 25 '23 at 11:31

score 4 · Answer 1 · answered Oct 12 '21 at 11:58

If you use Scala code then your df write should look like this:

df.write
    .format(SNOWFLAKE_SOURCE_NAME)
    .options(sfOptions)
    .option("dbtable", "t2")
    .mode(SaveMode.Append)
    .save()

If you use Python code then your df write should look like this:

df.write
    .format(SNOWFLAKE_SOURCE_NAME)
    .options(**sfOptions)
    .option("dbtable", "t2")
    .mode(SaveMode.Append)
    .save()

where:

SNOWFLAKE_SOURCE_NAME = "net.snowflake.spark.snowflake"

Observe there is a difference between the options on Scala vs Python.

Snowflake Table from Databricks using Python/Scala

1 Answers1