0

I am trying to merge an pyspark df (which I set up like this self.df.createOrReplaceTempView("df")) and a delta table, that is saved at path = '`/example/example/example/`' The path is working and the TempView also. I am getting following error: mismatched input 'NOT' expecting {<EOF>, ';'}(line 1, pos 981) When using my acutal path pos 981 points between WHEN[]NOT in the last line. I think I checked all the commas, all the " and tried to find a similiar example here on stackoverflow but I must be missing something... Can someone point me to where I went wrong?

self.spark.sql(f"MERGE INTO delta.{path} USING df "
                               f"ON (delta.{path}.a= df.a"
                               f"and delta.{path}.b= df.b) "
                               f"WHEN MATCHED THEN UPDATE SET "
                               f"delta.{path}.c= df.c,"
                               f"delta.{path}.d= df.d,"
                               f"delta.{path}.e= df.e, "
                               f"delta.{path}.a= df.c,"
                               f"delta.{path}.b= df.d"
                               f"WHEN NOT MATCHED THEN INSERT * ")
Alex Ott
  • 80,552
  • 8
  • 87
  • 132
Bondgirl
  • 107
  • 7
  • 1
    try to rewrite it to `MERGE INTO delta.{path} AS target`, and then replace all `delta.{path}` with `target` - it should simplify your code. Also, use `f"""....your full code ..."""` for whole statement, including newlines, etc. - it will be easier to read – Alex Ott Oct 14 '21 at 11:11
  • that were some good hints to simplify the code! somehow that also made me get rid of that error :) thanks! – Bondgirl Oct 14 '21 at 11:51

0 Answers0