I am using the following query to upsert into databricks table:
MERGE INTO my_target_table AS target
USING (SELECT MAX(__my_timestamp) AS checkpoint FROM my_source_table) AS source
ON target.name = 'some_name'
AND target.address = 'some_address'
WHEN MATCHED AND source.checkpoint IS NOT NULL THEN
UPDATE SET checkpoint = source.checkpoint
WHEN NOT MATCHED THEN
INSERT (name, address, checkpoint)
VALUES ('some_name', 'some_address', source.checkpoint)
Whenever it does 'insert', it also deletes from my_source_table. Any explanation why it deletes from my_source_table and can I avoid it, so the logic will stay the same, without anything being deleted from the source
EDIT: I was using view for my_source_table, once I changed that to a table the issue stoped.
