İs it possible to create data pipeline with Google Cloud Data fusion using multiple database tables Update or Upsert?

Question

After using multiple database tables plugin and load data to bigquery I would like to make incremental load for every table in one data pipeline.

I wonder if I can use UPSERT on multiple database tables plugin. How can I overcome any advice ?

Answer 1

The UPSERT operation is not currently supported in the BigQuery Multi Table Sink plugin.

One suggestion is to store records to upsert into another BigQuery Dataset and use a BigQuery Execute Action to execute one merge operation into the desired destination table per each of the source tables.

See Google BQ - how to upsert existing data in tables? on how to execute the UPSERT operation using SQL.