1

i am creating one pipeline from sql server instance in compute engine and i want to migrate this data to bigquery but in previews instance its all fine and i can to watch the rows without problem but when i run the deployment in datafusion the instance give me the next error

0:c.g.c.h.i.b.BigQueryFactory@76] - Creating BigQuery from given credential.
2021-05-24 21:22:28,967 - INFO  [Executor task launch worker for task 0:c.g.c.h.i.b.BigQueryFactory@76] - Creating BigQuery from default credential.
2021-05-24 21:22:28,978 - INFO  [Executor task launch worker for task 0:c.g.c.h.i.b.BigQueryFactory@76] - Creating BigQuery from given credential.
2021-05-24 21:22:28,991 - INFO  [Executor task launch worker for task 0:c.g.c.h.i.b.o.ForwardingBigQueryFileOutputFormat@76] - Delegating functionality to 'AvroOutputFormat'.
2021-05-24 21:22:31,811 - ERROR [Executor task launch worker for task 0:o.a.s.u.Utils@91] - Aborting task  
**java.lang.IllegalArgumentException:** **Input record does not contain the DatabaseLogId field.** ```

what is your suggestion about this
FRANCISCO FELIPE ANDRADE SANCH
  • 11
  • 1

1 Answers1

0

The column name must be the same in input and output when you simply use SQLServer Source --> BQ Table Sink.

e.g:

query source: select dt as DatabaseLogId

column in sink: data_base_log_id (will throw your exception)

column in sink: DatabaseLogId (OK)

Ricardo Krohn de Jesus
  • 1