0

Hello I'm working with flink-MSK-Hudi architecture and I want to ingest data in my AWS Glue catalog. Currently If I try to ingest the data in a S3 bucket in hudi format in worked good, the problem is when I set the hive properties in the hudi sink When I create the source and sink there is no problem, the problem is when I insert the data in the sink.

enter image description here

The sink code I use is:

  CREATE TABLE IF NOT EXISTS sink_table (
        Col1 String,
        Col2 String,
        Col3 timestamp,
        primary key (Col1) not enforced
  ) 
  WITH (
  'connector' = 'hudi',
  'write.tasks' = '4',
  'path' = '<Path in S3 where hudi table must be saved>', 
  'table.type'= 'MERGE_ON_READ',
  'hive_sync.enable' = 'true',
  'hive_sync.auto_create_db'= 'true',
  'hive_sync.db' = '<Database that already exists>',
  'hive_sync.table'= 'TSTTable'
  );


    INSERT INTO sink_table
      SELECT 
            *
      from Source_table;

Can someone help me? I just want to see my hudi table in my Athena catalog.

Valle1208
  • 43
  • 4

0 Answers0