0

I have a COW Table and able to insert and update the records using Glue ETL with out any issues. How ever when i try to delete the records for some records i am getting the following error:

  An error occurred while calling o115.pyWriteDynamicFrame.
  : org.apache.hudi.exception.HoodieUpsertException: Failed to delete for commit time 20220408160018 at org.apache.hudi.table.action.commit.SparkDeleteHelper.execute(SparkDeleteHelper.java:119)at org.apache.hudi.table.action.commit.SparkDeleteCommitActionExecutor.execute(SparkDeleteCommitActionExecutor.java:46)

Not sure why it is throwing this error while trying to delete some records but not for others.

Here is the Hudi write configs i am using

hudiWriteConfig = {
    'className' : 'org.apache.hudi',
    'hoodie.table.name': hudi_table_name,
    'hoodie.datasource.write.operation': 'delete',
    'hoodie.datasource.write.table.type': 'COPY_ON_WRITE',
    'hoodie.datasource.write.precombine.field': 'date',
    'hoodie.datasource.write.recordkey.field': 'name',
    'hoodie.datasource.write.partitionpath.field': 'name:SIMPLE,year:SIMPLE,month:SIMPLE,day:SIMPLE',
    'hoodie.datasource.write.keygenerator.class': 'org.apache.hudi.keygen.CustomKeyGenerator',
    'hoodie.deltastreamer.keygen.timebased.timestamp.type': 'MIXED',
    'hoodie.deltastreamer.keygen.timebased.input.dateformat': 'yyyy-mm-dd',
    'hoodie.deltastreamer.keygen.timebased.output.dateformat':'yyyy/MM/dd'
}
Sateesh K
  • 1,071
  • 3
  • 19
  • 45
  • It's not an issue. The record key i was trying to delete and the partition doesn't exists and hence Hudi is behaving ax expected. – Sateesh K Apr 13 '22 at 13:08

0 Answers0