0

I am trying to add the Hudi Sink Connector to AWS MSK using the below config

    bootstrap.servers=******
    connector.class=org.apache.hudi.connect.HoodieSinkConnector
    tasks.max=4
    flush.size=10
    s3.region=us-east-1
    key.converter=org.apache.kafka.connect.storage.StringConverter
    value.converter=org.apache.kafka.connect.storage.StringConverter
    value.converter.schemas.enable=false
    topics=hudi-test-topic
    hoodie.table.name=hudi-test-topic
    hoodie.table.type=MERGE_ON_READ
    hoodie.base.path=s3://data_bucket/hudi-test-address
    hoodie.datasource.write.recordkey.field=id_field
    hoodie.datasource.write.partitionpath.field=ts_field
    hoodie.schemaprovider.class=org.apache.hudi.schema.SchemaRegistryProvider
    hoodie.kafka.commit.interval.secs=60

The connector status shows running but theres is no data that getting written to s3. When i check the logs it shows the below error

Bootstrap broker **************************** (id: -2 rack: null) disconnected

I have verified if the brokers are in a healthy state. I have also tested other sink connectors using similar config and those connectors are correctly producing data to s3.

0 Answers0