Kafka connect failed with OCI Stream service

Question

Im trying to setup Kafka connect with OCI Stream Service.

Below is Kafka connect docker configuration,

  connect:
    image: confluentinc/cp-kafka-connect-base:7.0.0
    hostname: connect
    ports:
      - 8083:8083
    container_name: connect
    environment:
      CONNECT_BOOTSTRAP_SERVERS: '<BOOTSTRAP_SERVER>'
      CONNECT_REST_ADVERTISED_HOST_NAME: connect
      CONNECT_REST_PORT: 8083
      CONNECT_GROUP_ID: compose-connect-group
      CONNECT_CONFIG_STORAGE_TOPIC: '<KAFKA_CONNECT_OCI_ID>'-config
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 3
      CONNECT_OFFSET_FLUSH_INTERVAL_MS: 200000
      CONNECT_OFFSET_STORAGE_TOPIC: '<KAFKA_CONNECT_OCI_ID>'-offset
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 3
      CONNECT_STATUS_STORAGE_TOPIC: '<KAFKA_CONNECT_OCI_ID>'-status
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 3
      CONNECT_KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_LOG4J_ROOT_LOGLEVEL: "INFO"
      CONNECT_LOG4J_LOGGERS: "org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR,com.mongodb.kafka=DEBUG"
      CONNECT_PLUGIN_PATH: /usr/share/confluent-hub-components    
      CONNECT_SECURITY_PROTOCOL: SASL_SSL
      CONNECT_SASL_MECHANISM: PLAIN
      CONNECT_SASL_JAAS_CONFIG: "<JASS_CONFIG_AS_IN_STREAMINGPOOL_SETTINGS>"
      CONNECT_PRODUCER_SECURITY_PROTOCOL: SASL_SSL
      CONNECT_PRODUCER_SASL_MECHANISM: PLAIN
      CONNECT_PRODUCER_SASL_JAAS_CONFIG: "<JASS_CONFIG_AS_IN_STREAMINGPOOL_SETTINGS>"
      CONNECT_CONSUMER_SECURITY_PROTOCOL: SASL_SSL
      CONNECT_CONSUMER_SASL_MECHANISM: PLAIN
      CONNECT_CONSUMER_SASL_JAAS_CONFIG: "<JASS_CONFIG_AS_IN_STREAMINGPOOL_SETTINGS>"
      CONNECT_CONSUMER_MAX_POLL_RECORDS: 1000
      KAFKA_HEAP_OPTS: "-Xms1G -Xmx4G"

Getting below error

ERROR [Worker clientId=connect-1, groupId=compose-connect-group] Uncaught exception in herder work thread, exiting:  (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before the position for partition <KAFKA_CONNECT_OCI_ID>-offset-0 could be determined
INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect)

I don't think there are network issues, Because when I have different topic name or wrong bootstrap server then I get different error, not even starts up. But for me this error gets after a min, mean time REST calls for GET requests work.

Please note that similar config for AWS MSK works for me.

Please help me to resolve the issue.

UPDATE: OCI streaming service doesn't support latest version of Kafka connect. It worked after I changed to 5.x

I'd recommend reaching out to OCI Customer Service and asking this question. That would let you provide more specific details, like which region, stream id, and detailed kafka logs. More info here: https://docs.oracle.com/en-us/iaas/Content/GSG/Tasks/contactingsupport.htm This looks like a timeout making a metadata request, most commonly caused by getting throttled. If you provide your stream id to customer support, they will be able to confirm. — Jon, Nov 09 '21 at 17:27
In addition to what Jon said, please take a look at our Kafka Connect documentation https://docs.oracle.com/en-us/iaas/Content/Streaming/Tasks/kafkacompatibility_topic-Kafka_Connect.htm — jb., Nov 09 '21 at 17:36

score 0 · Accepted Answer · answered Dec 22 '21 at 06:43

0

OCI streaming service doesn't support latest version of Kafka connect. It worked after I changed to 5.x

answered Dec 22 '21 at 06:43

Selvakumar Ponnusamy

5,363
7
40
78

Kafka connect failed with OCI Stream service

1 Answers1