I'm using Kafka Connect with Debezium MongoDB Connector to export some data from a CosmoDB (MongoDB API).
My connector config is bellow:
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
name: ingest-ledger-small-collections
labels:
strimzi.io/cluster: edh
spec:
class: io.debezium.connector.mongodb.MongoDbConnector
config:
tasks.max: 1
mongodb.hosts: <host>
mongodb.user: <username>
mongodb.name: mongo
mongodb.password: <password>
mongodb.ssl.enabled: true
capture.mode: change_streams_update_full
snapshot.max.threads: 1
transforms: unwrap
transforms.unwrap.type: io.debezium.connector.mongodb.transforms.ExtractNewDocumentState
transforms.unwrap.drop.tombstones: false
transforms.unwrap.delete.handling.mode: drop
key.converter: org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable: false
value.converter: org.apache.kafka.connect.json.JsonConverter
value.converter.schemas.enable: false
publish.full.document.only: true
When I start the process snapshot is take and works well, however, after that it gives the error bellow and stops to work.
io.debezium.DebeziumException: Failed to find starting position 'Timestamp{value=0, seconds=0, inc=0}' in oplog
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.readOplog(MongoDbStreamingChangeEventSource.java:189)
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$streamChangesForReplicaSet$0(MongoDbStreamingChangeEventSource.java:114)
at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.execute(ConnectionContext.java:288)
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.streamChangesForReplicaSet(MongoDbStreamingChangeEventSource.java:113)
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.execute(MongoDbStreamingChangeEventSource.java:94)
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.execute(MongoDbStreamingChangeEventSource.java:51)
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:166)
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:127)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Someone have an ideia how to solve?
