I am getting the following error:
"ERROR WorkerSourceTask(id=test-mysql-dbc-source-0) Failed to flush, timed out while waiting for producer to flush outstanding N messages.
ERROR Failed to commit offsets. (org.apache.kafka.connect.runtime.WorkerSourceTask:XXX)"
The Setup:
I have an EC2 instance in AWS (t2.medium - 2 cores 4GB RAM) which serves as Kafka Server. Another EC2 instance has a sandbox MySQL database and Kafka Connect with Confluent JDBC Source Connector. Python script inserts a couple of rows in a table randomly, to simulate some activity. On my laptop I opened Kafka Console Consumer to read the messages from Kafka Server.
Ports 22, 3306, 9092, 2888 are open to all IP addresses on both EC2 instances.
Below are config files I used for Kafka Connect Source
Config files:
connect-standalone.properties
bootstrap.servers=FIRST_EC2_PUBLIC_IP:9092
key.converter.schemas.enable=true
value.converter.schemas.enable=true
offset.storage.file.filename=/tmp/connect.offsets
acks=1
compression.type=lz4
plugin.path=/usr/share/java
jdbc_source.properties
name=test-mysql-jdbc-source
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
connection.url=jdbc:mysql://localhost:3306/DB_NAME
connection.user=root
connection.password=DB_PASSWORD
table.whitelist=TEST_TABLE
mode=timestamp+incrementing
incrementing.column.name=ID_RECORD
timestamp.column.name=TIMESTMP
topic.prefix=mysql-source-
acks=1
compression.type=lz4
I tried to manipulate with various settings and options. Mostly I tried to play around with offset.flush.timeout.ms and buffer.memory options as advised in this link.
The Behavior of Producer:
So basically, after starting a producer on my second EC2 instance, I can see the messages on my laptop in kafka console consumer, so it works. I can see new records as they appear for some time. However, very often when a new row in a table is created, producer just does not push it to kafka server (first EC2 instance) throwing above mentioned error for about 5 to 20 minutes. The number of outstanding messages doesn't get big. Most of the time it is 2-6 messages. After throwing an error for 5-20 minutes it finally sends the data and the console consumer consumes the data and work fine for some time and after that an error appears again.
If I manually stop the producer and start it again, the outstanding messages flush instantly and can be seen in a console consumer on my laptop.
Could you please point me, where the problem can be, and what can cause such a behavior.
