0

We are using Transactional concept of Kafka where we consume the message from one topic and process it and produce the message on another topic. Our cluster has security enabled, so we have provided write operation to transaction-id using ACL. But when the transactional producer tries to execute commitTransaction method then it is throwing below error. Kindly suggest.

Caused by: org.apache.kafka.common.KafkaException: Unexpected error in TxnOffsetCommitResponse: Topic authorization failed.
    at org.apache.kafka.clients.producer.internals.TransactionManager$TxnOffsetCommitHandler.handleResponse(TransactionManager.java:1697) ~[kafka-clients-5.5.6-ce.jar:?]
    at org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1260) ~[kafka-clients-5.5.6-ce.jar:?]
    at org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109) ~[kafka-clients-5.5.6-ce.jar:?]
    at org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:575) ~[kafka-clients-5.5.6-ce.jar:?]
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:567) ~[kafka-clients-5.5.6-ce.jar:?]
 exception_class=org.apache.kafka.common.KafkaException, exception_message=Cannot execute transactional method because we are in an error state}

Earlier when starting my service i was getting TransactionId not authorized error then i ran below command :

bin/kafka-acls --bootstrap-server <server-name-comma-separated> --command-config adminclient-configs.conf --add --allow-principal User:<exact-user-name> --transactional-id 16000- --resource-pattern-type prefixed --operation Write --operation Read

After running above the error was gone and was able to up my service, but when i try to consume and produce message using transaction then during commitTransaction I got above error. I tried to give below acl but issue still present.

kafka-acls --bootstrap-server <server-name-comma-separated> --command-config adminclient-configs.conf --add --allow-principal User:<exact-user-name> --operation All --group <consumer-group-name>  --topic <consumer-topic>
kafka-acls --bootstrap-server <server-name-comma-separated> --command-config adminclient-configs.conf --add --allow-principal User:<exact-user-name> --operation All --topic <producer-topic>
Smith
  • 151
  • 3
  • 11
  • Error is self descriptive. We need more details about how you've authorized your broker and settings you added to the producer (did you add necessary ACL to the internal transaction topic?) – OneCricketeer Mar 31 '22 at 13:33
  • @OneCricketeer I edited and provided the ACL which I have given. – Smith Mar 31 '22 at 14:21
  • So you added the ACL to the producer and consumer topic, but there's still an internal transactions topic that is used – OneCricketeer Mar 31 '22 at 15:21

0 Answers0