1

I'm using Kafka 0.10 REST proxy service, I have the following questions please:

1, Can I have two consumer instances in the same consume group?

2, How long the timeout of consumer instance once it was created or last used?

3, Is there Java or Python API for using Kafka REST APIs?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Jack
  • 5,540
  • 13
  • 65
  • 113

1 Answers1

1

1) yes you can have multiple consumers in the same consumer group

2) the timeout is set in the configuration parameters for the REST Proxy so this may vary from installation to installation but the default is 5 minutes (300000 ms)

From the Confluent documentation site https://docs.confluent.io/current/kafka-rest/docs/config.html

consumer.instance.timeout.ms
Amount of idle time before a consumer instance is automatically destroyed.

Type: int
Default: 300000

3) There is no provided Java or Python Wrapper API for using Kafka REST API because most people would just use the native Kafka Java or Python APIs and bypass the extra step of going through a REST Proxy. If that is not an option in your use case then you might search github for third party REST API wrappers in Java or Python. I know there is one in JavaScript for use in browser apps. It should not be too hard to build a Java or Python wrapper on the REST API if one does not already exist.

Hans Jespersen
  • 8,024
  • 1
  • 24
  • 31
  • Are you sure, we can register multiple consumer instances in Rest proxy in the same group? When I created two, they both no any response, but when I delete one, the other works. – Jack Apr 02 '18 at 14:54
  • 1
    You probably only had one partition in your topic because that’s exactly what you would see when two consumers in a group and only one partition in the topic – Hans Jespersen Apr 02 '18 at 22:56
  • @HansJespersen, are you sure you can have multiple consumers in one group, using kafka-rest? As far as I know, this is a known [unsolved issue](https://github.com/confluentinc/kafka-rest/issues/341). – sevavietl Sep 26 '18 at 12:25
  • how to create consumer groups and consumer instances is documented here https://docs.confluent.io/current/kafka-rest/docs/api.html#consumers – Hans Jespersen Sep 26 '18 at 14:44
  • @HansJespersen, thank you for the response. Have you looked at the issue I have mentioned? I have no problem creating multiple consumers under the same consumer group, but they are not working properly. Maybe you can help me to understand how to use it by commenting on the issue. Thank you in advance. – sevavietl Sep 27 '18 at 06:42
  • Are you using the V1 or the V2 REST API? Perhaps you can post a few curl commands to demonstrate the issue – Hans Jespersen Sep 27 '18 at 06:58
  • 1
    @HansJespersen, thank you for the quick response. I will prepare the demo – sevavietl Sep 27 '18 at 08:26