0

I am working on a new requirement to consume the records from Kafka and for that I have received the rest proxy APIs.

https://github.com/confluentinc/examples/blob/6.0.1-post/clients/cloud/rest-proxy/consume.sh

There seems to be already existing issue when calling GET records API and one has to call two times with 10 sec pause to read the records.

http://localhost:8082/consumers/cg1/instances/ci1/records

I am commiting the records manually by calling POST method offsets. Hence in order to read records I need to perform almost 6 api calls.

I dont know about REST API design. I am just curious to know, why a client must perform multiple calls for reading a records instead of single GET method.

For example, when client calls GET records method, Server can perform all (creating consumer, subscribing, and deleting) and return the records. Why should a client call all APIs in order to read records? Client has to take care of exceptions, logging for all api calls and it consumes more resource in client side.

We have been asked to poll every 5 mins. Hence client would be performing almost 1728 (6X12X24) calls everyday.

If it would have been just one call, we would then perform 288 calls per day.

Why should a client do multiple calls, instead of server taking care of it? is it the best design and what benefits one would achieve with this design?

userxsd
  • 1
  • 2
  • 1) In general, one action should be limited to one request 2) two calls aren't needed to consume, if you do it in the correct order https://github.com/confluentinc/kafka-rest/issues/432#issuecomment-695850955 3) you found a bug and doesn't expand out to other APIs, but less than 2k calls per day is nothing – OneCricketeer Jan 16 '21 at 16:09
  • we have already 1500 interfaces running already on my SOA system. It may seem 2k calls nothing. Just curios to know why would one need 5 call api calls to read records. – userxsd Jan 18 '21 at 16:47
  • You should only need one to initialize the consumer group (only ever done once) then another to actually read records that have already been produced. Then submitting the offsets is an acknowledgement that you've gotten (and processed) the records successfully, making one more call. – OneCricketeer Jan 18 '21 at 16:58
  • Thanks. I appreciate your help on this. I now understand it better. – userxsd Jan 19 '21 at 14:57

0 Answers0