0

This question is related to why do I need to poll message hub, The documentation linked with that answer shows that Kafka will support a 'long-poll' concept, but there is no clear way in the existing nodejs support for message-hub to implement such a mechanism. The demo app provided for nodejs just uses a 250mSec timer interval to handle retrieving messages from the server. I'd like to replace that with a more sophisticated long-poll approach using the Kafka support long-poll approach: To avoid this we have parameters in our pull request that allow the consumer request to block in a "long poll" waiting until data arrives However the existing implementation does not appear to allow for configuring any such kind of parameter, nor is it clear what the necessary parameter would be. The prototype for the get function is defined as: 

MessageHub.ConsumerInstance.prototype.get(topicName [toValue])

Retrieves a message from the provided topic name.

    topicName - (String) (required), the topic to retrieve messages from.
    toValue - (Boolean) (optional), unwraps base64 encoded messages, if true. Defaults to true.

Returns a Promise object which will be fulfilled when the request to the service resolves.

so no config options. Alternately, could you provide a link the documentation which defines the URLs and available options for those URLs which are implemented in the message-hub.js module?

Bob Dill
  • 1,000
  • 5
  • 13

1 Answers1

1

that one you're mentioning is an npm module built on top of the REST API that Message Hub provides - which is essentially the Confluent REST Proxy API minus the AVRO support http://docs.confluent.io/2.0.1/kafka-rest/docs/api.html

That npm is not offering you a full Kafka api. It is offering just a subset of the REST API

The long poll which the the Kafka doc refer to is the timeout in the Java Consumer poll https://kafka.apache.org/0102/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#poll(long)

A good client for Node.js is https://github.com/Blizzard/node-rdkafka

Please see our coding sample in https://github.com/ibm-messaging/message-hub-samples

Edoardo Comar
  • 531
  • 2
  • 5
  • Thanks. The first URL is an essential one for me to understand how to make this work in nodeJS. What I see in the example (3rd URL) is that nodeJS returns immediately if there are no messages to be consumed rather than waiting until a timeout occurs or a message is provided. Query parameters for Consumer/Get only allows maxbytes, but config options implies that max.timeout.ms should default to 1000ms. This does not occur - rest returns instantly with an empty result if there are no messages to retrieve. – Bob Dill Jun 07 '17 at 15:28