0

I came up with the below class that takes in 3 parameters and returns consumed Kafka messages.

package kafka

@Grab(group='org.apache.kafka', module='kafka-clients', version='2.8.0')
@Grab(group='org.slf4j', module='slf4j-simple', version='2.0.0')

import com.dell.techops.TechopsKafkaProducer

import org.apache.kafka.clients.consumer.KafkaConsumer
import org.apache.kafka.clients.consumer.ConsumerConfig
import org.apache.kafka.common.serialization.StringDeserializer
import java.time.Duration
import groovy.util.logging.Slf4j


class KConsumer {
  def consumer

  KConsumer(String NexusServers, String topic, String groupId) {
    def props = new Properties()
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers)
    props.put(ConsumerConfig.GROUP_ID_CONFIG, groupId)
    props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName())
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName())

    consumer = new KafkaConsumer<String, String>(props)
    consumer.subscribe([topic])
  }
  
  List<String> consumeMessages() {
    def messages = []
    while (true) {
      def records = consumer.poll(100)
      if (!records.isEmpty()) {
        records.each { record ->
          messages.add(record.value())
        }
        return messages
      }
    }
  }
 void close() {
   consumer.close()
  }
}  

Using the below class like (Code snippet.)

def servers = "localhost:9092"
def topic = "my-topic"
def groupId = "my-group"

def consumer = new KConsumer(servers, topic, groupId)
messages = consumer.consumeMessages()
consumer.close()

Everything works fine, but, I'm interested to pass an optional parameter called key, if passed, it should do return the messages of only that specific key. I have the logic for that something like this .

 def messages = []
 def records = consumer.poll(100)
 for (ConsumerRecord<String, String> record : records) {
        if (record.key() == key) {
          messages.add(record.value())
        }
      }

Question here is, do I have to make the Class necessarily to take 4 parameters by default or is there an option like *args in Groovy, akin to Python. I wanted to handle key parameter as an optional one. Also, do I have to come up with a new function to cater that or is it possible to repurpose the consumeMessages function.

Rafa S
  • 45
  • 5
  • 1
    My suggestion is to pass all parameters as a map which can handle optional values. See my answer here: https://stackoverflow.com/a/65186274/5595263. – Catalin Jun 14 '23 at 23:07

1 Answers1

1

Groovy basics say, that you can define an optional parameter and fill it with default value, if the method was called without it:

class KConsumer {
  KConsumer(String NexusServers, String topic, String groupId, String key = null) {}

  //or

  List<String> consumeMessages( String key = null ) {}
}


// ...

def consumer1 = new KConsumer(servers, topic, groupId)
def consumer2 = new KConsumer(servers, topic, groupId, 'someKey' )

// or

consumer.consumeMessages()
consumer.consumeMessages( 'someKey' )
injecteer
  • 20,038
  • 4
  • 45
  • 89
  • Thank you, that clarifies. But I've to come up with a new function to filter messages with the passed key right ? I believe I can't modify the existing `consumeMessages` function to handle both the cases ? – Rafa S May 12 '23 at 17:03
  • I put an example of the modified method signature with an optional param in my sample – injecteer May 12 '23 at 20:04