I came up with the below class that takes in 3 parameters and returns consumed Kafka messages.
package kafka
@Grab(group='org.apache.kafka', module='kafka-clients', version='2.8.0')
@Grab(group='org.slf4j', module='slf4j-simple', version='2.0.0')
import com.dell.techops.TechopsKafkaProducer
import org.apache.kafka.clients.consumer.KafkaConsumer
import org.apache.kafka.clients.consumer.ConsumerConfig
import org.apache.kafka.common.serialization.StringDeserializer
import java.time.Duration
import groovy.util.logging.Slf4j
class KConsumer {
def consumer
KConsumer(String NexusServers, String topic, String groupId) {
def props = new Properties()
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers)
props.put(ConsumerConfig.GROUP_ID_CONFIG, groupId)
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName())
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName())
consumer = new KafkaConsumer<String, String>(props)
consumer.subscribe([topic])
}
List<String> consumeMessages() {
def messages = []
while (true) {
def records = consumer.poll(100)
if (!records.isEmpty()) {
records.each { record ->
messages.add(record.value())
}
return messages
}
}
}
void close() {
consumer.close()
}
}
Using the below class like (Code snippet.)
def servers = "localhost:9092"
def topic = "my-topic"
def groupId = "my-group"
def consumer = new KConsumer(servers, topic, groupId)
messages = consumer.consumeMessages()
consumer.close()
Everything works fine, but, I'm interested to pass an optional parameter called key
, if passed, it should do return the messages of only that specific key. I have the logic for that something like this .
def messages = []
def records = consumer.poll(100)
for (ConsumerRecord<String, String> record : records) {
if (record.key() == key) {
messages.add(record.value())
}
}
Question here is, do I have to make the Class necessarily to take 4 parameters by default or is there an option like *args
in Groovy, akin to Python. I wanted to handle key
parameter as an optional one.
Also, do I have to come up with a new function to cater that or is it possible to repurpose the consumeMessages
function.