0

I am playing around with Spring Boot v2 at the moment. So far, my set up looks like this: Spring -> Telegraf -> Kafka -> Telegraf -> influx

I am wondering whether or not it's possible to take out the the first telegraf inbetween Spring and Kafka, so something like this: Spring -> Kafka -> Telegraf -> Influx

I've looked at the configurations of micrometer and there is no config for Kafka. Also, telegraf was pulling data from Spring.. and as Kafka is a push model (i.e. you are pushing data into Kafka), would Spring be able to push data to Kafka? If yes, how? Through the use of HTTP POST methods?

New to the whole concept.

kjvf
  • 81
  • 7

1 Answers1

1

would Spring be able to push data to Kafka? If yes, how? Through the use of HTTP POST methods?

Kafka uses its own TCP protocol, not HTTP so no. At least not without using the Kafka REST Proxy.

You would basically be embedding the same thing that Telegraf does into your Spring code.

It's possible, sure, but built into Micrometer? Not that I'm aware of.

Plus, it would be overhead on your app having an internal producer thread, and you'd be required to include kafka clients with each of your monitored apps, plus have some control preventing your app from failing if Kafka connection isn't possible...

I would suggest keeping Telegraf installed on each host machine, or at the very least, Prometheus JMX exporter or Jolokia for your individual Java apps. From this, JMX metrics can be collected and pushed to downstream monitoring systems

Or, as commented, you could skip Kafka, but I'm guessing you want to keep it there as a buffer.

On the other side, you can use Kafka Connect Influxdb sink to get optimal performance of consumer scaling

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245