-1

I am trying to learn/understand scala+lagom+kafka in detail. While searching through the web found a sample Twitter project that explains it pretty nicely (https://github.com/knoldus/lagom-scala.g8/tree/master/src/main/g8). In this project two Lagom microservices (consumer & producer) are created and they communicate over a kafka topic. However both the service run on the same server/machine.

I was wondering if it is possible to have communication between two lagom+kafka microservice running on two different servers/machines using kafka topic. I was able to achieve this configuration by simple kafka (without Lagom), by modifying kafka configuration files (zookeeper.properties and server.properties).

Questions:

  1. Is this configuration (producer and consumer on two different servers) possible using Lagom+Kafka.
  2. How do I add kafka configuration (zookeeper.properties and server.properties) inside lagom framework.
  3. Is there any project that utilizes above configuration. So that, I can use this as a reference to understand Lagom+Kafka in easily.
erip
  • 16,374
  • 11
  • 66
  • 121
Aseem
  • 1
  • 5

1 Answers1

0
  1. Is this configuration (producer and consumer on two different servers) possible using Lagom+Kafka.

Of course, Yes.

2.How do I add kafka configuration (zookeeper.properties and server.properties) inside lagom framework.

There's documentation about that:

Is there any project that utilizes above configuration. So that, I can use this as a reference to understand Lagom+Kafka in easily.

There're some example projects in the Lagom Github:

Philosophically, it's because Lagom services are hosted on different servers (so are separated by networks) that we need Kafka, no matters where Kafka is hosted.
I've already explained this need here: https://discuss.lightbend.com/t/need-some-insight-on-lagoms-architecture/1369

Jules

Jules Ivanic
  • 1,579
  • 2
  • 15
  • 28
  • Thanks for your response. However, I had a few followup question: – Aseem Aug 02 '18 at 09:34
  • 1. Where do we add "lagom.broker.kafka" configuration? Does it go in resources/application.conf? 2. I do not see parameters like "zoo servers", "host.name" & "zookeeper.connect" in https://www.lagomframework.com/documentation/1.4.x/ documentation. How do I pass these parameters to Lagom? 3. Do I have to disable in-built kafka and use "external Kafka server" for communication between producer and consumer running on different servers? 4. I skimmed through the "online-auction-scala" project and did not find zookeeper/kafka configuration specifically. Could you please provide some pointers? – Aseem Aug 02 '18 at 09:53
  • I created a kafka-server.properties configuration file (with required zookeeper & server properties). Added following lines to build.sbt to be able to compile these server properties in my lagom project: `lagomKafkaPropertiesFile in ThisBuild := Some((baseDirectory in ThisBuild).value / "project" / "kafka-server.properties")` The broker.id was set to a same value (say "1") on both the lagom service to be able to setup communication between them. Now the next issue that I am working on is to be able to compile the consumer code without the reference of kafka topic created by the Producer. – Aseem Aug 20 '18 at 15:57