0

I have been trying to read this data (Topics) to my MSK Cluster from my React application, I have found very few tools and information that can help me, and I have not succeeded

How can I get this data from MSK cluster and display them in a React table?

Some of the items I found: https://www.npmjs.com/package/@aws-cdk/aws-msk, https://docs.aws.amazon.com/es_es/msk/latest/developerguide/client-access.html, https://akhq.io/, https://github.com/Microsoft/kafka-proxy-ws

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Alison Quintero
  • 13
  • 3

2 Answers2

0

First, @aws-cdk/aws-msk is a library to provision a cluster, not read from it.

React cannot consume from Kafka since it is client-side.

The AKHQ project (and similar Kafka GUI projects, like Confluent Control Center) generally use Java to render frontend HTML (AKHQ uses SSE with JSX, if you look at TailController.java) after using the native Java KafkaConsumer to poll a batch of records.

Basically, if your server and frontend are separate components, you need some type of proxy, such as the websocket from Microsoft (which has been archived so may no longer work with newer Kafka protocol versions), the Confluent REST Proxy, and/or your own backend server that is acting as a Kafka consumer that can expose messages to the frontend via some interface such as REST, GraphQL, WebSockets, or SSE.

The simplest solution, in my opinion, would be to use the REST Proxy

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
0

If you're trying to read data from a topic using Javascript, you should take a look at the kafkajs library. You can use this library to create a consumer to read from your topic and then (in theory) you can output your data to your React application.

You can make and run a basic consumer with the following code:

const { Kafka } = require('kafkajs');

const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['broker1:9092', 'broker2:9092'],
});

const consumer = kafka.consumer({ groupId: 'consumer-group-id' });

await consumer.connect();

await consumer.subscribe({ topics: ['my-topic'] });

await consumer.run({
  eachMessage: async ({ topic, partition, message, heartbeat, pause }) => {
    // process your data here
  } 
});

Where you initialize this consumer and how you run it is up to you of course.

The docs for consuming messages are pretty verbose as well, make sure to look over that for more info when you configure your consumer.

This link could also be helpful in structuring your project to use the consumer: https://www.cdata.com/kb/tech/kafka-connect-react.rst

RusskiT
  • 106
  • 7