1

I'm using a KafkaConsumer java client to connect with Kafka brokers. My user can define a proxy (ip : port) between the KafkaConsumer and the brokers.

----------------------         -------------         -----------------------
| Application client |  <--->  | TCP Proxy |  <--->  | Target (e.g. kafka) |
----------------------         -------------         -----------------------

Without the proxy, configuration 'boostrap.servers' with the broker's list is enough to establish an initial connection.

My question is, how can I manage the KafkaConsumer to initial the connection with the brokers via the proxy? does apache-Kafka have that kind of support?

I tried to find a property such as 'listeners' and 'advertised.listeners', however, they are a broker configuration and I can't configure the brokers.

NatalieBe
  • 11
  • 1
  • This https://stackoverflow.com/questions/47570246/connect-to-kafka-through-socks-proxy seems to imply that creating a socks proxy would be a challenge, so you're left with @cricket_007 's suggestion -- Kafka REST proxy. – mazaneicha Mar 24 '20 at 18:49

1 Answers1

1

The TCP protocol would need to establish a socket handshake, then request events from upstream Kafka then forward back as a streaming response to the requesting client....

If building a TCP protocol is too much for you, then you could use the KAFKA REST Proxy instead.

Alternatively, clarify why you cannot directly connect to any broker

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245