0

we are using the following Kafka client on our Linux red-hat servers

0.8.1 Release
Released March 12, 2014
Release Notes
Source download: kafka-0.8.1-src.tgz (asc, md5)
Binary downloads:
Scala 2.8.0 - kafka_2.8.0-0.8.1.tgz (asc, md5)
Scala 2.8.2 - kafka_2.8.2-0.8.1.tgz (asc, md5)
Scala 2.9.1 - kafka_2.9.1-0.8.1.tgz (asc, md5)
Scala 2.9.2 - kafka_2.9.2-0.8.1.tgz (asc, md5)
Scala 2.10 - kafka_2.10-0.8.1.tgz (asc, md5)
We build for multiple versions of Scala. This only matters if you are using Scala and you want a version built for the same Scala version you use. Otherwise any version should work (2.9.2 is recommended).

from Kafka broker side , we upgrade the Kafka brokers versions to 1.X version

now we need to know if Kafka Client compatibility ( 0.8.1 ) is fit the Kafka broker - 1.0

after searching we found this post Determine the Kafka-Client compatibility with kafka-broker

but we not found the Kafka client 0.8.1 compatibility with Kafka broker versions

how to know what is the Kafka brokers version that fit Kafka client version - 0.8.1 ?

jessica
  • 2,426
  • 24
  • 66

1 Answers1

3

Kafka clients running 0.8.1 are not compatible with 1.0 brokers, or even 0.9 brokers due to KIP-35 which includes protocol versions within the client API requests to handle better cross-version support

You can refer to the wiki here - https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix

I would suggest going to a 2.x release as well

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • do you means that we need to install the client 2.X ? – jessica Jan 07 '20 at 09:58
  • we are using HDP 2.6.5 – jessica Jan 07 '20 at 10:00
  • from the link - https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix , how you identify the right kafka version fro client 0.8.1? – jessica Jan 07 '20 at 10:02
  • I suggest you gather a list of all breaking changes, starting with 0.9, then going upwards until you're comfortable. I'd also suggest upgrading to 0.9, then 0.10, then 0.11, as the APIs changed enough in those versions. From there, it can be safer to go straight to 2.x, in my opinion... https://kafka.apache.org/090/documentation.html#upgrade – OneCricketeer Jan 07 '20 at 10:02
  • Click the first link. KIP-35 was not added until 0.10.2 – OneCricketeer Jan 07 '20 at 10:03
  • Alternatively, assuming that you need no old data, just get some more hardware and install a brand new cluster. – OneCricketeer Jan 07 '20 at 10:06
  • yes I click on the link - https://cwiki.apache.org/confluence/display/KAFKA/KIP-35+-+Retrieving+protocol+version , but not found the info about client 0.8.1 to kafka 1.X – jessica Jan 07 '20 at 10:16
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/205505/discussion-between-jessica-and-cricket-007). – jessica Jan 07 '20 at 10:17
  • Does it need to say it? The compatibility link says "any KIP-35 enabled client". If you refer to that KIP, it was not available until much after 0.8, therefore it is not compatible – OneCricketeer Jan 07 '20 at 10:20
  • how we get list of all kafka client versions that fit the kafka 1.X from HDP 2.6.5 ) – jessica Jan 07 '20 at 10:44
  • Just use the ones from maven central. Kafka is still kafka. 0.10.2 and higher will work, as shown in the wiki – OneCricketeer Jan 07 '20 at 10:54
  • what you means "Kafka is still kafka. 0.10.2" , we have kafka version 1.X – jessica Jan 07 '20 at 11:06
  • There is a period there. Kafka is still kafka... HDP provided or not should make no difference. Spark, Nifi, external clients will continue to work should you chose to maintain Kafka with its own tools outside of HDP – OneCricketeer Jan 07 '20 at 14:52
  • 1
    another question - we have also old Kafka version - 0.10 version ( from HDP 2.6.4 ) , what are the range of Kafka clients versions that supported the Kafka broker - 0.10? – jessica Jan 07 '20 at 16:15