0

Writing data from Avro file to topics by running Confluent 5.1.0.

when I ran Kafka consumer command, numeric values are not displaying.

kafka_2.12-2.1.0 root$ bin/kafka-console-consumer.sh
--bootstrap-server 127.0.0.1:9092 --topic Initiate_scans

I can see the expected output in java logs.

Expected:

{"app_id": "SQE", "app_name": "iSeal", "dev_stage": "DEV", "scan_name": "Veracode", "seq_num": 1, "result_flg": "N", "request_id": 5534, "scan_number": 1}

{"app_id": "SQE", "app_name": "iSeal", "dev_stage": "DEV", "scan_name": "Checkmarx", "seq_num": 3, "result_flg": "Y", "request_id": 5534, "scan_number": 2}

Actual:

SQE iSealDEVVeracodeN?V

SQE iSealDEVCheckmarxY?V

Eshwar P
  • 379
  • 4
  • 13
  • maybe you need the avro console consumer, as the message is not serialized as string (default for console consumer) – aran Apr 03 '19 at 17:26
  • got it..! thank you so much. bin/kafka-avro-console-consumer.sh --bootstrap-server 127.0.0.1:9092 --topic Initiate_scans above command gave me expected result. – Eshwar P Apr 03 '19 at 17:58
  • glad to help mate : ) – aran Apr 03 '19 at 18:05

1 Answers1

0

kafka-console-consumer will show UTF-8 decoded binary data.

If you have Avro in the topic that was producer by Confluent's KafkaAvroSerializer (or similar library), then you'll need to use kafka-avro-console-consumer, which will convert Avro into JSON records.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245