-1

producer.send() accepting 2 parameters one is kafka topic and 2nd is generated output.How can we make kafkaproducer.py using below scripts.***

Kindly help me to merge both python into single file so that we can use for this script to push the data into kafka topic.

import time
from kafka import KafkaProducer
kafka_bootstrap_servers = 'localhost:9092'
kafka_topic_name = 'test1topic'
producer = KafkaProducer(bootstrap_servers=kafka_bootstrap_servers,value_serializer=lambda v: json.dumps(v).encode('utf-8'))
producer.send(kafka_topic_name, script_output)
Anjali Rao
  • 3
  • 5

2 Answers2

0

Basicly, you can't add code to the KafkaProducer, but you can wrap it as new producer, and makes it get a code, or just add to the Generator object a kafka producer and make the start function send the data thru kafka

Reznik
  • 2,663
  • 1
  • 11
  • 31
  • can you send me the full code, because i have not all things what could do.. but still not working.. – Anjali Rao Dec 12 '19 at 10:40
  • 1
    @AnjaliRao Im not going to write this code, just add the producer to your generator object, and makes the start function send the output – Reznik Dec 12 '19 at 10:43
0

You just pass it from the output of your generator function

For example, don't use it one file, use proper Python best practices of importing your own modules 

from somescript import generate_data

for data in generate_data():
    producer.send(kafka_topic_name, data)

Rather than having the generator just print data, it should yield the data, thus returning a Python generator object

Otherwise, you would replace the print statement in the Generator class with producer.send, like you've already shown

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245