5

I am using django for a project and want to trigger a Kafka event on some request. I know how to produce a kafka event in a view.

class TestList(generics.ListCreateAPIView):
    permission_classes = [permissions.IsAuthenticated]

    queryset = Test.objects.all()
    serializer_class = TestSerializer

    def post(self, request, *args, **kwargs):
        serializer = TestSerializer(data=request.data)

        if serializer.is_valid():
            serializer.save()

            producer = KafkaProducer(bootstrap_servers=[f'{settings.SECRETS.kafka.host}:{settings.SECRETS.kafka.port}'],
                                     value_serializer=lambda m: json.dumps(m).encode('ascii'))
            kafka_msg = {
                'msg': {
                    'hello': 'world',
                },
            }
            producer.send('test', key=serializer.instance.id.bytes, value=kafka_msg) \
                .add_callback(on_send_success).add_errback(on_send_error)

            return Response(serializer.data, status=status.HTTP_201_CREATED)

        return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

However, I am not sure if this is optimal to create the KafkaProducer every time in the request. Is there a way to create the producer on startup with a persistent connection and how or would it make no difference?

Greeneco
  • 691
  • 2
  • 8
  • 23
  • 1
    Have a look at this repo : https://github.com/koslibpro/django-kafka-demo/blob/master/requests_logger/middleware.py – Maverick Aug 30 '20 at 09:10
  • @Maverick thanks. However for testing purpose I need to mock my kafka producers, but I am not sure on how to do this if the Producer is created like in the repo. – Greeneco Jun 22 '21 at 13:57

0 Answers0