I have a python project that employees Kafka Queues, the project runs as expected but fails to run when run as part of a docker container.It keeps throwing this error can't find the module kafka
This is how my Dockerfile..
FROM ubuntu:18.04
RUN apt-get update
RUN apt -y upgrade
RUN apt-get install -y python
RUN apt install -y python3-pip
RUN apt-get install -y libsnappy-dev
RUN mkdir -p /app
WORKDIR /app
COPY . /app
RUN python3 -m pip install pip --upgrade
RUN python3 -m pip install pkgconfig
RUN python3 -m pip install kafka-python
RUN python3 -m pip install -r requirements.txt
CMD python __init__.py
This is my python script that is throwing the error...
import multiprocessing
from kafka import KafkaConsumer
class Consumer(multiprocessing.Process):
def __init__(self,host,topic,groupid):
multiprocessing.Process.__init__(self)
self.stop_event = multiprocessing.Event()
self.topic = topic
self.consumer = KafkaConsumer(
bootstrap_servers=host,
auto_offset_reset='earliest',
enable_auto_commit=True,
auto_commit_interval_ms=1000,
consumer_timeout_ms=1000,
group_id=groupid
)
def stop(self):
self.stop_event.set()
def run(self,topic,cb):
self.consumer.subscribe([topic])
while not self.stop_event.is_set():
for message in self.consumer:
cb(message.value)
if self.stop_event.is_set():
break
I am positive the module is being installed by this line
RUN python3 -m pip install Kafka-python
But the script can't find it.
