I'm trying to get a kafka consumer to subscribe to 2 topics, yet whenever I try and assign more than one Topic I get this message:
INFO:__main__:Controller module is running and listening...
WARNING:kafka.coordinator.consumer:group_id is None: disabling auto-commit.
INFO:kafka.consumer.subscription_state:Updating subscribed topics to: ('folder-data', 'password')
INFO:kafka.conn:<BrokerConnection node_id=bootstrap-0 host=172.17.0.1:9092 <connecting> [IPv4 ('172.17.0.1', 9092)]>: connecting to 172.17.0.1:9092 [('172.17.0.1', 9092) IPv4]
INFO:kafka.conn:Probing node bootstrap-0 broker version
INFO:kafka.conn:<BrokerConnection node_id=bootstrap-0 host=172.17.0.1:9092 <connecting> [IPv4 ('172.17.0.1', 9092)]>: Connection complete.
INFO:kafka.conn:Broker version identified as 2.5.0
INFO:kafka.conn:Set configuration api_version=(2, 5, 0) to skip auto check_version requests on startup
INFO:__main__:Sending message: <Logger __main__ (INFO)> to topics: find-password, analyze-folder
INFO:kafka.conn:<BrokerConnection node_id=1001 host=172.17.0.1:9092 <connecting> [IPv4 ('172.17.0.1', 9092)]>: connecting to 172.17.0.1:9092 [('172.17.0.1', 9092) IPv4]
INFO:kafka.conn:<BrokerConnection node_id=1001 host=172.17.0.1:9092 <connecting> [IPv4 ('172.17.0.1', 9092)]>: Connection complete.
INFO:kafka.conn:<BrokerConnection node_id=bootstrap-0 host=172.17.0.1:9092 <connected> [IPv4 ('172.17.0.1', 9092)]>: Closing connection.
INFO:__main__:Sent.
INFO:kafka.conn:<BrokerConnection node_id=bootstrap-0 host=172.17.0.1:9092 <connecting> [IPv4 ('172.17.0.1', 9092)]>: connecting to 172.17.0.1:9092 [('172.17.0.1', 9092) IPv4]
INFO:kafka.conn:<BrokerConnection node_id=bootstrap-0 host=172.17.0.1:9092 <connecting> [IPv4 ('172.17.0.1', 9092)]>: Connection complete.
WARNING:kafka.cluster:Topic password is not available during auto-create initialization
WARNING:kafka.cluster:Topic folder-data is not available during auto-create initialization
INFO:kafka.consumer.subscription_state:Updated partition assignment: []
For some reason, if I restart the container afterwards it works.
Here's my code:
analye_folder = 'analyze-folder'
folder_data = 'folder-data'
find_password = 'find-password'
get_password = 'password'
consumer = KafkaConsumer (auto_offset_reset='earliest',
bootstrap_servers=bootstrap_servers,
api_version=(0,10))
consumer.subscribe([get_password, folder_data])
How do I get around this?
