python 3.8 and above - No module named 'confluent_kafka.cimpl'

Question

After migrating python version from 3.7 to 3.8/3.9, seeing an issue with Kafka confluent compatibility in one of our Lambda functions. I am using Kafka confluent 1.6 library.

Error while running lambda:

{
  "errorMessage": "Unable to import module 'pzn-kafka-emitter': No module named 
  'confluent_kafka.cimpl'",
  "errorType": "Runtime.ImportModuleError",
  "stackTrace": []
}