I want to use a self-hosted (in EC2 instance) Kafka stream to use as an event source to trigger the lambda function. Although lamda can easily be configured(here) to get triggered from a Kafka stream, I don't want to trigger for all the messages but for only a few conditions.
We can use pattern matching in Eventbridge but can we use self-hosted Kafka as a source there? If yes, then how?
You need to implement your custom filtering somewhere. I think your best bet would be to use Kafka Streams to find the relevant messages which you want to act upon and then send them to a separate topic. Here's an article explaining how this can be implemented.
Other alternative solutions I could suggest would be to use AWS Kinesis or a custom kafka consumer, nevertheless they all follow the same concept:
