0

I am building a Flink Application and currently planning on deploying using Amazon Kinesis Analytics. I need to connect to a Kafka topic using SSL truststore/keystore certificates and am looking at using Flink's out of the box Kafka connector. It looks like the connector and internal Kafka security options allow me to specify the local path to my jks file, or pass the credentials directly as an encoded PEM. Because AWS KDA doesn't give the option to execute bootstrap steps on new task nodes on the cluster AND the Kafka connector doesn't give me control over the startup process for each reader to install certificates programmatically there, I think I will need to use the PEM option when creating the Kafka source builder.

My question is: what will happen when the certificates expire after set amount of time? Can I surround the builder declaration in a try catch and refresh the PEM there? Or will the failure cause the pipeline to crash before that? Are there any options I'm missing besides to build my own custom source or use a different type of infrastructure?

I will try to test the certificates expiring myself but thought I might ask here in case anyone has any insight or suggestions.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245

0 Answers0