How can i Run Web socket In Apache Flink Serverless Java

Question

1. I have a Java Program to run in Apache flink in AWS i want to run real time communication through web socket how can i integrate serverless web socket in Apache flink Java ??? Thanks You

Answer 1

Flink is designed to help you process and move data continuously between storage or streaming solutions. It is not intended to, and would not work well with websockets directly for these reasons:

• When submitting a job, the runtime serializes your logic and moves it to other TaskManager instances so that it can parallelize them. These can be on another machine entirely. Now, if you were intending to service a websocket with that code, it has just moved elsewhere!

• TaskManagers can be stopped and restarted (scaling event, recovering from a checkpoint/savepoint, etc). That's where your websocket connection will be cut.

• Also, the Flink planner can decide that your source functions need be read twice if it helps the processing. This means that your websockets would need to maintain a history of messages received, and make sure they are sent once to each operator instance.

---

This being said you can have a webserver managing the websocket, piping messages back and forth to a Kafka topic, which then Flink can operate on.

Since you're talking about AWS, I suggest you learn about their Websocket API Gateway service. I believe these can be connected easily with Kinesis, which Flink can read from and write to easily.