0

Is there a similar module to Storm's TopologyBuilder in Flink to build StreamExecutionEnvironment?

Basically, I want to configure my source and sinks in a JSON file and I would like to build the StreamExecutionEnvironment seamlessly from the JSON data without having to add code every time when I want to modify my DAG.

My JSON representation would be something like this

{
 [
  {
   "id" : "source",
   "subscriptions": ["intermediateSink", "destinationSink"]
  },
  {
   "id" : "intermediateSink",
   "subscriptions" : ["destinationSink"]
  },
  {
   "id" : "destinationSink",
   "subscriptions" : []
  }
 ]
}

I would like to know if Flink exposes a way to build the ExecutionEnvironment from a file like this.

I already have a Storm topology built using Storm's TopologyBuilder class. Something similar to this but the Spouts and Bolts are created and linked based on my JSON instead of creating and linking them one by one.

I would like to do something similar for my Flink setup.

Olaf Kock
  • 46,930
  • 8
  • 59
  • 90
Vicky
  • 26
  • 3
  • 1
    I don't know of any support in Flink for defining a workflow from JSON – kkrugler Jul 25 '22 at 22:19
  • Thanks. Yes, I didn't see anything provided by Flink, but I was able to write some custom code to build the ExecutionEnvironment from JSON, for our use case. – Vicky Aug 10 '22 at 21:12

0 Answers0