I have two windowed dstreams that I would like to zip like the normal zipping in RDDs.
Note : The main goal is to calculate the mean and stdv of the window dstream in case there is a better way to calculate this.
I have two windowed dstreams that I would like to zip like the normal zipping in RDDs.
Note : The main goal is to calculate the mean and stdv of the window dstream in case there is a better way to calculate this.
I used DStream's transformWith to use this rdd operation and this solved my problem.
Example :
valueStream.transformWith(mean, (rdd1: RDD[Int], rdd2 : RDD[Double]) => {rdd1.zip(rdd2)}