If I want to dynamically change the number of partition during runtime, which parameter and which file should I make some modification for the Spark source code. I don't want to use the coalesce or repartition operation for the experiment's need.
Asked
Active
Viewed 676 times
0
-
are you looking to have the same partition for all the DF? else I don't think there is an option apart form `coalesce` or `repartition` – Gaurang Shah Mar 29 '18 at 16:22
-
Yep, so i am trying to modify the source code of Spark – user3322839 Mar 29 '18 at 17:08