1

I can see how to configure a SparkConf when creating a streaming application (see here)

I assume that I can configure the SparkConf through the SnappyStreamingContext for a streaming job similar to a streaming application. Let's say I get a handle to the SparkConf in a streaming job and modify some settings. Do these settings only apply to this streaming job or is this a global configuration update for all jobs?

thanks!

mike w
  • 131
  • 6

1 Answers1

1

Yes, you can configure the SparkConf through the SnappyStreamingContext for a streaming job, and it is same as spark streaming configuration. Since SparkConf is a global configuration it is applicable to all the jobs in a streaming application. I think Spark doesn't allow you to change SparkConf after starting your application.

Yogesh Mahajan
  • 241
  • 1
  • 4