I am using the IDEA to develop the Spark Demo,How to set the params about spark memory size in the local mode?

Question

I am running a geoSpark Demo in the local mode,not standalone.The data's size is about 5GB. And I am getting the OOM Error.The I want to change the spark memory in the local mode ,how to do it?

I use IDEA to develop spark demo – 谭凯中 Jun 10 '20 at 08:38 — 谭凯中, Jun 10 '20 at 08:38
can you add the code you are trying? – Narsireddy Aug 27 '20 at 13:33 — Narsireddy, Aug 27 '20 at 13:33

Григорий · Answer 1 · 2020-06-11T06:59:00.770

0

conf=SparkConf()
conf.set("spark.executor.memory", "1g")

edited Jun 11 '20 at 06:59

answered Jun 10 '20 at 11:41

Григорий

41
9

oh,yes.But I want to use config file. where to put it in the maven project ? – 谭凯中 Jun 10 '20 at 15:36
Locally there is no such possibility. – Григорий Jun 11 '20 at 06:56

I am using the IDEA to develop the Spark Demo,How to set the params about spark memory size in the local mode?

1 Answers1