3

Is there a way to increase the memory during assembly (the plugin)? I notice that adding more memory to sbt through SBT_OPTS does't really help. I suspect that assembly is forking a separate JVM that doesn't inherit the JVM configurations from sbt.

Here is the error:

java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space

Here is what I tried without much luck:

javaOptions in assembly += "-Xmx2g"

Eugene Yokota
  • 94,654
  • 45
  • 215
  • 319
marios
  • 8,874
  • 3
  • 38
  • 62
  • I don't believe SBT forks the JVM for separate commands by default unless you explicitly enable it via the `fork := true` (I just tried it out on one of my projects). Can you double check that you have the `SBT_OPTS` correctly exported? – Akos Krivachy Jun 12 '15 at 00:53
  • Here is my SBT_OPTS `SBT_OPTS='-Xmx4g -Xms512M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m'` – marios Jun 12 '15 at 05:17

4 Answers4

4

I suspect that assembly is forking a separate JVM that doesn't inherit the JVM configurations from sbt.

I'm the author of sbt-assembly. assembly does not fork a separate JVM. You can see the code here and check - https://github.com/sbt/sbt-assembly/blob/0.13.0/src/main/scala/sbtassembly/Assembly.scala#L239-L246

Eugene Yokota
  • 94,654
  • 45
  • 215
  • 319
  • 1
    Thanks for the reply! So if I get memory issues during sbt-assembly what can I do? I tried to increase the memory of SBT but it didn't seem to help much. The only thing that helped is `assemblyOption in assembly := (assemblyOption in assembly).value.copy(cacheUnzip = false)`. Is that what you would also suggest? – marios Jun 20 '15 at 07:12
  • Could you open a Github issue on this issue please? If that's what helped, basically we have an unbound cache acting like a memory leak. – Eugene Yokota Jun 20 '15 at 23:30
  • Eugene, does sbt-assembly currently use `.sbtopts` file? – Tvaroh Sep 23 '15 at 11:38
  • 1
    I've just seen this too and disabling caching as in @marios' comment did the trick for me. –  Oct 26 '15 at 04:04
  • @marios where do you put that? I have the issue when trying to compile spark via the guide on the spark site, on OS X 10.11 – New Alexandria Feb 10 '16 at 17:42
  • Did you increase the `-XX:MaxPermSize`? You might want to do that first for building spark. If that doesn't work, you can add the Setting as any other setting in your SBT project. If you only have one project then just pass that in a new line is your SBT project. If it's a multi-project use the `.settings()` setter. – marios Feb 10 '16 at 19:34
0

Make sure to export SBT_OPTS. Mine is "-Xmx2G -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=2G -Xss2M"

Mariusz
  • 2,617
  • 1
  • 23
  • 26
0

I met the same issue before, Please do it as below for windows. The issue could be gone.

set SBT_OPTS="-Xmx2G"
sbt assembly
Fatemeh Abdollahei
  • 3,040
  • 1
  • 20
  • 25
Ken Zhu
  • 41
  • 1
0

You can increase heap size by using below mentioned command in linux environment:-

sbt -J-Xms2048m -J-Xmx2048m assembly
RobC
  • 22,977
  • 20
  • 73
  • 80
Amarjeet Singh
  • 23
  • 10