15

For a multi-player programming game, I'm working on a background compilation server for Scala that supports compilation of multiple, independent source trees submitted by the players. I succeeded in running fast, sequential compilations without reloading the compiler by instantiating the Global compiler object via

val compilerGlobal = new Global(settings, reporter)

and then running individual compile jobs via

val run = new compilerGlobal.Run
run.compile(sourceFilePathList)

I would now ideally like to parallelize the server (i.e. make multiple compilation runs concurrently), but still without reloading the compiler (primarily to avoid re-parsing the lib) from scratch each time. Is this possible, i.e. is the second part shown above (safely :-) re-entrant, or does it hold global state? If not, is there something else I can try? I am currently focused on supporting Scala 2.9.1.

Michael Petrotta
  • 59,888
  • 27
  • 145
  • 179
Scalatron Botwar
  • 255
  • 2
  • 6
  • It holds the 'global state' of the symbol table, for a start. – user207421 Apr 01 '12 at 08:18
  • 2
    Use a pool of compiler instances. – retronym Apr 01 '12 at 10:26
  • @retronym Thanks for the suggestion. My main question, though, is "how much can I recycle?". Your comment suggests that there is no global shared state between (in-memory) compiler instances (class Global), which is a good start. But are you also suggesting that the individual runs DO share state (beyond the parsed standard lib)? I'd need more info. – Scalatron Botwar Apr 01 '12 at 11:38
  • 1
    Yes, compiler runs share state, so you should not share them between threads. It's one of the issues that comes up in the Eclipse plugin. – Iulian Dragos Apr 02 '12 at 09:36
  • @IulianDragos Hi Iulian, yours seems to be the "final" answer, so if you'd like to post that as an answer I'll accept it to close the question. – Scalatron Botwar Apr 07 '12 at 12:12

1 Answers1

4

Yes, compiler Runs share state, so you should not share them between threads. It's one of the issues that comes up in the Eclipse plugin. As @EJP noted, the symbol table is shared.

This is not so important in your case, but comes up in an IDE: the compiler uses laziness in types, meaning additional computation (and mutation) may happen when calling methods on Symbol. Because of visibility issues, it's important that these methods are called on the same thread as the one that created them.

Iulian Dragos
  • 5,692
  • 23
  • 31