How to use scala.js compiler as a compiler plugin, thus allowing it to integrate into Gradle or maven?

Question

I have a Scala project that can only be compiled with Gradle, recently someone has asked it to be ported to Scala.js, ideally by cross-build.

When I read the code of Scala.js, I realised that the bulk of the plugin is actually a compiler (scalac) plugin:

import scala.tools.nsc.plugins.{
  Plugin => NscPlugin, PluginComponent => NscPluginComponent
}
...
class ScalaJSPlugin(val global: Global) extends NscPlugin {
  import global._
...

But all the documentations have suggested to use it as part of the sbt-plugin, which are useless in my case. So I try to attach it to my scalac process, invoked through gradle:

    dependencies {

        implementation("org.scala-js:scalajs-library_${vs.scalaBinaryV}:${vs.scalaJSV}")
        scalaCompilerPlugins("org.scala-js:scalajs-compiler_${vs.scalaV}:${vs.scalaJSV}")
...

This experiment unfortunately doesn't work. The compiler threw the following error:

> Task :compileTestFixturesScala FAILED
## Exception when compiling 1 sources to /home/peng/git-scaffold/scaffold-gradle-kts/build/classes/scala/testFixtures
scala.reflect.internal.MissingRequirementError: object scala.scalajs.js.Dynamic in compiler mirror not found.
scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:24)
scala.reflect.internal.Mirrors$RootsBase.$anonfun$getModuleOrClass$6(Mirrors.scala:66)
scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:66)
scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:56)
scala.reflect.internal.Mirrors$RootsBase.getRequiredClass(Mirrors.scala:56)
org.scalajs.nscplugin.JSDefinitions$JSDefinitionsClass.JSDynamicClass$lzycompute(JSDefinitions.scala:52)
org.scalajs.nscplugin.JSDefinitions$JSDefinitionsClass.JSDynamicClass(JSDefinitions.scala:52)
org.scalajs.nscplugin.JSDefinitions$JSDefinitionsClass.JSDynamicModule$lzycompute(JSDefinitions.scala:83)
org.scalajs.nscplugin.JSDefinitions$JSDefinitionsClass.JSDynamicModule(JSDefinitions.scala:83)
org.scalajs.nscplugin.JSDefinitions$JSDefinitionsClass.JSDynamic_newInstance$lzycompute(JSDefinitions.scala:84)
org.scalajs.nscplugin.JSDefinitions$JSDefinitionsClass.JSDynamic_newInstance(JSDefinitions.scala:84)
org.scalajs.nscplugin.JSPrimitives.initWithPrimitives(JSPrimitives.scala:89)
org.scalajs.nscplugin.JSPrimitives.initPrepJSPrimitives(JSPrimitives.scala:77)
org.scalajs.nscplugin.PrepJSInterop$JSInteropPhase.run(PrepJSInterop.scala:65)
scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1524)
scala.tools.nsc.Global$Run.compileUnits(Global.scala:1508)
scala.tools.nsc.Global$Run.compileSources(Global.scala:1500)
scala.tools.nsc.Global$Run.compile(Global.scala:1634)
xsbt.CachedCompiler0.run(CompilerInterface.scala:153)
xsbt.CachedCompiler0.run(CompilerInterface.scala:125)
xsbt.CompilerInterface.run(CompilerInterface.scala:39)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
sbt.internal.inc.AnalyzingCompiler.call(AnalyzingCompiler.scala:248)
sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:122)
sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:95)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:91)
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:186)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$3(MixedAnalyzingCompiler.scala:82)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$3$adapted(MixedAnalyzingCompiler.scala:77)
sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:215)
sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:77)
sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:146)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:343)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:343)
sbt.internal.inc.Incremental$.doCompile(Incremental.scala:120)
sbt.internal.inc.Incremental$.$anonfun$compile$4(Incremental.scala:100)
sbt.internal.inc.IncrementalCommon.recompileClasses(IncrementalCommon.scala:180)
sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:98)
sbt.internal.inc.Incremental$.$anonfun$compile$3(Incremental.scala:102)
sbt.internal.inc.Incremental$.manageClassfiles(Incremental.scala:155)
sbt.internal.inc.Incremental$.compile(Incremental.scala:92)
sbt.internal.inc.IncrementalCompile$.apply(Compile.scala:75)
sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:348)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:301)
sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:168)
sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:248)
sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:74)
org.gradle.api.internal.tasks.scala.ZincScalaCompiler.execute(ZincScalaCompiler.java:157)
org.gradle.api.internal.tasks.scala.ZincScalaCompilerFacade.execute(ZincScalaCompilerFacade.java:47)
org.gradle.api.internal.tasks.scala.ZincScalaCompilerFacade.execute(ZincScalaCompilerFacade.java:32)
org.gradle.api.internal.tasks.compile.daemon.AbstractDaemonCompiler$CompilerWorkAction.execute(AbstractDaemonCompiler.java:135)
org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:63)
org.gradle.workers.internal.AbstractClassLoaderWorker$1.create(AbstractClassLoaderWorker.java:49)
org.gradle.workers.internal.AbstractClassLoaderWorker$1.create(AbstractClassLoaderWorker.java:43)
org.gradle.internal.classloader.ClassLoaderUtils.executeInClassloader(ClassLoaderUtils.java:97)
org.gradle.workers.internal.AbstractClassLoaderWorker.executeInClassLoader(AbstractClassLoaderWorker.java:43)
org.gradle.workers.internal.IsolatedClassloaderWorker.run(IsolatedClassloaderWorker.java:49)
org.gradle.workers.internal.IsolatedClassloaderWorker.run(IsolatedClassloaderWorker.java:30)
org.gradle.workers.internal.WorkerDaemonServer.run(WorkerDaemonServer.java:87)
org.gradle.workers.internal.WorkerDaemonServer.run(WorkerDaemonServer.java:56)
org.gradle.process.internal.worker.request.WorkerAction$1.call(WorkerAction.java:138)
org.gradle.process.internal.worker.child.WorkerLogEventListener.withWorkerLoggingProtocol(WorkerLogEventListener.java:41)
org.gradle.process.internal.worker.request.WorkerAction.run(WorkerAction.java:135)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

Despite that, this error seems to be caused by trivial missing class, and it happens at a fairly late stage of compilation, when test code are being compiled.

So my question is: what does it take to make scalajs to work as a compiler plugin (WITHOUT sbt)? And what could possibly cause this error?

score 1 · Accepted Answer · answered Mar 06 '22 at 09:26

1

That error is typically symptomatic of not having scalajs-library on the compilation classpath. But I see you already added

implementation("org.scala-js:scalajs-library_${vs.scalaBinaryV}:${vs.scalaJSV}")

which, from what I understand of Gradle, should do that.

Perhaps double-check that that library indeed ends up on the compilation classpath, using whatever command Gradle offers to check that?

That said, after you're done with this, all you'll have are .sjsir files, which cannot be run or tested. You will need the Scala.js linker to process the .sjsir files on the classpath, and produce one .js file.

answered Mar 06 '22 at 09:26

sjrd

21,805
2
61
91

Ah, solved the first question: "implementation" scope of gradle indeed ignores testFixtures (it only includes compile & text, testFixture is introduced much later through a plugin) – tribbloid Mar 06 '22 at 17:08
thanks a lot, accepted. In the meantime, would you suggest me to work on invoking the linker in the scalac plugin? – tribbloid Mar 06 '22 at 17:09
The linker should be called *after* compilation ends; not called in the scalac plugin. You can probably find inspiration in this old (unmaintained) Gradle plugin for Scala.js: https://github.com/gtache/scalajs-gradle – sjrd Mar 06 '22 at 21:03
I see, ideally it should be made part of the scalac plugin to make it agnostic to dependency manager. the linking stage doesn't depends on other code except scala so it is theoretically feasible – tribbloid Mar 06 '22 at 23:57
1

No, no. There are extremely good reasons to separate them. This is an architecture that has worked very well for 9 years, across several build tools (sbt, mill, Maven, scala-cli). We are definitely not going to change it. – sjrd Mar 07 '22 at 06:45
Thanks a lot, I wasn't aware that scala-cli (the REPL environment) has a scala-js plugin, as the readiness of scala-reflect seems to be much higher for JVM. Do you have a link to its implementation? – tribbloid Mar 07 '22 at 22:55
My intention is still to avoid maintaining plugins for 4 dependency managers (perhaps more), which takes a lot of time and manpower – tribbloid Mar 07 '22 at 22:57
1

scala-cli is this: https://scala-cli.virtuslab.org/ . It's not quite the same as the basic REPL. – sjrd Mar 07 '22 at 23:04
thanks a lot. It is a fairly large project and will take some time to integrate. Right now I've resorted to invoke the linker in test runtime (using method in https://stackoverflow.com/questions/65092014/build-compile-latest-salajs-1-3-using-gradle-on-a-windows-machine). Which is strictly after scalac compilation & have good support from all dependency managers. Will post the result once I'm finished – tribbloid Mar 09 '22 at 21:50
@tribbloid Could you post an answer here if you'll find a solution? – antonone Jun 07 '22 at 04:59

How to use scala.js compiler as a compiler plugin, thus allowing it to integrate into Gradle or maven?

1 Answers1