sbt illegal dynamic reference in runMain

Question

I'm trying to run a code generator, and passing it the filename to write the output:

resourceGenerators in (proj, Compile) += Def.task {
  val file = (resourceManaged in (proj, Compile)).value / "swagger.yaml"
  (runMain in (proj, Compile)).toTask(s"api.swagger.SwaggerDump $file").value
  Seq(file)
}.value

However, this gives me:

build.sbt:172: error: Illegal dynamic reference: file
  (runMain in (proj, Compile)).toTask(s"api.swagger.SwaggerDump $file").value

Answer 1

Your code snippet has two problems:

1. You use { ... }.value instead of { ... }.taskValue. The type of resource generators is Seq[Task[Seq[File]]] and when you do value, you get Seq[File] not Task[Seq[File]]. That causes a legitimate compile error.
2. The dynamic variable file is used as the argument of toTask, which the current macro implementation prohibits.

Why static?

Sbt forces task implementations to have static dependencies on other tasks. Otherwise, sbt cannot perform task deduplication and cannot provide correct information in the inspect commands. That means that whichever task evaluation you perform inside a task cannot depend on a variable (a value known only at runtime), as your file in toTask does.

To overcome this limitation, there exists dynamic tasks, whose body allows you to return a task. Every "dynamic dependency" has to be defined inside a dynamic task, and then you can depend on the hoisted up dynamic values in the task that you return.

Dynamic solution

The following Scastie is the correct implementation of your task. I copy-paste the code so that folks can have a quick look, but go to that Scastie to check that it successfully compiles and runs.

resourceGenerators in (proj, Compile) += Def.taskDyn {
  val file = (resourceManaged in (proj, Compile)).value / "swagger.yaml"
  Def.task {
    (runMain in (proj, Compile))
      .toTask(s"api.swagger.SwaggerDump $file")
      .value
    Seq(file)
  }
}.taskValue

Discussion

If you had fixed the taskValue error, should your task implementation correctly compile?

In my opinion, yes, but I haven't looked at the internal implementation good enough to assert that your task implementation does not hinder task deduplication and dependency extraction. If it does not, the illegal reference check should disappear.

This is a current limitation of sbt that I would like to get rid of, either by improving the whole macro implementation (hoisting up values and making sure that dependency analysis covers more cases) or by just improving the "illegal references checks" to not be over pessimistic. However, this is a hard problem, takes time and it's not likely to happen in the short term.

If this is an issue for you, please file a ticket in sbt/sbt. This is the only way to know the urgency of fixing this issue, if any. For now, the best we can do is to document it.