Reading resources from a macro in an SBT project

Question

Suppose I have a Scala project with three sub-projects, with files like this:

foo/src/main/scala/Foo.scala
foo/src/main/resources/foo.txt

bar/src/main/scala/Bar.scala
bar/src/main/resources/bar.txt

baz/src/main/scala/Baz.scala
baz/src/main/resources/baz.txt

Foo.scala contains a simple macro that reads a resource at a given path:

import scala.language.experimental.macros
import scala.reflect.macros.Context

object Foo {
  def countLines(path: String): Option[Int] = macro countLines_impl

  def countLines_impl(c: Context)(path: c.Expr[String]) = {
    import c.universe._

    path.tree match {
      case Literal(Constant(s: String)) => Option(
        this.getClass.getResourceAsStream(s)
      ).fold(reify(None: Option[Int])) { stream =>
        val count = c.literal(io.Source.fromInputStream(stream).getLines.size)
        reify(Some(count.splice))
      }
      case _ => c.abort(c.enclosingPosition, "Need a literal path!")
    }
  }
}

If the resource can be opened, countLines returns the number of lines; otherwise it's empty.

The other two Scala source files just call this macro:

object Bar extends App {
  println(Foo.countLines("/foo.txt"))
  println(Foo.countLines("/bar.txt"))
  println(Foo.countLines("/baz.txt"))
}

And:

object Baz extends App {
  println(Foo.countLines("/foo.txt"))
  println(Foo.countLines("/bar.txt"))
  println(Foo.countLines("/baz.txt"))
}

The contents of the resources don't really matter for the purposes of this question.

If this is a Maven project, I can easily configure it so that the root project aggregates the three sub-projects and baz depends on bar, which depends on foo. See this Gist for the gory details.

With Maven everything works as expected. Bar can see the resources for foo and bar:

Some(1)
Some(2)
None

And Baz can see all of them:

Some(1)
Some(2)
Some(3)

Now I try the same thing with SBT:

import sbt._
import Keys._

object MyProject extends Build {
  lazy val root: Project = Project(
    id = "root", base = file("."),
    settings = commonSettings
  ).aggregate(foo, bar, baz)

  lazy val foo: Project = Project(
    id = "foo", base = file("foo"),
    settings = commonSettings
  )

  lazy val bar: Project = Project(
    id = "bar", base = file("bar"),
    settings = commonSettings,
    dependencies = Seq(foo)
  )

  lazy val baz: Project = Project(
    id = "baz", base = file("baz"),
    settings = commonSettings,
    dependencies = Seq(bar)
  )

  def commonSettings = Defaults.defaultSettings ++ Seq(
    scalaVersion := "2.10.2",
    libraryDependencies <+= scalaVersion("org.scala-lang" % "scala-compiler" % _)
  )
}

But now Bar can only see the resources in foo:

Some(1)
None
None

And Baz can only see foo and bar:

Some(1)
Some(2)
None

What's going on here? This SBT build file seems to me to be a pretty literal translation of the Maven configuration. I have no problem opening a console in the bar project and reading /bar.txt, for example, so why can't these projects see their own resources when calling a macro?

@0__: Thanks, I should have mentioned that—yes, they're there (and are available from non-macro code both in the console and in the project source). — Travis Brown, Jun 16 '13 at 15:09
Not a macro expert, but it must be a class loader issue IMO. At what points exactly is `this.getClass.getResourceAsStream` executed? Perhaps you need to use the context or something else as the class loader reference? No clue... — 0__, Jun 16 '13 at 15:11
@0__: I'm assuming it's something like that, but none of my flailing hit a solution, and the fact that it works as expected in the Maven case makes me think it should be something simple. — Travis Brown, Jun 16 '13 at 15:16
Just thinking... Since the resource is queried at compile time, you assume that it is on the class path. But I wonder, wouldn't it be perfectly legal if sbt excluded the dependencies resources from the compile class path—after all in "normal" (non-macro) conditions there would be no way to access those resources anyway. [Maybe related problem](http://stackoverflow.com/questions/8193904/sbt-test-dependencies-in-multiprojects-make-the-test-code-available-to-dependen) — 0__, Jun 16 '13 at 15:21
Looking at the [classpaths doc](https://github.com/harrah/xsbt/wiki/Classpaths), maybe you need to add something like `unmanagedResources in Compile <++= ...` or `unmanagedClasspath in Compile <++= ...`. Using `show` from the console you may be able to find out if everything you need is in the compile time class path. — 0__, Jun 16 '13 at 15:33
An alternative would be using System.getProperties to communicate between SBT and macros. That's not very pretty, though. — Eugene Burmako, Jun 19 '13 at 12:27

score 6 · Accepted Answer · edited Mar 09 '16 at 19:25

SBT does not add the resources of the current project into the build class-path. Probably because it is rarely needed.

You just have to pipe one (the resources) into the other (the classpath):

def commonSettings = Defaults.defaultSettings ++ Seq(
  scalaVersion := "2.10.2",
  libraryDependencies <+= scalaVersion("org.scala-lang" % "scala-compiler" % _),
  // add resources of the current project into the build classpath
  unmanagedClasspath in Compile <++= unmanagedResources in Compile
)

In that project you would only need that for bar and baz.

UPDATE: Since sbt 0.13, <+= and <++= are unnecessary (and undocumented) thanks to the new macro-based syntax:

libraryDependencies += "org.scala-lang" % "scala-compiler" % scalaVersion.value
unmanagedClasspath in Compile ++= (unmanagedResources in Compile).value

this adds resource files, not resource directories. See https://github.com/sbt/sbt/issues/1965#issuecomment-376958438 for the real fix — fommil, Mar 28 '18 at 17:12

Reading resources from a macro in an SBT project

1 Answers1