SBT clone git dependencies to a custom path using a plugin

Question

I'm creating an aggregate SBT project which depends on several other Git projects. I understand that I can refer to them as a dependency using RootProject(uri("...")) and SBT clones them into an SBT-managed path.

However, I need to download these into a custom path. The idea is to create a workspace that automatically downloads the related Git projects that can be worked on as well.

I was able to create a plugin with a task that clones the git repos using sbt-git plugin:

BundleResolver.scala

  def resolve: Def.Initialize[Task[Seq[String]]] = Def.task {
    val log = streams.value.log
    log.info("starting bundle resolution")

    val bundles = WorkspacePlugin.autoImport.workspaceBundles.value
    val bundlePaths = bundles.map(x => {
      val bundleName = extractBundleName(x)
      val localPath = file(".").toPath.toAbsolutePath.getParent.resolveSibling(bundleName)

      log.info(s"Cloning bundle : $bundleName")
      val (resultCode, outStr, errStr) = runCommand(Seq("git", "clone", x, localPath.toString))
      resultCode match {
        case 0 =>
          log.info(outStr)
          log.info(s"cloned $bundleName to path $localPath")

        case _ =>
          log.err(s"failed to clone $bundleName")
          log.err(errStr)
      }
      localPath.toString
    })

    bundlePaths
  }

WorkspacePlugin.scala

object WorkspacePlugin extends AutoPlugin {

  override def trigger = allRequirements
  override def requires: Plugins = JvmPlugin && GitPlugin

  object autoImport {
    // settings
    val workspaceBundles   = settingKey[Seq[String]]("Dependency bundles for this Workspace")
    val stagingPath        = settingKey[File]("Staging path")

    // tasks
    val workspaceClean     = taskKey[Unit]("Remove existing Workspace depedencies")
    val workspaceImport    = taskKey[Seq[String]]("Download the dependency bundles and setup builds")
  }

  import autoImport._

  override lazy val projectSettings = Seq(
    workspaceBundles   := Seq(), // default to no dependencies
    stagingPath        := Keys.target.value,

    workspaceClean     := BundleResolver.clean.value,
    workspaceImport    := BundleResolver.resolve.value,
  )
  override lazy val buildSettings = Seq()
  override lazy val globalSettings = Seq()
}

However, this will not add the cloned repos as sub projects to the main project. How can I achieve this?

UPDATE:: I had an idea to extend RootProject logic, so that I can create custom projects that would accept a git url, clone it in a custom path, and return a Project from it.

object WorkspaceProject {
  def apply(uri: URI): Project = {
    val bundleName = GitResolver.extractBundleName(uri.toString)
    val localPath = file(".").toPath.toAbsolutePath.getParent.resolveSibling(bundleName)

    // clone the project
    GitResolver.clone(uri, localPath)

    Project.apply(bundleName.replaceAll(".", "-"), localPath.toFile)
  }
}

I declared this in a plugin project, but can't access it where I'm using it. Do you think it'll work? How can I access it in my target project?

Have you seen [sbt-sriracha](https://eed3si9n.com/hot-source-dependencies-using-sbt-sriracha/)? It doesn't solve cloning repos to a set location, but it can add both remote and _local_ repos as source dependencies. — laughedelic, Aug 24 '22 at 14:26
thanks, but can't we already specify remote projects using ProjectRoot(uri(..)) and local ones using "project in files(..)"? Also, I can download the remote sources using a custom plugin that invokes Git clone as I described above, but how would I declare them as a dependency of my main project, after I've downloaded them? If I declare them as a placeholder path in dependsOn, then SBT will error out since they don't exist yet. — devin, Aug 24 '22 at 14:47

score 1 · Answer 1 · answered Aug 24 '22 at 20:11

Can't believe it was this simple.

In my plugin project, I created a new object to use in place of RootProject

object WorkspaceProject {
  def apply(uri: URI): RootProject = {
    val bundleName = GitResolver.extractBundleName(uri.toString)

    val localPath = file(".").toPath.toAbsolutePath.getParent.resolve(bundleName)

    if(!localPath.toFile.exists()) {
      // clone the project
      GitResolver.clone(uri, localPath)
    }

    RootProject(file(localPath.toString))
  }
}

Then use it like this:

build.sbt

lazy val depProject = WorkspaceProject(uri("your-git-repo.git"))

lazy val root = (project in file("."))
  .settings(
    name := "workspace_1",
  ).dependsOn(depProject)

SBT clone git dependencies to a custom path using a plugin

1 Answers1