how to shade before compile with SBT?

Question

Our project mainly consists of two parts

Build.scala where the root project lies
BuildShaded.scala where some external dependencies are shaded with sbt-assembly. The shaded jars will be depended upon by sub projects under the root project through unmanagedJars setting.

The question is how to assembly the shaded project before compiling the root project. Otherwise, the root project will fail to compile since those classes in the shaded jars are not available.

Why are you using a separate build to shade the external dependencies? Why not take them as managed dependencies and shade them inside the `Build.scala` root? — Yuval Itzchakov, Nov 03 '16 at 08:53
because I need to depend on the classes after shading. "managed dependencies" will introduce classes before shading. — manuzhang, Nov 03 '16 at 13:18
Not necessarily. If you shade the project both `inLibrary` and `inProject`, it will shed the *internal dependencies* as well. — Yuval Itzchakov, Nov 03 '16 at 13:23
what do you mean ? Do you have any examples or documentation ? — manuzhang, Nov 03 '16 at 13:24

Yuval Itzchakov · Accepted Answer · 2021-01-15T15:07:02.710

19

As I said in the comments, I would take a different route. If you take on the dependencies as managed dependencies, you can shade them both in the library itself and inside your project.

Let's see an example:

Assume that I have a project which takes dependency on com.typesafe.config. I can shade it inside it's own library, meaning inside the code of com.typesafe.config, and also in the consuming library.

You define it like this:

assemblyShadeRules in assembly ++= Seq(
  ShadeRule.rename("com.typesafe.config.**" -> "my_conf.@1")
    .inLibrary("com.typesafe" % "config" % "1.3.0")
    .inProject
)

Which basically means "take any package that beings with com.typesafe.config and shade it to my_conf."

Note that we're using both inLibrary and inProject. The former means "change the package names and references to them inside com.typesafe.config" and inProject means "change all references to com.typesafe.config inside my code".

Now, the output of that looks like this:

This is how the package internally looks now (my_conf was originally com.typesafe.config before shading):

And this is the package your code will reference:

edited Jan 15 '21 at 15:07

answered Nov 03 '16 at 14:38

Yuval Itzchakov

146,575
32
257
321

"managed dependencies" don't seem to work. The depending projects fail to compile with the shaded classes not found. By "managed dependencies" you mean like `Project("core").dependsOn(shaded)` ? By the way, how is `inLibrary.inProject` different from `inAll` ? – manuzhang Nov 04 '16 at 00:59
@manuzhang No, by managed dependencies I mean dependencies managed by SBT. Can you show an example of a `build.sbt` so we can fit the shading with it. – Yuval Itzchakov Nov 04 '16 at 06:17
1

I finally understood what you mean and shading the depending projects with managed dependencies works. Thanks for saving my day. – manuzhang Nov 11 '16 at 03:02
[This](https://github.com/apache/incubator-gearpump/blob/master/project/BuildGearpump.scala) is our build file finally. The approach is not so clean that I have to manually add/remove dependencies in the published pom file though. – manuzhang Nov 12 '16 at 11:38
Your answer is slightly misleading. `inLibrary` means: change packages name AND references. – tomek.xyz Jan 15 '21 at 12:55
1

@tomek.xyz Edited for clarity – Yuval Itzchakov Jan 15 '21 at 13:10

how to shade before compile with SBT?

1 Answers1

Linked