Stack Overflow
Stack Overflow

Questions tagged [sbt-assembly]

sbt-assembly is an sbt plugin for building a single JAR file of a project.

sbt-assembly is open-source and is available under the MIT licence.

528 questions
2
votes
1 answer

How to change Provided dependencies when using Intellij with SBT?

My build.sbt looks like this: libraryDependencies ++= Seq( "org.apache.hadoop" % "hadoop-aws" % sparkVersion % Provided, "org.apache.spark" %% "spark-core" % sparkVersion % Provided, "org.apache.spark" %% "spark-sql" % sparkVersion %…
Igor Gatis
  • 4,648
  • 10
  • 43
  • 66
2
votes
0 answers

How to shade packages inside a fat jar depdency

I've an SBT project that depends on "com.google.cloud.bigdataoss" % "gcs-connector" % "hadoop3-2.2.2" which is bringing a recent version of google-api-services-storage. I've also another dependency to Sparkling Water which is a fat jar that seems…
bachr
  • 5,780
  • 12
  • 57
  • 92
2
votes
1 answer

SBT Assembly remove classes from another JAR

This is the package structure of my repo: src |-- main |-- scala |-- me.sparker0i.spark |-- runner |-- Runner.scala |-- utils |-- Constants.scala |--…
Sparker0i
  • 1,787
  • 4
  • 35
  • 60
2
votes
1 answer

How to fix the dependency graph to resolve deduplicate errors?

I'm facing some deduplicate errors like below, which indicate that some of my dependencies import other dependencies which contains files with the same path name. Since they have the same path, they cannot be included together in the jar file that…
Guillaume Chérel
  • 1,478
  • 8
  • 17
2
votes
2 answers

sbt assembly deduplicate: different file contents found in the following: jackson-annotations

having those deps: libraryDependencies ++= Seq( "com.typesafe.akka" %% "akka-http-spray-json" % akkaHttpVersion, "com.typesafe.akka" %% "akka-stream" % akkaVersion, "com.typesafe.akka" %% "akka-http" % akkaHttpVersion, "com.typesafe.akka" %%…
FrancMo
  • 2,459
  • 2
  • 19
  • 39
2
votes
1 answer

Which version of Sbt and Sbt assembly to use for Spack 2.4.5 and Scala 2.11?

Currently, in AWS EMR Cluster, I am using Spark v2.4.5 which comes with Scala v2.11. So in my project, I want to use Scala v2.11 and corresponding SBT and Sbt-Assembly versions. But I am getting one or the other Version conflicts with all the…
sunny_dev
  • 765
  • 3
  • 15
  • 34
2
votes
1 answer

Uber jar with custom folder structure with IntelliJ and SBT

I am fairly new to cloud and SBT/IntelliJ, So trying my luck with IntelliJ & SBT build environment to deploy my jar on data proc cluster. Here's a screen shot of my project structure: Code is quite simple with main defined in 'mytestmain' which…
vikrant rana
  • 4,509
  • 6
  • 32
  • 72
2
votes
1 answer

Play JSON library and sbt assembly merge error

Even with a simple Hello program, if I add the play-json as a library dependency in the build.sbt file, I always get the an merge error when I execute the 'assembly' command in sbt (sbt-assembly plugin). [error] stack trace is suppressed; run last…
Heitor Barbieri
  • 135
  • 8
2
votes
1 answer

Dockerfile, sbt-assembly - is it possible to use sbt-assembly in dockerfile?

I have a Dockerfile script: FROM hseeberger/scala-sbt:8u222_1.3.4_2.13.1 RUN sbt assembly But when I ran it, I got an error: [error] Not a valid command: assembly [error] Not a valid project ID: assembly [error] Expected ':' [error] Not a valid…
Developus
  • 1,400
  • 2
  • 14
  • 50
2
votes
0 answers

Flink ML - java.lang.ClassNotFoundException: org.apache.flink.ml.math.DenseVector

When submitting a job to my Flink 1.8.1 cluster, it fails with the following exception: java.lang.ClassNotFoundException: org.apache.flink.ml.math.DenseVector However, the mentioned class seems to be in my jar according to: jar -tf myjar.jar | grep…
Guenter
  • 106
  • 7
2
votes
2 answers

What does double colon (or colon-colon) :: mean in Scala?

I was having an issue with the sbt build of my Scala project (duplicate entry: META-INF/MANIFEST.MF) and the following lines solved the problem: assemblyMergeStrategy in assembly := { case PathList("META-INF", xs @ _*) => (xs map…
user1485864
  • 499
  • 6
  • 18
2
votes
3 answers

sbt assembly fails due to different file contents found

I am trying to build a project in GitLab. In gitlab-ci.yml, I ran sbt assembly and encountered annoying exception. [error] (soda/*:assembly) deduplicate: different file contents found in the following: [error]…
EyalP
  • 195
  • 1
  • 11
2
votes
1 answer

How to configure SBT to produce separate jars for dependencies and application code?

I'm using SBT as a build tool for spark projects. I'm able to create a fat jar of my dependencies using the sbt-assembly plugin. However, this produces a ~120M jar, mostly of dependencies which I need to keep uploading to S3 to run my code -- this…
rongenre
  • 1,334
  • 11
  • 21
2
votes
1 answer

ClassNotFoundException: net.logstash.log4j.JSONEventLayoutV1 in assembled Spark jar file

I am running Spark application from jar file. When I define in log4j custom file and run my application simply from the main class the logs output works fine, however, when I run the application from jar file I get such an error: log4j:ERROR Could…
Cassie
  • 2,941
  • 8
  • 44
  • 92
2
votes
1 answer

Sbt what is the difference placing settings in the file or in commonSettings in multi project

Beginners question, I have a multi project sbt file is there a difference if I put general settings at the beginning of the file? e.g : organization := "com.example" version := "0.0.1-SNAPSHOT" scalaVersion := "2.11.12" resolvers ++= Seq( …
igx
  • 4,101
  • 11
  • 43
  • 88
1 2 3
…
35 36