I have a scala project using geotools / JAI dependencies in order to process ESRi ASCII Grid files to WKT in java. When deploying a fat-jar, I get a SIGSEV in the native part of the jai code.
It is using the following dependencies:
lazy val geotools = "17.0"
libraryDependencies ++= Seq(
"org.geotools" % "gt-main" % geotools,
"org.geotools" % "gt-arcgrid" % geotools,
"org.geotools" % "gt-process-raster" % geotools)
When creating a fat jar like:
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs@_*) =>
xs match {
// Concatenate everything in the services directory to keep GeoTools happy.
case ("services" :: _ :: Nil) =>
MergeStrategy.concat
// Concatenate these to keep JAI happy.
case ("javax.media.jai.registryFile.jai" :: Nil) | ("registryFile.jai" :: Nil) | ("registryFile.jaiext" :: Nil) =>
MergeStrategy.concat
case (name :: Nil) => {
// Must exclude META-INF/*.([RD]SA|SF) to avoid "Invalid signature file digest for Manifest main attributes" exception.
if (name.endsWith(".RSA") || name.endsWith(".DSA") || name.endsWith(".SF")) {
MergeStrategy.discard
}
else {
MergeStrategy.deduplicate
}
}
case _ => MergeStrategy.deduplicate
}
case _ => MergeStrategy.deduplicate
}
One of JAIS JNI native C dependencies blows up with a SIGSEV. When instead simply using the (a lot unsafer MergeStrategy.first) like:
assemblyMergeStrategy in assembly := {
// TODO bad strategy, find out why below does not work.
case PathList("org", "datasyslab", xs@_*) => MergeStrategy.singleOrError
case PathList("com", "esotericsoftware", xs@_*) => MergeStrategy.last
case PathList("META-INF", "MANIFEST.MF") => MergeStrategy.discard
case _ => MergeStrategy.first
}
The code works without SIGSEV. When looking at the logs of MergeStrategy.SingleOrError, I cant figure out what is the dependency causing the problems in the native code:
[error] (*:assembly) deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/org.geotools/gt-coverage/jars/gt-coverage-17.0.jar:META-INF/registryFile.jai
[error] /Users/geoheil/.ivy2/cache/org.jaitools/jt-zonalstats/jars/jt-zonalstats-1.4.0.jar:META-INF/registryFile.jai
[error] /Users/geoheil/.ivy2/cache/org.geotools/gt-process-raster/jars/gt-process-raster-17.0.jar:META-INF/registryFile.jai
[error] /Users/geoheil/.ivy2/cache/org.jaitools/jt-rangelookup/jars/jt-rangelookup-1.4.0.jar:META-INF/registryFile.jai
[error] /Users/geoheil/.ivy2/cache/org.jaitools/jt-contour/jars/jt-contour-1.4.0.jar:META-INF/registryFile.jai
[error] /Users/geoheil/.ivy2/cache/org.jaitools/jt-vectorize/jars/jt-vectorize-1.4.0.jar:META-INF/registryFile.jai
[error] deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.affine/jt-affine/jars/jt-affine-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.scale/jt-scale/jars/jt-scale-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.vectorbin/jt-vectorbin/jars/jt-vectorbin-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.translate/jt-translate/jars/jt-translate-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.algebra/jt-algebra/jars/jt-algebra-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.bandmerge/jt-bandmerge/jars/jt-bandmerge-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.bandselect/jt-bandselect/jars/jt-bandselect-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.bandcombine/jt-bandcombine/jars/jt-bandcombine-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.border/jt-border/jars/jt-border-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.buffer/jt-buffer/jars/jt-buffer-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.crop/jt-crop/jars/jt-crop-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.mosaic/jt-mosaic/jars/jt-mosaic-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.lookup/jt-lookup/jars/jt-lookup-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.nullop/jt-nullop/jars/jt-nullop-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.rescale/jt-rescale/jars/jt-rescale-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.stats/jt-stats/jars/jt-stats-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.warp/jt-warp/jars/jt-warp-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.zonal/jt-zonal/jars/jt-zonal-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.binarize/jt-binarize/jars/jt-binarize-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.format/jt-format/jars/jt-format-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.colorconvert/jt-colorconvert/jars/jt-colorconvert-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.errordiffusion/jt-errordiffusion/jars/jt-errordiffusion-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.orderdither/jt-orderdither/jars/jt-orderdither-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.colorindexer/jt-colorindexer/jars/jt-colorindexer-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.imagefunction/jt-imagefunction/jars/jt-imagefunction-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.piecewise/jt-piecewise/jars/jt-piecewise-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.classifier/jt-classifier/jars/jt-classifier-1.0.13.jar:META-INF/registryFile.jaiext
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.rlookup/jt-rlookup/jars/jt-rlookup-1.0.13.jar:META-INF/registryFile.jaiext
[error] deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/javax.media/jai_imageio/jars/jai_imageio-1.1.jar:META-INF/services/javax.imageio.spi.ImageInputStreamSpi
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.imageio-ext/imageio-ext-streams/jars/imageio-ext-streams-1.1.17.jar:META-INF/services/javax.imageio.spi.ImageInputStreamSpi
[error] deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/javax.media/jai_imageio/jars/jai_imageio-1.1.jar:META-INF/services/javax.imageio.spi.ImageOutputStreamSpi
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.imageio-ext/imageio-ext-streams/jars/imageio-ext-streams-1.1.17.jar:META-INF/services/javax.imageio.spi.ImageOutputStreamSpi
[error] deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/javax.media/jai_imageio/jars/jai_imageio-1.1.jar:META-INF/services/javax.imageio.spi.ImageReaderSpi
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.imageio-ext/imageio-ext-tiff/jars/imageio-ext-tiff-1.1.17.jar:META-INF/services/javax.imageio.spi.ImageReaderSpi
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.imageio-ext/imageio-ext-arcgrid/jars/imageio-ext-arcgrid-1.1.17.jar:META-INF/services/javax.imageio.spi.ImageReaderSpi
[error] deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/javax.media/jai_imageio/jars/jai_imageio-1.1.jar:META-INF/services/javax.imageio.spi.ImageWriterSpi
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.imageio-ext/imageio-ext-tiff/jars/imageio-ext-tiff-1.1.17.jar:META-INF/services/javax.imageio.spi.ImageWriterSpi
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.imageio-ext/imageio-ext-arcgrid/jars/imageio-ext-arcgrid-1.1.17.jar:META-INF/services/javax.imageio.spi.ImageWriterSpi
[error] deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/javax.media/jai_imageio/jars/jai_imageio-1.1.jar:META-INF/services/javax.media.jai.OperationRegistrySpi
[error] /Users/geoheil/.ivy2/cache/it.geosolutions.jaiext.crop/jt-crop/jars/jt-crop-1.0.13.jar:META-INF/services/javax.media.jai.OperationRegistrySpi
[error] deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/org.jaitools/jt-zonalstats/jars/jt-zonalstats-1.4.0.jar:META-INF/services/javax.media.jai.OperationsRegistrySpi
[error] /Users/geoheil/.ivy2/cache/org.geotools/gt-process-raster/jars/gt-process-raster-17.0.jar:META-INF/services/javax.media.jai.OperationsRegistrySpi
[error] /Users/geoheil/.ivy2/cache/org.jaitools/jt-rangelookup/jars/jt-rangelookup-1.4.0.jar:META-INF/services/javax.media.jai.OperationsRegistrySpi
[error] /Users/geoheil/.ivy2/cache/org.jaitools/jt-contour/jars/jt-contour-1.4.0.jar:META-INF/services/javax.media.jai.OperationsRegistrySpi
[error] /Users/geoheil/.ivy2/cache/org.jaitools/jt-vectorize/jars/jt-vectorize-1.4.0.jar:META-INF/services/javax.media.jai.OperationsRegistrySpi
[error] deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/org.geotools/gt-main/jars/gt-main-17.0.jar:META-INF/services/org.geotools.filter.FunctionFactory
[error] /Users/geoheil/.ivy2/cache/org.geotools/gt-process/jars/gt-process-17.0.jar:META-INF/services/org.geotools.filter.FunctionFactory
[error] deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/org.geotools/gt-main/jars/gt-main-17.0.jar:META-INF/services/org.geotools.util.ConverterFactory
[error] /Users/geoheil/.ivy2/cache/org.geotools/gt-process-raster/jars/gt-process-raster-17.0.jar:META-INF/services/org.geotools.util.ConverterFactory
[error] deduplicate: different file contents found in the following:
[error] /Users/geoheil/.ivy2/cache/org.geotools/gt-main/jars/gt-main-17.0.jar:META-INF/services/org.opengis.filter.expression.Function
[error] /Users/geoheil/.ivy2/cache/org.geotools/gt-coverage/jars/gt-coverage-17.0.jar:META-INF/services/org.opengis.filter.expression.Function
[error] /Users/geoheil/.ivy2/cache/org.geotools/gt-cql/jars/gt-cql-17.0.jar:META-INF/services/org.opengis.filter.expression.Function
and here the sigsev information
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fd780b9bfe1, pid=21657, tid=0x00007f0674c4a700
#
# JRE version: OpenJDK Runtime Environment (8.0_121-b13) (build 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13)
# Java VM: OpenJDK 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 )
# Problematic frame:
# V [libjvm.so+0x8a4fe1]
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/vagrant/development/projects/HybridAccess/spark/hs_err_pid21657.log
Compiled method (c1) 6599346 22508 ! 3 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue::take (203 bytes)
total in heap [0x00007fd76bda5b90,0x00007fd76bda83a8] = 10264
relocation [0x00007fd76bda5cb8,0x00007fd76bda5eb8] = 512
main code [0x00007fd76bda5ec0,0x00007fd76bda7360] = 5280
stub code [0x00007fd76bda7360,0x00007fd76bda7508] = 424
metadata [0x00007fd76bda7508,0x00007fd76bda7560] = 88
scopes data [0x00007fd76bda7560,0x00007fd76bda7d80] = 2080
scopes pcs [0x00007fd76bda7d80,0x00007fd76bda8170] = 1008
dependencies [0x00007fd76bda8170,0x00007fd76bda8188] = 24
handler table [0x00007fd76bda8188,0x00007fd76bda8308] = 384
nul chk table [0x00007fd76bda8308,0x00007fd76bda83a8] = 160
And the links I have found for maven: JAI dependencies in geotools are hard deploy as a single fat jar (http://docs.geotools.org/latest/userguide/faq.html#how-do-i-create-an-executable-jar-for-my-geotools-app, http://osgeo-org.1560.x6.nabble.com/Building-runnable-jar-file-td4320365.html).
