Version Compatibility issues with Scala, Spark, Spark NLP

Asked Jul 09 '19 at 03:39

Active Jul 09 '19 at 03:39

Viewed 219 times

I am new to 'Spark NLP' and I got stuck in version compatibility issues only. That may seems to be silly but still I request you to help me in this:

‘Spark NLP’ is built on top of Apache Spark 2.4.0 and such is the only supported release (mentioned on https://nlp.johnsnowlabs.com/docs/en/quickstart)
Spark 2.4.3 uses Scala 2.12 (mentioned on https://spark.apache.org/docs/latest/)
But maven repo repository just have spark_nlp libararies for scala 2.11 only not after that (see https://repo1.maven.org/maven2/com/johnsnowlabs/nlp/, only spark-nlp_2.11 is present and not spark-nlp_2.12)

Please help

asked Jul 09 '19 at 03:39

amandeep1991

2

could you not use spark-core_2.11/2.4.3 with scala 2.11? https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11/2.4.3 – Sai Kiran KrishnaMurthy Jul 09 '19 at 04:23
1

Yes, he can. Spark 2.4.3 is also coming with Scala 2.11 and Apache Spark has had some problems moving to Scala 2.12. The evidence is this back and forth for supporting it and then reverting it in `2.4.x` branch :) Just use Apache Spark with Scala 2.11 and you'll be fine. – Maziyar Sep 08 '19 at 09:37

0 Answers0