4

Its confusing to understand version compatibility between different versions of Spark and hadoop. Similarly for Hadoop and hive.

Is there any table that can be followed to know which version of one is compatible with which version of other ?

Puneet Chaurasia
  • 441
  • 6
  • 14
  • The pre-built Spark packages on the download page explicitly says "for Hadoop 2.6 and later", so what's the confusion there? Probably same for Hive, but the "asparagus chart" by Hortonworks is what they officially support. http://hortonworks.com/wp-content/uploads/2016/03/asparagus-chart-hdp24.png – OneCricketeer May 26 '16 at 12:28
  • @cricket_007 Thanks for the reply. Thats helpful. I am adding the chart you provided as an answer to my question. Thanks again. – Puneet Chaurasia May 27 '16 at 09:07
  • 1
    It's not a compatibility chart, though, just what is used in each release of HDP – OneCricketeer May 27 '16 at 12:32

1 Answers1

0

The compatibility chart as shared by @cricket_007

http://hortonworks.com/wp-content/uploads/2016/03/asparagus-chart-hdp24.png

Puneet Chaurasia
  • 441
  • 6
  • 14
  • What about the ongoing compatibility for Spark with other libraries. Currently I using Spark 2.2 and not able to get working Hadoop 2.8.1 for saving some data to Azure blob storage from Spark. Refereing @cricket_007 who gave the chart earlier – Joy George Kunjikkuru Sep 01 '17 at 17:22