I have Elasticsaerch cluster and want to use Apache Spark to run computations on it. The ES-Hadoop connector contains a Spark library that can run computations on Elasticsearch. Do I have to install Hadoop first or can I just put elasticsearch-spark_2.10-2.2.0-rc1.jar in Spark classpath as documentation mentions in order to make it work?
Asked
Active
Viewed 468 times
-2
Eyal.Dahari
- 760
- 6
- 13
-
@AlbertoBonsanto - this is not a duplicated question. I am asking about Elasticsearch hadoop connector that contains Spark and many more Hadoop solutions. My concern is about this connector and not about if Spark needs Hadoop in order to run which I know it does not. Please kindly redeem my points. – Eyal.Dahari Mar 10 '16 at 00:02
-
1yes you can just put the jar on the classpath and all will be working fine. Why would the documentation lie? – Mateusz Dymczyk Mar 10 '16 at 01:59
1 Answers
1
If you want to use ElasticSearch with Spark then I would suggest you to use Elasticsearch's Spark Native Support.
It is more easy to use and you can directly laod/save data to/from Elastic search as Spark RDD. For more detail refer : Elasticsearch Apache Spark support
Mahendra
- 1,436
- 9
- 15