0

I have set up a Hadoop cluster with 3 DataNodes and 1 NameNode. I have also installed elasticsearch on one of the DataNodes. But I'm not able to access the HDFS using elasticsearch.(Hadoop cluster and Elasticsearch are working fine independently) Now, I want to integrate my Hadoop cluster with elasticsearch. I found there is a seperate plugin for that. But I'm not able to download it.(bin/plugin -i elasticsearch/elasticsearch-repository-hdfs/1.3.0.M3 command is not working. It is failing everytime I executed it). Can anyone suggest me which plugin I should download. Also the path to place that plugin and how to aceess it using the url.

Thanks in advance

user3595569
  • 1
  • 1

3 Answers3

0

Try this

1) Download jars from this link

2) Unzip it and place the jars in plugin folder of Elasticsearch

3) restart the server and start using it..!

BlackPOP
  • 5,657
  • 2
  • 33
  • 49
  • How will I use it? Through url?? – user3595569 May 05 '14 at 06:10
  • after plugin available... follow this link to configure... http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/configuration.html – BlackPOP May 05 '14 at 07:15
  • the link you have given to download the jars is showing 404 error. – user3595569 May 05 '14 at 11:31
  • pls try this https://oss.sonatype.org/content/repositories/snapshots/org/elasticsearch/elasticsearch-hadoop/1.3.0.BUILD-SNAPSHOT/elasticsearch-hadoop-1.3.0.BUILD-20140501.012020-413.zip – BlackPOP May 05 '14 at 12:34
  • I have done the configurations. Now how will I be able to use it? Can I access it through url like we access normal elasticsearch with port 9200? – user3595569 May 07 '14 at 09:24
  • HADOOP_CLASSPATH: HADOOP_CLASSPATH=" Now I have jars in elasticsearch/plugins. So writing HADOOP_CLASSPATH="/elasticsearch/plugins/" will be fine? – user3595569 May 07 '14 at 09:37
0

I suggest you try to use this repo.

It's an Elasticsearch real-time search and analytics natively integrated with Hadoop and you can follow the documentation provided here to use it.

The repo is provided by Elasticsearch.

eliasah
  • 39,588
  • 11
  • 124
  • 154
  • Thanks a ton for your reply. I have already placed that repo in plugin folder of elasticsearch. Is that correct? Should I do any configuration settings in that? And please tell me how to use it, I mean how to access it using url after evrything is done. – user3595569 May 05 '14 at 05:32
  • I haven't worked with that repo quite much yet, but you can find the configurations [here](https://github.com/elasticsearch/elasticsearch-hadoop#usage) and [here](http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/configuration.html) on the official page. They explain it better than will I think ;) – eliasah May 05 '14 at 08:44
0

The elasticsearch hadoop library is not a plugin. You need to download or build it and put it into the classpath of the hadoop/spark application you will use.

Mogsdad
  • 44,709
  • 21
  • 151
  • 275
Sarwar Bhuiyan
  • 344
  • 1
  • 7