4

Sorry I am pretty new to Google Clouds and Elasticsearch but I didn't find any doc on this. Basically we just deployed an Elasticsearch node on Google Clouds using Bitnami. But I am unable to connect with Python to it. I tried a lot of different formulation for the code below (host is the google cloud node external IP, user & password are those asked by Bitnami to connect to my application). Can someone let me know from where is it coming from and what is the correct syntax? 

from elasticsearch import Elasticsearch

connection_parameters = [{'host': 'http://104.196.x.x', 'port': 80}]
    es = Elasticsearch(connection_parameters, http_auth=('user', 'password'))
print(es.info())

Here is the error:

elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f72e757a090>: Failed to establish a new connection: [Errno -2] Name or service not known) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f72e757a090>: Failed to establish a new connection: [Errno -2] Name or service not known)

Thanks a lot if you can help me on that!

Barrec
  • 63
  • 6
  • This could be caused by an authentication error in your connection. It looks like ElasticSearch is using urllib3 to make it's HTTP connections and I specifically remember having an issue somewhat like this when I was using GAE. I had to turn `verify=False` or something to that effect (it's been a while) in order to make my connection. Also, are you sure that ElasticSearch is serving on port 80? – sulimmesh Mar 03 '16 at 17:28
  • Hi, thanks for your answer. I tried but it didn't work. – Barrec Mar 03 '16 at 20:39
  • I'd bet on it being the port number then being the problem. Is this ElasticSearch server running on your GCE as well? GCE can sometiems not like outgoing HTTP requests although GAE usually doesn't have nay problems with that in my experience. There might options in GCE to let you control the allowed ports. – sulimmesh Mar 03 '16 at 20:50
  • Yes Elastic is running on the GCE. I made sure all ports are open (tcp:80; tcp:9200; tcp:9300) and the HTTP traffic is allowed... – Barrec Mar 03 '16 at 21:51

2 Answers2

3

Looks like the problem is with your firewall rules on Google Compute Engine. You will need to explicitly allow traffic from outside of Google to your ES node/s.

You can create new firewall rule using Google Cloud SDK [1]:

gcloud compute firewall-rules create allow-traffic-to-es --allow tcp:80,tcp:443 --source-ranges 0.0.0.0/0 --target-tags es

Please note that this rule will allow traffic from (0.0.0.0/0) to your ElasticSearch cluster, so you might implement more strict CIDR range.

[1] Google Cloud SDK

DoiT International
  • 2,405
  • 1
  • 20
  • 24
  • Hi, unfortunately the traffic was already allowed: 'bitnami-elasticsearch-2-2-0-0-firewall Network default Source filter Allow from any source (0.0.0.0/0) Allowed protocols and ports tcp:80 tcp:443' – Barrec Mar 03 '16 at 19:25
  • @Barrec - have you tried to connect to port 9200? of course you will need to allow this port in your firewall as well. – DoiT International Mar 03 '16 at 19:50
  • Hi, yes I did but still didn't work. Maybe I did missed something in the elasticsearch configuration? – Barrec Mar 03 '16 at 20:38
  • Have you made sure you host config in elasticsearch conf is binded to 0.0.0.0? – DoiT International Mar 03 '16 at 20:48
  • Yes as well: `network.host: 0.0.0.0 network.publish_host: 10.240.0.4 transport.tcp.port: 9300 http.port: 9200 cluster.name: elastic-pool-dev` – Barrec Mar 03 '16 at 21:52
0

I think that you should exclude the protocol part of the connection string like this instead:

connection_parameters = [{'host': '104.196.x.x', 'port': 80}]

Or this way:

es = Elasticsearch(['http://user:f00b4r42@104.196.x.x:80/elasticsearch'])

Niklas Rosencrantz
  • 25,640
  • 75
  • 229
  • 424