I am trying to set up an elasticsearch cluster on aws ec2 machines using elasticsearch-cloud-aws plugin. I followed instructions from the github readme file but wasn't able to make it work with IAM role set up.
I am using ES 1.7.1 and plugin version 2.7.0
Started EC2 instances with IAM role associated with the following policies:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1439999260000",
"Effect": "Allow",
"Action": [
"ec2:DescribeAvailabilityZones",
"ec2:DescribeInstances",
"ec2:DescribeRegions",
"ec2:DescribeSecurityGroups",
"ec2:DescribeTags"
],
"Resource": [
"*"
]
}
]
}
In the elasticsearch configuration files on all the instances a modified following settings:
cluster.name: my_cluster
discovery.zen.ping.multicast.enabled: false
discovery.type: ec2
discovery.type: ec2
discovery.ec2.groups: my_security_group
discovery.ec2.availability_zones: eu-west-1b
With the DEBUG level for discovery module following is found in the es log.
[2015-08-19 18:41:12,024][DEBUG][discovery.ec2 ] [Misfit] using ping.timeout [3s], join.timeout [1m], master_election.filter_client [true], master_election.filter_data [false]
[2015-08-19 18:41:12,025][DEBUG][discovery.zen.fd ] [Misfit] [master] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2015-08-19 18:41:12,027][DEBUG][discovery.zen.fd ] [Misfit] [node ] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2015-08-19 18:41:12,785][DEBUG][discovery.ec2 ] [Misfit] using host_type [PRIVATE_IP], tags [{}], groups [[ES-development-expose]] with any_group [true], availability_zones [[eu-west-1b]]
[2015-08-19 18:41:13,479][INFO ][node ] [Misfit] initialized
[2015-08-19 18:41:13,479][INFO ][node ] [Misfit] starting ...
[2015-08-19 18:41:13,560][INFO ][transport ] [Misfit] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.0.0.29:9300]}
[2015-08-19 18:41:13,581][INFO ][discovery ] [Misfit] es-aws-dev/W24WvY0yQyew0khFFDrQVA
[2015-08-19 18:41:14,805][DEBUG][discovery.ec2 ] [Misfit] using dynamic discovery nodes []
[2015-08-19 18:41:16,433][DEBUG][discovery.ec2 ] [Misfit] using dynamic discovery nodes []
[2015-08-19 18:41:18,064][DEBUG][discovery.ec2 ] [Misfit] using dynamic discovery nodes []
[2015-08-19 18:41:18,067][DEBUG][discovery.ec2 ] [Misfit] filtered ping responses: (filter_client[true], filter_data[false]) {none}
[2015-08-19 18:41:18,074][INFO ][cluster.service ] [Misfit] new_master [Misfit][W24WvY0yQyew0khFFDrQVA][ip-10-0-0-29.eu-west-1.compute.internal][inet[/10.0.0.29:9300]], reason: zen-disco-join (elected_as_master)
[2015-08-19 18:41:18,101][INFO ][http ] [Misfit] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.0.0.29:9200]}
[2015-08-19 18:41:18,101][INFO ][node ] [Misfit] started
[2015-08-19 18:41:18,102][INFO ][gateway ] [Misfit] recovered [0] indices into cluster_state
Cluster is not formed,rather two separate masters.
I have verified that IAM role on machine works. But is it possible to verify what entries were returned to ec2 node? Any hint what I've overlooked?
Thx
