Logstash warning Opensearch unreachable

Question

I have deployed Logstash 8.6.1 with the logstash-output-opensearch plugin. This is the config related to pushing the logs to OpenSearch:

output {
    if [type]=="logback" {
         opensearch {
             hosts => [ "https://search-x.ap-southeast-1.es.amazonaws.com" ]
             auth_type => {
                type => 'basic'
                user => 'x'
                password => 'y'
             }
             index => "logback-%{+YYYY}"
             ecs_compatibility => disabled  
        }
    }
}

When running logstash, I noticed that the log prints:

[2023-03-01T07:02:47,126][WARN ][logstash.outputs.opensearch][main] Attempted to resurrect connection to dead OpenSearch instance, but got an error {:url=>"https://search-x.ap-southeast-1.es.amazonaws.com:9200/", :exception=>LogStash::Outputs::OpenSearch::HttpClient::Pool::HostUnreachableError, :message=>"OpenSearch Unreachable: [https://search-x.ap-southeast-1.es.amazonaws.com:9200/][Manticore::ConnectTimeout] Connect to search-x.ap-southeast-1.es.amazonaws.com:9200 [search-x.ap-southeast-1.es.amazonaws.com/x.x.x.x, search-x.ap-southeast-1.es.amazonaws.com/x.x.x.x] failed: Connect timed out"}

The Opensearch network access is on Public, its version is 1.1, is this warning related to firewall settings?

When I tried curl https://search-x.ap-southeast-1.es.amazonaws.com with the required Basic auth header, it does return information:

{
    "name": "e",
    "cluster_name": "x",
    "cluster_uuid": "y",
    "version": {
        "number": "7.10.2",
        "build_type": "tar",
        "build_hash": "unknown",
        "build_date": "2022-10-20T07:32:13.999133Z",
        "build_snapshot": false,
        "lucene_version": "8.9.0",
        "minimum_wire_compatibility_version": "6.8.0",
        "minimum_index_compatibility_version": "6.0.0-beta1"
    },
    "tagline": "The OpenSearch Project: https://opensearch.org/"
}

I can also access the OpenSearch dashboard.

score 1 · Accepted Answer · answered Mar 01 '23 at 07:28

1

if it's Amazon OpenSearch service (managed service), the default port is not 9200, but 443 or 80 (I'm not sure you can change this configuration). Please also make sure that you don't have a IAM policy blocking your requests

answered Mar 01 '23 at 07:28

glenacota

2,314
1
11
18

Hm I did not specify the port in the hosts though. – Eugene Mar 02 '23 at 04:07
9200 it's the default when you don't specify the port in the hosts property: https://github.com/opensearch-project/logstash-output-opensearch/blob/main/lib/logstash/outputs/opensearch/http_client.rb#L285 – glenacota Mar 02 '23 at 07:03
thanks! I needed to specifically mention port 443 in the hosts line. – Eugene Mar 08 '23 at 05:11

Logstash warning Opensearch unreachable

1 Answers1