fluentd cannot connect to elasticsearch

Question

I run the stack EFK through docker-compose, the stack starts, kibana and elasticsearch go through the initial setup. fluentd throws the following error at startup:

"The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product."

kibana 8.2.0 elasticsearch 8.2.0

Maybe fluent can't log in over https, where to look for the problem?

Dockerfile fluentd:

FROM fluentd:latest

# Use root account to use apk
USER root

# below RUN includes plugin as examples elasticsearch is not required
# you may customize including plugins as you wish
RUN apk add --no-cache --update --virtual .build-deps \
        sudo build-base ruby-dev \
 && sudo gem install fluent-plugin-elasticsearch \
 && sudo gem sources --clear-all \
 && apk del .build-deps \
 && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

COPY conf/fluent.conf /fluentd/etc/
COPY entrypoint.sh /bin/

USER fluent

fluentd.conf

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match *.**>
  @type elasticsearch
  host elasticsearch
  port 9200
  user fluent
  password 6M9eXThhypVjV8h
  logstash_format true
  logstash_prefix fluentd
  logstash_dateformat %Y%m%d
</match>

the username and password for fluent are created in kibana

The initial configuration of kubana and elasticsearch has been carried out, I can open the ip address:5601 in the browser, and walk around the kibana web interface — Rostislav Udaltsov, May 26 '22 at 02:32

score 1 · Accepted Answer · answered May 27 '22 at 03:55

The problem was solved by downgrading the EFK version and changing the Dockerfile

Dockerfile

FROM fluent/fluentd:v1.12.0-debian-1.0
USER root
RUN gem uninstall -I elasticsearch && gem install elasticsearch -v 7.17.0
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.0.3"]
USER fluent

docker-compose.yml

## EFK Stack
  fluentd:
    build: ./fluentd
    volumes:
      - ./fluentd/conf/fluent.conf:/fluentd/etc/fluent.conf
    ports:
      - "24224:24224"
      - "24224:24224/udp"
    depends_on:
      - elasticsearch
      - kibana
    networks:
      - efk

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.13.1
    container_name: elasticsearch
    environment:
      - "discovery.type=single-node"
    expose:
      - "9200"
    ports:
      - "9200:9200"
    networks:
      - efk

  kibana:
    image: docker.elastic.co/kibana/kibana:7.13.1
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch
    networks:
      - efk

networks:
  efk:
    driver: bridge

fluent.conf

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match *.**>
  @type copy

  <store>
    @type elasticsearch
    host elasticsearch
    port 9200
    logstash_format true
    logstash_prefix fluentd
    logstash_dateformat %Y%m%d
    include_tag_key true
    type_name access_log
    tag_key @log_name
    flush_interval 1s
  </store>

  <store>
    @type stdout
  </store>
</match>

fluentd cannot connect to elasticsearch

1 Answers1