Kibana 5.5.1 behind a nginx 1.13 proxy (dockerized)

Question

Goal:

I want to run the elk stack in a docker container. To be able to access the ELK Stack over a nginx proxy to bypass the individual ports for the services.

The Kibana service (default port 5601)

http://<server>.com:5601

should be reachable over the following address:

http://<server>.com/kibana

Problem:

The problem is, that it is not possible to reach the kibana site after I add the server.basePath setting to the config. I only can bring up the service if I add every base api call of Kibana to the nginx config (/api, /ui, ...).

Config:

The config for Kibana:

/opt/kibana/config/kibana.yml

Has the following entries:

server.host: "0.0.0.0"
server.basePath: "/kibana"

everything else is default

Doku server.basePath

# Enables you to specify a path to mount Kibana at if you are running behind a proxy. This only affects
# the URLs generated by Kibana, your proxy is expected to remove the basePath value before forwarding requests
# to Kibana. This setting cannot end in a slash.

The nginx config:

location /kibana/ {
  rewrite ^/kibana(/.*)$ $1 break;
  proxy_pass http://<server>.com:5601/;
}

I use the sebp/elk:551 docker image and the following docker-compose file:

version: '2'
services:
  elk:
    image: sebp/elk:551
    container_name: "elk"
    volumes:
      - /etc/kibana/config/kibana.yml:/opt/kibana/config/kibana.yml
    ports:
      - "5601:5601"
      - "9200:9200"
      - "5044:5044"
    environment:
      SERVICE_5601_NAME: "kibana"
      SERVICE_9200_NAME: "elasticsearch"
      SERVICE_5044_NAME: "logstash"
    restart: always

What I have tried:

I have tried the same setup with Kibana 4.6.1 and it worked perfectly as expected.

Versions that I have tested and do not work: 5.4.3, 5.1.2, 5.0.2

What I dont want:

I dont want to add every subdirectory of Kibana like /api, /ui, /app/kibana, ... to add to the proxy config.

Is there an other solution or version?

Edit1: @whites11: The browser return the 502 Bad Gateway site from nginx. Browser infos:

General

Request URL:http://<server-name>.com/kibana/
Request Method:GET
Status Code:502 Bad Gateway
Remote Address:<server-ip>:80
Referrer Policy:no-referrer-when-downgrade

Response Headers

Connection:keep-alive
Content-Length:575
Content-Type:text/html
Date:Thu, 24 Aug 2017 13:54:49 GMT
Server:nginx/1.13.3

Request Headers

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding:gzip, deflate
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Host:<server-name>.com
Upgrade-Insecure-Requests:1

Log from nginx

34#34: *8 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: <IP>, server: , request: "GET /kibana/ HTTP/1.1", upstream: "http://<server-ip>:5601/", host: "<server-name>.com"

Answer 1

I don't have exactly the same deployment as you do, but I am using a dockerized kibana.

First of all, you need the following in your nginx settings:

    location /kibana/ {
      proxy_pass http://kibana_server:5601/;
    }

Change the values according to your environment, but the final slashes are critical! Don't remove them! They ensure that the rewriting is done as expected by Kibana --i.e., the kibana is removed from the URL in the petition that goes to the kibana server.

Then maintain the following:

server.basePath: /kibana

in your kibana settings. That ensures that the documents provided by kibana (links and urls) have the prefix /kibana.

It works for me.

Answer 2

I spent couple hours to figure this out. Basically if you are running kibana behind nginx you should set the environment variable SERVER_BASEPATH=/yourpath

nginx config:

upstream kibana {
    server kibana:5601;
}

server {
    listen 80;

    location /kibana/ {
        rewrite /kibana/(.*) /$1 break;
        proxy_pass http://kibana/;
    }
}

Kibana setup in docker-compose:

kibana:
image: kibana:7.7.1
ports:
  - "5601:5601"    # Important: In a production environment remove external port
environment:
  - ELASTICSEARCH_URL=http://elasticsearch:9200
  - SERVER_BASEPATH=/kibana
depends_on:
  - elasticsearch
networks:
  - backend

Note: elasticsearch is another service in docker-compose file.

Check implementation for more details: https://github.com/Jamaxack/Kangaroo

Answer 3

First there is no point in touching Kibana as we are anyways going to use Nginx as a reverse proxy. So ditch your server.basePath

Next change your nginx config yo below

location /kibana/ {
  proxy_pass http://<server>.com:5601/;
}

What this would mean is when you access http://<nginxhost>:<port>/kibana/xyz/abc. It would be equivalent to use http://<server>.com:5601/xyz/abc. Removes any complexity from your system

Edit-1

For those who think this doesn't work, that is not the case. Here is a sample test case I had set before posting this answer.

events {
    worker_connections  1024;
}
http {
server {
   listen 80;

   location /test1 {
     proxy_pass http://127.0.0.1:81;
   }

   location /test2 {
     proxy_pass http://127.0.0.1:81/;
   }

   location /test3/ {
     proxy_pass http://127.0.0.1:81;
   }

   location /test4/ {
     proxy_pass http://127.0.0.1:81/;
   }

}

server {
   listen 81;

   location / {
     echo "$request_uri";
   }
}
}

Now the results explains the difference between all 4 location blocks

$ curl http://192.168.33.100/test1/abc/test
/test1/abc/test

$ curl http://192.168.33.100/test2/abc/test
//abc/test

$ curl http://192.168.33.100/test3/abc/test
/test3/abc/test

$ curl http://192.168.33.100/test4/abc/test
/abc/test