Running Elasticsearch 8 on concourse container

Question

I'm using Concourse for building my java package.

In order to run integration tests of that package, I need a local instance of elasticsearch present.

Prior to ES version 8, all I was doing was installing ES in Docker image that I would then use as Concourse task's image resource to build my java package in:

FROM openjdk:11-jdk-slim-stretch
RUN apt-get update && apt-get install -y procps
ADD "https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.9.3-amd64.deb" /es.deb
RUN dpkg -i es.deb
RUN rm es.deb

Later I would just start it right before building with: /etc/init.d/elasticsearch start

Problems started when upgrading ES to version 8. That init.d file does not seem to exist anymore. Some of the advices I found suggest running ES as a container, so running ES container inside of the concourse container which seems a bit too complex for my use case.

If you had similar problems in your projects, how did you solve them?

Answer 1

This is what I would do:

1. Build your docker image off of an official Elastic docker image, e.g.:

FROM elasticsearch:8.2.2
USER root
RUN apt update && apt install -y sudo

2. Start Elastic within your task. Suppose the image got pushed to oozie/elastic on docker. Then the following pipeline job should succeed:

jobs:
  - name: run-elastic
    plan:
    - task: integtest
      config:
        platform: linux
        image_resource:
          type: docker-image
          source:
            repository: oozie/elastic
        run:
          path: /bin/bash
          args:
          - -c
          - |
            (sudo -u elasticsearch /usr/share/elasticsearch/bin/elasticsearch -Expack.security.enabled=false -E discovery.type=single-node > elastic.log) &
            while ! curl http://localhost:9200; do sleep 10; done

It should result in the following task run: