How can I use an elasticsearch add-on container/service with ddev?

Question

How can I set up a service/container to provide elasticsearch with ddev? I have tried some experiments from https://ddev.readthedocs.io/en/latest/users/extend/additional-services/ but don't have enough docker-compose knowhow to do one for elasticsearch.

rfay · Answer 1 · 2022-04-01T19:08:57.210

1

Edit 2022-03: There is now an official elasticsearch ddev-get add-on for ddev v1.19+, ddev get drud/ddev-elasticsearch, see https://github.com/drud/ddev-elasticsearch.

@thursdaybw provided this recipe in https://github.com/drud/ddev/pull/1320, but it never gained traction and nobody reviewed it, so it's being moved here to percolate and incubate in the community. Please provide your suggestions if you use it.

Edit 2019-09-30: There is now an Elasticsearch example in ddev-contrib at https://github.com/drud/ddev-contrib/tree/master/docker-compose-services/elasticsearch

Basic information (and reviewed examples) for setting up additional services is at https://ddev.readthedocs.io/en/latest/users/extend/additional-services/

version: '3.6'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.5.1
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - VIRTUAL_HOST=$DDEV_HOSTNAME # This defines the host name the service should be accessible from. This will be sitename.ddev.local
      - HTTP_EXPOSE=9200
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata1:/usr/share/elasticsearch/data
    ports:
      - 9200
    labels:
    # These labels ensure this service is discoverable by ddev
      com.ddev.site-name: ${DDEV_SITENAME}
      com.ddev.approot: $DDEV_APPROOT

volumes:
  esdata1:
    driver: local

edited Apr 01 '22 at 19:08

answered Feb 07 '19 at 14:39

rfay

9,963
1
47
89

1

Ran into the "max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]" error. Added `- discovery.type=single-node` under `environment:` to get it working. From what I read this disables bootstrap checks which should be fine for development. – Madis Jun 05 '19 at 14:41
Is the connect hostname supposed to be "http://ddev-PROJECTNAME-elasticsearch:9200"? I'm trying to get it working with the elasticsearch_connector module but it's failing to connect. – Damien McKenna Jun 25 '19 at 10:15
That `container_name` isn't useful; it's already the default name. The hostname inside the project will be "elasticsearch", the service name. I'll edit. But to test, inside the web container, use `telnet elasticsearch 9200` to see if you can connect. I brought this up and it worked fine to connect. The "expose" there may be unnecessary, that's to expose it on the router, but unless you're going to hit the web interface from the host, that wouldn't seem useful. – rfay Jun 25 '19 at 14:13
I tried this in `.ddev/docker-compose.elastic.yaml`. The container starts (`Creating ddev-foobar_elasticsearch_1 ... done`), but I can't reach it from within my web container. I tried `ddev ssh`, then `telnet elasticsearch 9200` - `Name or service not known`. The host is not available. I used elasticsearch 7.2.0 – Mateng Aug 01 '19 at 13:31
@Mateng the "elasticsearch" name is not exposed inside the container I don't think. You'd want to `telnet ddev-_elasticsearch_1 9200` inside the web container. However... you might want to go back to the elasticsearch version in this example (which seems to work). 7.2.0 shows `ERROR: [1] bootstrap checks failed [1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured` on `ddev logs -s elasticsearch` – rfay Aug 02 '19 at 14:17

score 0 · Answer 2 · answered Jul 12 '19 at 15:01

For starting a single node, the given example hasn't helped me out. Without providing further error messages, the container was stopped again. Using the following configuration, I was able to start just one ES node and not as cluster (as given in the previous answer):

version: '3.6'
services:
  elasticsearch:
    container_name: ddev-${DDEV_SITENAME}-elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:6.5.1
    environment:
      - node.name=${DDEV_SITENAME}-es01
      - discovery.type=single-node
      - cluster.name=docker-${DDEV_SITENAME}-es-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    labels:
      com.ddev.site-name: ${DDEV_SITENAME}
      com.ddev.platform: ddev
      com.ddev.app-type: elasticsearch
      com.ddev.approot: $DDEV_APPROOT
  web:
    links:
      - elasticsearch:elasticsearch

volumes:
  esdata01:
    driver: local
    name: "${DDEV_SITENAME}-es"

Additionally, using this configuration, you could directly access the node using the host name elasticsearch from within another container.

How can I use an elasticsearch add-on container/service with ddev?

2 Answers2