Monstache Configuration in Kubernetes Cluster

Question

I'm currently moving an application into a Kubernetes Cluster. Monstache is giving me headaches over and over. I have one StatefulSet for MongoDB and one Deployment for ElasticSearch and Monstache. For Mongo I have configured a Headless service and a LoadBalancer, and for ElasticSearch a LoadBalancer service only. Unfortunately monstache does not seem to be configured quiet correct - No DB entries shown in the frontend and the Monstache logs just spam "Ping Fail".

What am I missing? I'm assuming the monstache.config.toml to be the issue, maybe the URLS for elasticsearch and mongo?

monstache.config.toml

mapper-plugin-path = "/app/CreateIndexPlugin.so"

mongo-url = "mongodb://loadbalancer-mongo:27017"

elasticsearch-urls = ["http://loadbalancer-elasticsearch:9200"]

elasticsearch-max-conns = 10

resume = true

resume-name = "resume-entries"

namespace-regex = '^data\.entrys$|^data\.groups$'

direct-read-namespaces = ["data.entrys","data.groups"]
index-as-update = true

verbose = true

exit-after-direct-reads = false

[[mapping]]
namespace = "data.entrys"
index = "entries"

[[mapping]]
namespace = "data.groups"
index = "groups"

statefulset-mongo.yml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: statefulset-mongo
spec:
  replicas: 1
  serviceName: headless-mongo
  selector:
    matchLabels:
      app: mongo
  template:
    metadata:
      labels:
        app: mongo
    spec:
      containers:
        - name: mongo
          image: merkuri.azurecr.io/mongo:5.0.3
          imagePullPolicy: Always
          ports:
            - containerPort: 27017
          command:
            - "mongod"
            - "--replSet"
            - "rs0"
            - "--bind_ip_all"
          volumeMounts:
            - name: mongo-data
              mountPath: /data/db
  volumeClaimTemplates:
    - metadata:
        name: mongo-data
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi

deployment-elasticsearch-monstache.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-elasticsearch-monstache
spec:
  replicas: 1
  selector: 
    matchLabels:
      app: elasticsearch-monstache
  template:
    metadata:
      labels:
        app: elasticsearch-monstache
    spec:
      containers:
      - name: elasticsearch
        image: merkuri.azurecr.io/elasticsearch:7.16.2
        imagePullPolicy: Always
        ports:
        - containerPort: 9200
        env:
        - name: ES_JAVA_OPTS
          value: "-Xms512m -Xmx512m"
        - name: discovery.type
          value: "single-node"
          
      - name: monstache
        image: merkuri.azurecr.io/monstache:latest
        imagePullPolicy: Always
          
      imagePullSecrets:
      - name: acr-merkuri-secret
      restartPolicy: Always

headless-mongo.yml

apiVersion: v1
kind: Service
metadata:
  name: headless-mongo
spec:
  type: ClusterIP
  clusterIP: None
  selector:
    app: mongo
  ports:
    - name: mongo-port
      protocol: TCP
      port: 27017
      targetPort: 27017

loadbalancer-mongo.yml

apiVersion: v1
kind: Service
metadata:
  name: loadbalancer-mongo
spec:
  type: LoadBalancer
  selector:
    app: mongo
  ports:
  - name: mongo-port
    protocol: TCP
    port: 27017
    targetPort: 27017

loadbalancer-elasticsearch.yml

apiVersion: v1
kind: Service
metadata:
  name: loadbalancer-elasticsearch
spec:
  type: LoadBalancer
  selector:
    app: elasticsearch-monstache
  ports:
  - name: elasticsearch-port
    protocol: TCP
    port: 9200
    targetPort: 9200

I tried different URLS, eg. with the public IP adresses exposed by the LoadBalancer services but that did not work either. I really feel like the URLS not not correct 100% but I did not find anything in the internet regarding that.

Monstache Configuration in Kubernetes Cluster

0 Answers0