1

I'm trying to run Apache Druid on M1 Mac with an ARM 64 processor as docker containers. This works fine as it runs via emulation. But I've encountered two issues with it.

  1. The druid containers (specifically broker) keep crashing with an exit code of 137. I was able to make it work by increasing the docker memory allocation to 8 GB, but this is not ideal.
  2. The entire druid setup is very slow. It takes a long time for the entire cluster to spin up and even once it is up, it is slow during ingestion and querying.

I'm wondering if there is a docker image built for ARM64 configuration or dockerfile/instructions to build it from source.

f_puras
  • 2,521
  • 4
  • 33
  • 38
Janson
  • 93
  • 6

2 Answers2

0

Regarding #1 - Yes, I've hit this as well. The default configuration of the services requires about 7GB of memory to run as a cluster of containers.

Regarding #2 - Druid is designed to run on a cluster. Running on a single computer will allow you to test functionality, but it won't be indicative of the performance you can expect when running on a cluster. If you are just testing functionality, I would suggest a single server execution. There are configurations that use very little resources. Look at the nano and micro quickstart options here: https://druid.apache.org/docs/latest/operations/single-server.html

Here are the instructions to build the docker image: https://github.com/apache/druid/tree/master/distribution/docker

Dharman
  • 30,962
  • 25
  • 85
  • 135
Sergio Ferragut
  • 161
  • 5
  • Thanks. Yes, I understand it is designed to run on a cluster and single computer setup is meant only for testing. The same setup works flawlessly with amd64 configuration in intel macs and windows. The slowness is probably due to emulation. I'll try to build it natively for arm and try it. Hope the above instructions remain same for arm builds as well. – Janson Nov 04 '21 at 12:06
0

Might be too late for this, but I have tried to build the druid image for arm64 and amd64 archs.

https://github.com/arm64-compat/apache-druid

These are not production ready images, only helpful for local development on M1 Chip Macbook. Even I am testing these images are the arm64 image is performing far better than qemu emulated images.

I am using druid-operator to deploy a tiny cluster using CRD on my local setup.

anuragagarwal561994
  • 56
  • 1
  • 5