-2

How can I add fluent-bit to an image that builds on docker:latest?

I have tried this:

FROM docker:latest

RUN apk add python3 py-pip python3-dev libffi-dev openssl-dev gcc libc-dev make curl libc6-compat
RUN apk add --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing/

ENTRYPOINT ["/fluent-bit/bin/fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.conf"]

But it does not find /fluent-bit/bin/fluent-bit. I do need Python, Docker, and all the other specified dependencies.

naraghi
  • 430
  • 1
  • 6
  • 18
  • I'd install it in its own container. If it does require a Python interpreter, I'd use the `python` image with the `docker-py` library, and not the `docker` image. (I'm not sure when I'd use the `docker` image at all – if I was able to run the `docker` image, I'd be able to launch a container and otherwise interact with Docker without needing the CLI in a container.) – David Maze Mar 05 '23 at 22:28
  • Do you mean you'd use the `fluent/fluent-bit` image and install python and docker within, or the `python` image and install fluent-bit within? – naraghi Mar 06 '23 at 12:42
  • I'd use the unmodified `fluent/fluent-bit` image. My Python application would be in a separate container with a separate image. I wouldn't run the `docker` image at all. – David Maze Mar 06 '23 at 14:44
  • Problem is, I need to run a Python script that creates symlinks to the logs of certain docker containers so that only those can be uploaded. I guess I can use a volume that is shared between the Python container and the fluent-bit container. Or what would be your suggestion? – naraghi Mar 06 '23 at 15:45
  • I checked the official [`Dockerfile`](https://hub.docker.com/r/fluent/fluent-bit/dockerfile). You just didn't write Dockerfile correctly. You need to change the Debian commands and packages in the official and merge your code with them. – Constantin Hong Mar 06 '23 at 16:21
  • @ConstantinHong should I just copy all the commands from the Dockerfile you pointed at into my own Dockerfile? – naraghi Mar 06 '23 at 17:30
  • 1
    @naraghi Oh, I didn't clarify well. I'm sorry. The alpine you choose is not the Debian in the Dockerfile I suggested. They have incompatibility in terms of the naming of the packages(such a 'dnf vs apt-get', 'openssl-dev vs openssl-devel'). You need to change it. After that it seems to work. – Constantin Hong Mar 06 '23 at 17:51

2 Answers2

1

I ended up installing docker and fluent-bit in a Python Debian image, and it works. Dockerfile:

FROM python:3.11-slim-buster

# install dependencies
RUN apt-get update
RUN apt-get upgrade
RUN apt-get install -y curl bash gpg

# install fluent-bit
RUN curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh

ENV PATH="/opt/fluent-bit/bin:${PATH}"

# install concurrently
RUN curl -fsSL https://deb.nodesource.com/setup_16.x | bash -
RUN apt install nodejs
RUN npm install --global concurrently

# install docker
RUN apt update
RUN apt install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
RUN add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
RUN apt update
RUN apt-cache policy docker-ce
RUN apt install -y docker-ce

# set up custom files
RUN mkdir -p /app
RUN mkdir -p /logs

COPY uploader/sync.py /app/sync.py
COPY uploader/entrypoint.sh /app/entrypoint.sh

RUN chmod +x /app/entrypoint.sh

ENTRYPOINT /app/entrypoint.sh

entrypoint.sh:

command1="python3 /app/sync.py --compose_project_name $COMPOSE_PROJECT_NAME"
command2="/opt/fluent-bit/bin/fluent-bit -c /fluent-bit/etc/fluent-bit.conf"
concurrently "$command1" "$command2"
naraghi
  • 430
  • 1
  • 6
  • 18
  • 1
    Thanks for posting this dockerfile, really useful. – Cristi Iaroi Mar 09 '23 at 15:28
  • Have you actually got it working? I get the fluent-bit process terminated with the following error as soon as the tailed log get any data [2023/03/09 07:33:18] [engine] caught signal (SIGSEGV) #0 0x7fe98542e846 in ???() at ???:0 #8 0x56169bf68e01 in flb_engine_start() at src/flb_engine.c:853 #9 0x56169bf0fd09 in flb_lib_worker() at src/flb_lib.c:629 #10 0x7fe985bafea6 in ???() at ???:0 #11 0x7fe98548edee in ???() at ???:0 #12 0xffffffffffffffff in ???() at ???:0 Aborted – Cristi Iaroi Mar 09 '23 at 15:40
  • I got it working @Christi Iaroi – naraghi Mar 14 '23 at 16:04
-1

There is no official Fluent-bit docker on Alpine Linux.

Please, Check the official manual.

Alpine Linux uses Musl C library instead of Glibc. Musl is not fully compatible with Glibc which generated many issues when used with Fluent Bit

Therefore, you need to change your code's package name to Debian's package. Also, the link shows some packages of yours won't work.

So you need to find Debian equivalents. For example, openssl-dev's Debian equivalent is openssl-devel.

After that, please, merge your code with the official fluent-bit dockerfile.

Constantin Hong
  • 701
  • 1
  • 2
  • 16