awscli version 2 on alpine linux

Question

I was trying to put awscli_v2 into an alpine-based docker container and see that it fails with the following error message:

/aws/install: line 78: /aws/dist/aws: not found

Considering that the file itself is there and can be listed with ls, I would guess that some libraries that the executable ./aws/dist/aws relies upon are not present on alpine. Does someone know which libraries that might be?

Answer 1

Actually with a bit a effort it is possible to run AWS CLI v2 on Alpine:

ARG ALPINE_VERSION=3.17

FROM alpine:${ALPINE_VERSION}

ARG GLIBC_VERSION=2.35-r0
ARG AWSCLI_VERSION=2.11.11

# install glibc compatibility for alpine
RUN apk --no-cache add \
        binutils \
        curl \
    && curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-${GLIBC_VERSION}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-bin-${GLIBC_VERSION}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-i18n-${GLIBC_VERSION}.apk \
    && apk add --no-cache --force-overwrite \
        glibc-${GLIBC_VERSION}.apk \
        glibc-bin-${GLIBC_VERSION}.apk \
        glibc-i18n-${GLIBC_VERSION}.apk \
    && /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8 \
    && ln -sf /usr/glibc-compat/lib/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 \
    && curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64-${AWSCLI_VERSION}.zip -o awscliv2.zip \
    && unzip awscliv2.zip \
    && aws/install \
    && rm -rf \
        awscliv2.zip \
        aws \
        /usr/local/aws-cli/v2/current/dist/aws_completer \
        /usr/local/aws-cli/v2/current/dist/awscli/data/ac.index \
        /usr/local/aws-cli/v2/current/dist/awscli/examples \
        glibc-*.apk \
    && find /usr/local/aws-cli/v2/current/dist/awscli/botocore/data -name examples-1.json -delete \
    && apk --no-cache del \
        binutils \
        curl \
    && rm -rf /var/cache/apk/*

The above Dockerfile will install the 'glibc' package for Alpine, so that the AWS CLI will be able to find the required shared libraries. The Dockerfile also removes some stuff we probably don't need, such as auto-complete and examples. If you need some other specific packages you can of course add them to the Dockerfile.

Update 2022-08-01

AWS has improved the build process, so now it's also possible to build from git with MUSL glibc (kudos to @Julian Bueno):

ARG ALPINE_VERSION=3.17
FROM python:3.10-alpine${ALPINE_VERSION} as builder

ARG AWS_CLI_VERSION=2.11.11
RUN apk add --no-cache git unzip groff build-base libffi-dev cmake
RUN git clone --single-branch --depth 1 -b ${AWS_CLI_VERSION} https://github.com/aws/aws-cli.git

WORKDIR aws-cli
RUN ./configure --with-install-type=portable-exe --with-download-deps
RUN make
RUN make install

# reduce image size: remove autocomplete and examples
RUN rm -rf \
    /usr/local/lib/aws-cli/aws_completer \
    /usr/local/lib/aws-cli/awscli/data/ac.index \
    /usr/local/lib/aws-cli/awscli/examples
RUN find /usr/local/lib/aws-cli/awscli/data -name completions-1*.json -delete
RUN find /usr/local/lib/aws-cli/awscli/botocore/data -name examples-1.json -delete
RUN (cd /usr/local/lib/aws-cli; for a in *.so*; do test -f /lib/$a && rm $a; done)

# build the final image
FROM alpine:${ALPINE_VERSION}
COPY --from=builder /usr/local/lib/aws-cli/ /usr/local/lib/aws-cli/
RUN ln -s /usr/local/lib/aws-cli/aws /usr/local/bin/aws

The above reduces the image size even more (from 175MB to 95MB), since we no longer need the glibc-compat hack. Hopefully AWS will also distribute the Alpine binaries, so we do not have to build it ourselves.

Answer 2

AWS CLI version 2 is compiled against glibc, the GNU Project's implementation of the C standard library. Most common Linux distributions use glibc, but Alpine Linux is instead based on musl libc.

Because binaries compiled against glibc aren't compatible with musl, AWS CLI version 2 doesn't run on Alpine Linux.

The best way to run AWS CLI version 2 on Alpine Linux would be for Amazon to provide one of the following:

• Binaries compiled against musl
• A source distribution, which can be compiled on Alpine Linux
• An official AWS CLI Docker image based on Alpine Linux

Update

AWS now provides an example Dockerfile to build AWS CLI on Alpine Linux: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-source-install.html#source-getting-started-install-workflows-alpine

Dockerfile:

FROM python:3.8-alpine AS builder

ENV AWSCLI_VERSION=2.2.1

RUN apk add --no-cache \
    curl \
    make \
    cmake \
    gcc \
    g++ \
    libc-dev \
    libffi-dev \
    openssl-dev \
    && curl https://awscli.amazonaws.com/awscli-${AWSCLI_VERSION}.tar.gz | tar -xz \
    && cd awscli-${AWSCLI_VERSION} \
    && ./configure --prefix=/opt/aws-cli/ --with-download-deps \
    && make \
    && make install

FROM python:3.8-alpine

RUN apk --no-cache add groff

COPY --from=builder /opt/aws-cli/ /opt/aws-cli/

ENTRYPOINT ["/opt/aws-cli/bin/aws"]

Build and run:

$ docker build --tag awscli-alpine .
$ docker run --rm -it awscli-alpine --version
aws-cli/2.2.1 Python/3.8.11 Linux/5.10.25-linuxkit source-sandbox/x86_64.alpine.3 prompt/off

Update 2

Alpine Linux 3.18 provides AWS CLI in the aws-cli package. To install it, run:

$ apk update
$ apk add aws-cli

Answer 3

Can also use https://git.adelielinux.org/adelie/gcompat

apk add gcompat

source: https://wiki.alpinelinux.org/wiki/Running_glibc_programs

Edit sample:

ENV AWS_CLI_VER=2.0.30

RUN apk update && apk add --no-cache curl gcompat zip &&  \
    curl -s https://awscli.amazonaws.com/awscli-exe-linux-x86_64-${AWS_CLI_VER}.zip -o awscliv2.zip && \
    unzip awscliv2.zip && ./aws/install

Answer 4

From this comment and this . I have create an updated version at the time of writing. This version does not need gcompat and takes 6 minutes to build

FROM python:3.10.5-alpine3.16 as builder

ENV AWS_CLI_VERSION=2.7.15

RUN set -ex; \
apk add --no-cache \
git unzip groff build-base libffi-dev cmake

RUN set -eux; \
git clone --single-branch --depth 1 -b ${AWS_CLI_VERSION} https://github.com/aws/aws-cli.git; \
cd /aws-cli; \
sed -i'' 's/PyInstaller.*/PyInstaller==5.2/g' requirements-build.txt; \
python -m venv venv; \
. venv/bin/activate; \
./scripts/installers/make-exe

RUN set -ex; \
unzip /aws-cli/dist/awscli-exe.zip; \
./aws/install --bin-dir /aws-cli-bin; \
/aws-cli-bin/aws --version

FROM python:3.10.5-alpine3.16

COPY --from=builder /usr/local/aws-cli/ /usr/local/aws-cli/
COPY --from=builder /aws-cli-bin/ /usr/local/bin/  

#Check version
RUN  aws --version 

ENTRYPOINT ["/bin/ash", "-c"]

Answer 5

To make it work on gitlab.com with docker 19.x with current aws cli version 2.11.4 and current alpine 3.17.2 I had to make a few adjustments to the accepted answer from blagerweij.

• Stick with the gilibc variant because the musl variant apparently won't work with docker 19: https://github.com/aws/aws-codebuild-docker-images/issues/467
• Add --force-overwrite to apk add in the gilibc because of: https://github.com/sgerrand/alpine-pkg-glibc/issues/190
• Use glibc version 2.33 because of: https://github.com/sgerrand/alpine-pkg-glibc/issues/167

# install awscli v2 and other basic utilities that are needed by all docker images
# see: https://stackoverflow.com/a/75781104/1245992
ARG ALPINE_VERSION=3.17.2

FROM alpine:${ALPINE_VERSION}

ARG GLIBC_VERSION=2.33-r0
ARG AWSCLI_VERSION=2.11.4

# install glibc compatibility for alpine
RUN apk --no-cache add \
        binutils \
        curl \
    && curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-${GLIBC_VERSION}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-bin-${GLIBC_VERSION}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-i18n-${GLIBC_VERSION}.apk \
    && apk add --no-cache \
        --force-overwrite \
        glibc-${GLIBC_VERSION}.apk \
        glibc-bin-${GLIBC_VERSION}.apk \
        glibc-i18n-${GLIBC_VERSION}.apk \
    && /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8 \
    && ln -sf /usr/glibc-compat/lib/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 \
    && curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64-${AWSCLI_VERSION}.zip -o awscliv2.zip \
    && unzip awscliv2.zip \
    && aws/install \
    && rm -rf \
        awscliv2.zip \
        aws \
        /usr/local/aws-cli/v2/current/dist/aws_completer \
        /usr/local/aws-cli/v2/current/dist/awscli/data/ac.index \
        /usr/local/aws-cli/v2/current/dist/awscli/examples \
        glibc-*.apk \
    && find /usr/local/aws-cli/v2/current/dist/awscli/botocore/data -name examples-1.json -delete \
    && apk --no-cache del \
        binutils \
        curl \
    && rm -rf /var/cache/apk/*

ENV LANG='C.UTF-8'
```

Answer 6

awscli-v2 might be coming to alpine as a package.

Someone might be working on it in the community repository: https://pkgs.alpinelinux.org/packages?name=aws-cli&branch=edge&repo=community

Currently seems to be broken but if you want to try it out you can install with: apk add aws-cli --repository=https://dl-cdn.alpinelinux.org/alpine/edge/community

Answer 7

Rather than trying to fix the MUSL/LIBC gaps in Apline or to go the hard path with Python, I have found this image rather good and up to date. It's got the compiled version of AWS cli.

It is about 105 MB in size after some cleanup (you don t need docs, do you?)

In case you really want to start from Alpine (with no other party involved), then you can check in the Alpine package management that edge tag/branch supports AWS cli v2 v2.11.0-r0 as of 2023-03-05 (Python-interpreted version).

So, you Dockerfile can be as simple as:

FROM alpine:edge
RUN apk add --no-cache aws-cli
RUN rm -fr \
  /usr/bin/aws_completer \
  /usr/lib/python*/site-packages/awscli/data/ac.index \
  /usr/lib/python*/site-packages/awscli/examples
RUN find /usr/lib/python* -name completions-1*.json -delete

to get a 183MB image. Of course your mileage may vary with other versions.

Answer 8

Wanted to add this as a comment to the answer about the community repo, but I guess I don't have enough rep. I was able to get aws-cli v2 running on alpine just using apk repositories. These commands in your dockerfile should work to get the correct versions of Python, Pip, Aws-Cli and python dependencies.

RUN apk update --repository=https://dl-cdn.alpinelinux.org/alpine/edge/main/
RUN apk add python3 --repository=https://dl-cdn.alpinelinux.org/alpine/edge/main/
RUN apk update --repository=https://dl-cdn.alpinelinux.org/alpine/edge/community/
RUN apk add aws-cli py3-pip --repository=https://dl-cdn.alpinelinux.org/alpine/edge/community/
RUN pip install --ignore-installed wcwidth docutils cryptography urllib3 six