How to install hdf5 on Docker image with Linux alpine 3.13

Question

I am building a Docker image with python 3.7.10 (Linux Alpine v3.13) but when building the image with docker build . the package hdf5 will fail during the installation. This is my Dockerfile:

FROM python:3.7.10-alpine3.13

RUN mkdir /app
WORKDIR /app
COPY requirements.txt /requirements.txt
ENV CRYPTOGRAPHY_DONT_BUILD_RUST=1
RUN apk add --update --no-cache --virtual .tmp gcc libc-dev linux-headers
RUN apk add --no-cache jpeg-dev zlib-dev mariadb-dev libffi-dev openblas-dev libgfortran lapack-dev build-base openssl-dev
RUN apk add --no-cache hdf5 hdf5-dev
RUN pip install -r /requirements.txt
RUN apk --no-cache del build-base

ENV PYTHONUNBUFFERED 1
COPY . /app/

CMD ["uwsgi", "my_app"]

requirements.txt file:

h5py==2.10.0
numpy==1.19.2

I have tried with and without the --no-binary flag but still no luck. Was somebody able to install that library on this Alpine version?

Error logs:

    17858 |   __pyx_t_1 = H5Ovisit_by_name(__pyx_v_loc_id, __pyx_v_obj_name, __pyx_v_idx_type, __pyx_v_order, __pyx_v_op, __pyx_v_op_data, __pyx_v_lapl_id); if (unlikely(PyErr_Occurred())) __PYX_ERR(0, 1641, __pyx_L1_error)
          |               ^~~~~~~~~~~~~~~~
    In file included from /usr/include/H5Apublic.h:22,
                     from /usr/include/hdf5.h:23,
                     from /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/api_compat.h:27,
                     from /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/defs.c:654:
    /usr/include/H5Opublic.h:213:15: note: declared here
      213 | H5_DLL herr_t H5Ovisit_by_name3(hid_t loc_id, const char *obj_name,
          |               ^~~~~~~~~~~~~~~~~
    /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/defs.c: In function '__pyx_f_4h5py_4defs_H5Pget_driver_info':
    /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/defs.c:21768:13: warning: assignment discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]
    21768 |   __pyx_t_1 = H5Pget_driver_info(__pyx_v_plist_id); if (unlikely(PyErr_Occurred())) __PYX_ERR(0, 2016, __pyx_L1_error)
          |             ^
    In file included from /usr/include/H5public.h:32,
                     from /usr/include/hdf5.h:22,
                     from /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/api_compat.h:27,
                     from /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/defs.c:654:
    /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/defs.c: In function '__pyx_f_4h5py_4defs_H5Sencode':
    /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/defs.c:34528:15: error: too few arguments to function 'H5Sencode2'
    34528 |   __pyx_t_1 = H5Sencode(__pyx_v_obj_id, __pyx_v_buf, __pyx_v_nalloc); if (unlikely(PyErr_Occurred())) __PYX_ERR(0, 3303, __pyx_L1_error)
          |               ^~~~~~~~~
    In file included from /usr/include/hdf5.h:38,
                     from /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/api_compat.h:27,
                     from /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/defs.c:654:
    /usr/include/H5Spublic.h:129:15: note: declared here
      129 | H5_DLL herr_t H5Sencode2(hid_t obj_id, void *buf, size_t *nalloc, hid_t fapl);
          |               ^~~~~~~~~~
    In file included from /usr/local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:21,
                     from /usr/local/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                     from /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/api_compat.h:26,
                     from /tmp/pip-install-fkianwj6/h5py_dfa9366c1fdb47e98e9e2e8de85ac21e/h5py/defs.c:654:
    At top level:
    /usr/local/lib/python3.7/site-packages/numpy/core/include/numpy/__multiarray_api.h:1464:1: warning: '_import_array' defined but not used [-Wunused-function]
     1464 | _import_array(void)
          | ^~~~~~~~~~~~~
    error: command 'gcc' failed with exit status 1
    ----------------------------------------

Works fine up to the stage to install `requirements.txt` both on `X86` and `ARM64` platforms. It looks like the problem of versions compatibility, but without your `requirements.txt` is not reproducible. — jordanvrtanoski, Mar 19 '21 at 12:21
@jordanvrtanoski I have updated the question, adding the `requirements.txt` — Ander, Mar 19 '21 at 12:52
For me it works without issues, using the provided requirements.txt as well. Image build log: https://pastebin.com/CXXFaqN5 — valiano, Mar 19 '21 at 15:42
apologies to both jordanvrtanoski & @valiano. I have copied the wrong setup, it failed for me when using **h5py==2.10.0** but now with **h5py==3.2.1** works perfectly. — Ander, Mar 19 '21 at 16:10

score 4 · Accepted Answer · answered Mar 19 '21 at 16:16

Seems like I have used a wrong and old h5py version (h5py==2.10.0). The following setup worked just fine when updating to h5py==3.2.1 within the requirements.txt.

FROM python:3.7.10-alpine3.13

RUN mkdir /app
WORKDIR /app
COPY requirements.txt /requirements.txt
ENV CRYPTOGRAPHY_DONT_BUILD_RUST=1
RUN apk add --update --no-cache --virtual .tmp gcc libc-dev linux-headers
RUN apk add --no-cache jpeg-dev zlib-dev mariadb-dev libffi-dev openblas-dev libgfortran lapack-dev build-base openssl-dev
RUN apk add --no-cache hdf5-dev
RUN pip install -r /requirements.txt
RUN apk --no-cache del build-base

ENV PYTHONUNBUFFERED 1
COPY . /app/

CMD ["uwsgi", "my_app"]

requirements.txt file:

h5py==3.2.1
numpy==1.20.1

Thanks to @jordanvrtanoski and @valiano for the feedback.

How to install hdf5 on Docker image with Linux alpine 3.13

1 Answers1