How can I correctly install & run pip in a wine-emulated python version (inside a docker container)?

Question

I wanted to do a MetaTrader5 environment with Python. While I could do it in my Windows machine, I'd like to have it as a standalone Docker container to, say, deploy in my (I have two other systems that I prefer for coding: mac and linux) machine or perhaps an AWS ECR.

For that purpose, I created this docker file:

FROM ubuntu:20.04

# In order to use the python version, we'll have to install
# wine emulator. We're installing both 32 and 64 bits API.
RUN dpkg --add-architecture i386
RUN apt update
ENV TZ=UTC
RUN apt-get install --assume-yes apt-utils
RUN DEBIAN_FRONTEND=noninteractive apt-get install --assume-yes tzdata
RUN apt-get install --assume-yes wine64 wine32

# The root wine prefix will be: /root/.wine.

# Now MetaTrader5 is being moved to the Program Files directory.
COPY ["MetaTrader5", "/root/.wine/drive_c/Program Files/MetaTrader5"]

# Also the Python interpreter is moved to that directory.
ADD ["python-3.9.0", "/root/.wine/drive_c/Program Files/python-3.9.0"]

# And the get-pip script is added to the standard interpreter.
ADD ["https://bootstrap.pypa.io/get-pip.py", "/root/.wine/drive_c/Program Files/python-3.9.0/get-pip.py"]

# The server script itself is then added as another application.
RUN ["mkdir", "-p", "/root/.wine/drive_c/Program Files/MetaTrader5 AI Server"]
COPY ["server.py", "/root/.wine/drive_c/Program Files/MetaTrader5 AI Server/server.py"]

############### 'Till this point, everything is fine.

# Now, pip will be installed for windows.
RUN ["wine", "/root/.wine/drive_c/Program Files/python-3.9.0/pythonw.exe", "/root/.wine/drive_c/Program Files/python-3.9.0/get-pip.py"]

# And then, metatrader will be installed, along with aiohttp
# library for websockets.
# RUN pip install MetaTrader5 aiohttp

# Choosing this tail command is just for debug
# purposes, as the true implementation will use
# the server.py file as entry point.
CMD ["tail", "-f", "/dev/null"]

The issue I'm having involves the last part: When get-pip.py is executed, the Scripts/ directory (/root/.wine/drive_c/Program Files/python-3.9.0/Scripts) is created with the pip.exe and easy_install.exe executables, but when trying to invoke either of them, an error is raised, as the pip python module does not seem to be installed.

This can be tested by running this dockerfile:

$ docker build . --tag=mt5pyserver
$ docker run --name=prueba -it mt5pyserver bash

And then:

root@a169a4c4aae8:~# cd ".wine/drive_c/Program Files/python-3.9.0"
root@a169a4c4aae8:~/.wine/drive_c/Program Files/python-3.9.0# wine python.exe get-pip.py 
Collecting pip
  Using cached pip-20.2.3-py2.py3-none-any.whl (1.5 MB)
Collecting setuptools
  Using cached setuptools-50.3.0-py3-none-any.whl (785 kB)
Collecting wheel
  Using cached wheel-0.35.1-py2.py3-none-any.whl (33 kB)
Installing collected packages: pip, setuptools, wheel
  WARNING: The scripts pip.exe, pip3.9.exe and pip3.exe are installed in 'C:\Program Files\python-3.9.0\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts easy_install-3.9.exe and easy_install.exe are installed in 'C:\Program Files\python-3.9.0\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script wheel.exe is installed in 'C:\Program Files\python-3.9.0\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-20.2.3 setuptools-50.3.0 wheel-0.35.1
root@a169a4c4aae8:~/.wine/drive_c/Program Files/python-3.9.0# pip
bash: pip: command not found
root@a169a4c4aae8:~/.wine/drive_c/Program Files/python-3.9.0# wine Scripts/pip.exe
Traceback (most recent call last):
  File "runpy.py", line 197, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "C:\Program Files\python-3.9.0\Scripts\pip.exe\__main__.py", line 4, in <module>
ModuleNotFoundError: No module named 'pip'
root@a169a4c4aae8:~/.wine/drive_c/Program Files/python-3.9.0# wine --version
wine-5.0 (Ubuntu 5.0-3ubuntu1)
root@a169a4c4aae8:~/.wine/drive_c/Program Files/python-3.9.0# 
root@a169a4c4aae8:~/.wine/drive_c/Program Files/python-3.9.0# wine Scripts/pip.exe
Traceback (most recent call last):
  File "runpy.py", line 197, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "C:\Program Files\python-3.9.0\Scripts\pip.exe\__main__.py", line 4, in <module>
ModuleNotFoundError: No module named 'pip'
root@a169a4c4aae8:~/.wine/drive_c/Program Files/python-3.9.0#

What I need is to correctly install pip in the in-windows python interpreter. There is an underlying reason: package MetaTrader5 only exists for windows python versions. In the end, I'd like to install that package and also aiohttp. The server will then make use of those packages as described in the MetaTrader5 documentation for Python integration and some magic on my own.

What can I do to avoid getting that error?

score 2 · Answer 1 · edited Oct 15 '20 at 20:23

Have a look at this Dockerfile. It might give you some insight. The approach used is completely different.

Some noteworthy differences between this Dockerfile and your setup:

Use wine from the WineHQ's package repository, not Ubuntu's.
Download python msi setup files from python.org and install them using wine msiexec.

Here's an except from the Dockerfile:

...
...

ARG PYINSTALLER_VERSION=4.0

...
...

RUN set -x \
    && for msifile in `echo core dev exe lib path pip tcltk tools`; do \
        wget -nv "https://www.python.org/ftp/python/$PYTHON_VERSION/amd64/${msifile}.msi"; \
        wine msiexec /i "${msifile}.msi" /qb TARGETDIR=C:/Python37; \
        rm ${msifile}.msi; \
    done \
    && cd /wine/drive_c/Python37 \
    && echo 'wine '\''C:\Python37\python.exe'\'' "$@"' > /usr/bin/python \
    && echo 'wine '\''C:\Python37\Scripts\easy_install.exe'\'' "$@"' > /usr/bin/easy_install \
    && echo 'wine '\''C:\Python37\Scripts\pip.exe'\'' "$@"' > /usr/bin/pip \
    && echo 'wine '\''C:\Python37\Scripts\pyinstaller.exe'\'' "$@"' > /usr/bin/pyinstaller \
    && echo 'wine '\''C:\Python37\Scripts\pyupdater.exe'\'' "$@"' > /usr/bin/pyupdater \
    && echo 'assoc .py=PythonScript' | wine cmd \
    && echo 'ftype PythonScript=c:\Python37\python.exe "%1" %*' | wine cmd \
    && while pgrep wineserver >/dev/null; do echo "Waiting for wineserver"; sleep 1; done \
    && chmod +x /usr/bin/python /usr/bin/easy_install /usr/bin/pip /usr/bin/pyinstaller /usr/bin/pyupdater \
    && (pip install -U pip || true) \
    && rm -rf /tmp/.wine-*

ENV W_DRIVE_C=/wine/drive_c
ENV W_WINDIR_UNIX="$W_DRIVE_C/windows"
ENV W_SYSTEM64_DLLS="$W_WINDIR_UNIX/system32"
ENV W_TMP="$W_DRIVE_C/windows/temp/_$0"

# install Microsoft Visual C++ Redistributable for Visual Studio 2015, 2017 and 2019 dll files
RUN set -x \
    && rm -f "$W_TMP"/* \
    && wget -P "$W_TMP" https://aka.ms/vs/16/release/vc_redist.x64.exe \
    && cabextract -q --directory="$W_TMP" "$W_TMP"/vc_redist.x64.exe \
    && cabextract -q --directory="$W_TMP" "$W_TMP/a10" \
    && cabextract -q --directory="$W_TMP" "$W_TMP/a11" \
    && cd "$W_TMP" \
    && rename 's/_/\-/g' *.dll \
    && cp "$W_TMP"/*.dll "$W_SYSTEM64_DLLS"/

# install pyinstaller
RUN /usr/bin/pip install pyinstaller==$PYINSTALLER_VERSION

...
...

Also, why do you have to install pip when Python 2 >=2.7.9 or Python 3 >=3.4 already includes pip?

Further note that it is recommended to put related commands (apt-get update/install) in the same RUN step.

How can I correctly install & run pip in a wine-emulated python version (inside a docker container)?

1 Answers1