0

I'm trying to build a .simg file from a .recipe. In the recipe I upgraded pip. In the building process I get following error after downloading pytorch: can someone tell me what's wrong with pip?

Collecting torch==1.5.1
  Downloading torch-1.5.1-cp35-cp35m-manylinux1_x86_64.whl (753.2 MB)
     |############################### | 753.2 MB 6.8 MB/s eta 0:00:01
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/commands/install.py", line 333, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/legacy/resolver.py", line 314, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 469, in prepare_linked_requirement
    hashes=hashes,
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 259, in unpack_url
    hashes=hashes,
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 130, in get_http_url
    link, downloader, temp_dir.path, hashes
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 281, in _download_http_url
    for chunk in download.chunks:
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/progress_bars.py", line 166, in iter
    for x in it:
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/network/utils.py", line 39, in response_chunks
    decode_content=False,
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 564, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 507, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/cachecontrol/filewrapper.py", line 65, in read
    self._close()
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/cachecontrol/filewrapper.py", line 52, in _close
    self.__callback(self.__buf.getvalue())
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/cachecontrol/controller.py", line 309, in cache_response
    cache_url, self.serializer.dumps(request, response, body=body)
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/cachecontrol/serialize.py", line 72, in dumps
    return b",".join([b"cc=4", msgpack.dumps(data, use_bin_type=True)])
MemoryError

The .recipe file I used looks as follows: As you can see I tried to update pip, so it should be the most recent version.

#header
Bootstrap: docker
From: nvidia/cuda:10.1-cudnn7-devel-ubuntu16.04

#Sections


%post
#commands executed inside the container after os has been installed. Used for setup of the container
#devine env variables for build time

# Install the commonly used packages (from repo)
    apt-get -y update
    apt-get install -y python-pip
    apt-get install -y python3-pip
    pip3 install --upgrade setuptools pip

# Install the commonly used packages (from repo)
    apt-get update && apt-get install -y --no-install-recommends \
        apt-utils \
        build-essential \
        curl \
        git \
        libopenblas-dev \
        libcurl4-openssl-dev \
        libfreetype6-dev \
        libpng-dev \
        libzmq3-dev \
        locales \
        pkg-config \
        python \
        python-dev \
        python-setuptools \
        rsync \
        software-properties-common \
        unzip \
        vim \
        zip \
        zlib1g-dev


    
    pip3 install --no-cache-dir --upgrade future matplotlib scipy

    # Install other commonly-needed python packages
    pip3 install --no-cache-dir --upgrade \
        future \
        matplotlib \
        scipy


pip3 install opencv-contrib-python
python3 -m pip install keras
pip3 install tensorflow-gpu
CUDA_PATH=/usr/local/cuda pip3 install cupy-cuda101
CUDA_ROOT=/usr/local/cuda PATH=$PATH:/usr/local/cuda/bin pip3 install scikit-cuda
pip3 install pyro4
pip3 install pandas
pip3 install torchvision
pip3 install torch
pip3 install backpack-for-pytorch
apt-get update
apt-get -y install python3-tk
apt-get -y install python-opencv
pip3 install opencv-contrib-python
pip3 install appdirs requests
goodL
  • 1
  • 1
  • Hello Moreezee, welcome to SO. Can you please share the Dockerfile? The problem needs to be reproduced in order to be solved. This post is a great lecture, to increase the chances of receiving an answer: https://stackoverflow.com/help/how-to-ask Thanks you! – Neo Anderson Jul 26 '20 at 17:59
  • 2
    `MemoryError` would suggest that you don't have enough memory for what you are trying to do. – tripleee Jul 26 '20 at 18:20
  • okay thank you, but I wonder why... I build the image using vagrant and by running ` free -h ` it says : there's 985M Memory available, but only 35M are used. So there should be enough.. – goodL Jul 26 '20 at 20:05

1 Answers1

0

MemoryError says it: you've not got enough memory available/allocated to the build VM. The pytorch github issues have a similar problem and several solutions, the most simple being:

  • download the package and install directly instead of using pip and the package name
  • give them VM more memory
tsnowlan
  • 3,472
  • 10
  • 15