Pip install Python packages from Requirements.txt file on github

Question

Hey i have a Repository https://github.com/yotestyo/test with a text file Requirements.txt containing package names i want to install with pip in my Anaconda env. What is the command to install the packages from github. On my local machine it is pip install -r Requirements.txt in the dierectory. pip install -r https://github.com/yotestyo/test/Requiements.txt does not work, neither pip install -r https://github.com/yotestyo/test/blob/main/Requirements.txt

Answer 1

Use pip install -r https://raw.githubusercontent.com/yotestyo/test/main/Requirements.txt

Open that file requirement.txt in GitHub, then you will get the URL of the above form

Answer 2

To understand what's happening, check the Content-Type header of the request you make in the background with pip for the URL with curl -qI <url> (you can make pip tell you more with pip install -r ... --verbose):

• https://github.com/yotestyo/test/Requiements.txt = text/html; charset=utf-8
• https://github.com/yotestyo/test/blob/main/Requirements.txt = text/html; charset=utf-8

However pip needs a text, not HTML document. It's not that pip can't get the file, but the content is very different for what pip expects to get. For example, to extract data from an HTML document one would either need regex (for very simple HTML and in a naive approach) or a tree/node document parser such as lxml.

Even if pip were using that, it'd need to store a database with paths to the content because what's visible in GitHub's HTML will not be accessible on for example GitLab's or Bitbucket's HTML and a different traversal of the document would be necessary. That's why you need the plaintext format i.e. as if it were a file on your PC.

When you issue the request for the raw content:

• https://raw.githubusercontent.com/yotestyo/test/main/Requirements.txt = text/plain; charset=utf-8

then the Content-Type is matching the expected data i.e. plaintext, not an HTML document and afterwards pip can read it as a normal file instead of various ways for fetching content out of HTML.