pyGithub maximum use of API call rate

Question

I am attempting to use the pyGithub library for accessing v3 API of github. Although this library is simple to use. I found the documentation to be very vague.

Below I am successfully getting the contents of a file with the file path and its sha. My end goal is to reduce my API calls from 3 to just 1, since I want to make use of the full 5000 API calls within the hour.

from github import Github
gh = Github(access_token) # I supply an access token here.
user = gh.get_user(owner_name) # This takes 1 API call
repo = user.get_repo(repo_name) # This takes 1 API call


file = repo.get_file_contents(filename, ref=sha) # This takes 1 API call

Does anyone know how I can pass the repo and owner name to get_file_contents() or a similar function I can use to achieve this?

Any help is appreciated.

score 1 · Answer 1 · answered Dec 20 '18 at 06:00

1

Does anyone know how I can pass the repo and owner name to get_file_contents()

Given the current implementation of get_file_contents, it expects:

either a GithubObject (which costs API calls)
or a string (which does not cost API calls)

But both depends on a class Repository, which does involve API calls.
So if you can make your process a long-lived one, able to reuse within a single execution session that repository, that would be best.

But if you have to get files from multiple repos, that won't help.

answered Dec 20 '18 at 06:00

VonC

1,262,500
529
4,410
5,250

I ended writing my own API caller. That can do this in one API call. – XChikuX Dec 29 '18 at 02:55
@XChikuX Great! Do you have a link to an open-source version of that API caller? – VonC Dec 29 '18 at 07:42
I haven't commited it online anywhere. I'll just include it in my answer. Feel free to make use of it. – XChikuX Dec 30 '18 at 05:26

XChikuX · Accepted Answer · 2019-01-25T21:46:46.823

You can reduce it from 3 API calls to just 2 by using get_repo() with the format 'owner_name/repo_name'

from github import Github
gh = Github(access_token) # I supply an access token here.
repo = gh.get_repo(owner_name+'/'+repo_name) # This takes 1 API call

file = repo.get_file_contents(filename, ref=sha) # This takes 1 API call

Just mentioning this here for future reference. In actuality I ended up using the requests library and forming my own api calls.

Like This:

import requests
# Import python's base64 decoder
from base64 import b64decode as decode

def GET(owner_repo,filename,sha,access_token):
    # Supply Headers
    headers = {'content-type': 'application/json', 'Authorization':'token '+access_token}
    # This function is stable so long as the github API structure does not change. 
    # Also I am using the previously mentioned combo of owner/repo.
    url = 'https://api.github.com/repos/%s/contents/%s?ref=%s' % (owner_repo, filename, sha)
    # Let's stay within the API rate limits
    url_rate_limit = 'https://api.github.com/rate_limit'
    r = requests.get(url_rate_limit, headers=headers)
    rate_limit = int(r.json()['resources']['core']['remaining'])
    if(rate_limit > 0):
        # The actual request
        r = requests.get(url, headers=headers)
        # Note: you will need to handle the any error codes yourself. 
        # I am just safe checking for '200' OK
        if(r.status_code == 200):
            content = r.json()['content']
            # Decode base64 content
            decoded_content = decode(content)
            return decoded_content

I license the above code under the MIT license.

Vladimir Panteleev · Answer 3 · 2021-08-12T18:54:30.040

GitHub API supports conditional requests, and cache hits do not count against the rate limit:

Making a conditional request and receiving a 304 response does not count against your Rate Limit, so we encourage you to use it whenever possible.

However, PyGithub does not implement caching:

https://github.com/PyGithub/PyGithub/issues/585

It is, however, possible in GitHub3:

https://github.com/sigmavirus24/github3.py/issues/75#issuecomment-128345063

There are some packages which add caching to Requests:

There is requests-cache, which has a global patching mechanism, but it does not yet support HTTP preconditions
There is cachecontrol, which does not have a global patching mechanism, but I managed to integrate it with PyGithub by patching some internals:

gh = github.Github(token)
class CachingConnectionClass(gh._Github__requester._Requester__connectionClass):
    def __init__(self, *args, **kwargs):
        super(gh._Github__requester._Requester__connectionClass, self).__init__(*args, **kwargs)
        self.session = CacheControl(self.session,
                                    cache=FileCache('.github-cache'))
gh._Github__requester._Requester__connectionClass = CachingConnectionClass

pyGithub maximum use of API call rate

3 Answers3