Make .netrc authentication work with aiohttp

Question

I'd like to use a .netrc file with credentials to authenticate to an API using aiohttp. As far as I can tell this should be possible, as long as the file is in the home directory (or the relevant env variable set correctly) and trust_env=True set in the aiohttp.ClientSession.

But whatever I try, I get a 401 response. I've checked with requests, and it works just fine. I've browsed through the relevant code and it seems like it'll only pick up the credentials if a proxy is supplied. Can someone explain?

Here's an example that reproduces the issue:

First put a .netrc file in home directory:

machine httpbin.org
    login foo
    password bar

import aiohttp
import requests

url = "http://httpbin.org/basic-auth/foo/bar"

with requests.Session() as sess:
    r = sess.get(url)
    r.raise_for_status()

# no exception raised

async with aiohttp.ClientSession(trust_env=True) as session:
    r = await session.get(url)
    r.raise_for_status()

# exception raised

ClientResponseError: 401, message='UNAUTHORIZED', url=URL('http://httpbin.org/basic-auth/foo/bar')

score 1 · Accepted Answer · answered Jul 11 '22 at 11:05

From what I understand in the doc, the trust_env and .netrc credentials are only used for proxy authentication, not for regular server authentication.

For authentication to the server directly, the docs say that you have to use a BasicAuth object (as you surely know), but to use the .netrc file, one solution would be to use a custom authentication class, eg:

class NetrcAuth(aiohttp.BasicAuth):
    def __new__(cls, host):
        login, account, password = netrc.netrc().authenticators(host)
        return super().__new__(cls, login=login, password=password)

that you could then use as

from urllib.parse import urlparse

hostname = urlparse(url).hostname

async with aiohttp.ClientSession(auth=NetrcAuth(hostname)) as session:
    r = await session.get(url)
    r.raise_for_status()

Of course this is not optimal, as we would like to have the ClientSession take care of that for us, but it's maybe a step in the right direction?

Thanks for your answer! I guess it *technically* solves my problem, as you can work with `.netrc`. Unfortunately it doesn't help on a practical level as I need to know the hostname on a per-request basis. The desired behavior (as `requests.Session`) is that you can pass in different URLs to the same session, and auth gets applied based on a per-host basis automatically. — Val, Jul 12 '22 at 09:56
yes, this is of course far from optimal. Let's hope this gets implemented in aiohttp some day. — Martin Raspaud, Jul 13 '22 at 10:54

Make .netrc authentication work with aiohttp

1 Answers1