What's the correct way to use a unix domain socket in requests framework?

Question

Usually, doing a post request using requests framework is done by:

payload = {'key1': 'value1', 'key2': 'value2'}
r = requests.post("http://httpbin.org/post", data=payload)

But: How do I connect to a unix socket instead of doing a TCP connection?

On a related note, how to encode domain path in the URL?

libcurl allows application to supply own socket on which to perform request
LDAP invented own scheme ldapi where socket name is %-encoded in host field
httpie uses http+unix scheme and %-encoded path in host field

These are some examples, but is there an RFC or established best practice?

A socket is a file descriptor. You read/write the descriptor. I do not think I get what you want. — jim mcnamara, Nov 17 '14 at 02:30
Yes, also non-unix sockets are descriptors. But this is not a low-level implementation but an HTTP client. — Luis Masuelli, Nov 17 '14 at 02:32
requests is an HTTP library. If you write an adapter you can control how the connection is created. That said, it doesn't appear as if cURL supports this, so do not expect requests to go out of their way to add ways to support this for you — Ian Stapleton Cordasco, Nov 17 '14 at 03:58
@Kevin many services do, for example MySQL. It's a small step to services that use HTTP. Besides several server frameworks support unix domain sockets, e.g. cherrypy, flask, gunicorn. The main advantage is UNIX security model in multiuser environment where server process is not privileged. — Dima Tisnek, Dec 03 '14 at 10:10
@Kevin think on other type of servers, like an inner local server mounted with Tornado serving a proxiable outwards interface, and a unix socket interface for inner, control logic/commands — Luis Masuelli, Dec 08 '14 at 20:01

Dima Tisnek · Answer 1 · 2015-10-22T17:04:20.037

28

There's no need to reinvent the wheel:

https://github.com/msabramo/requests-unixsocket

URL scheme is http+unix and socket path is percent-encoded into the host field:

import requests_unixsocket

session = requests_unixsocket.Session()

# Access /path/to/page from /tmp/profilesvc.sock
r = session.get('http+unix://%2Ftmp%2Fprofilesvc.sock/path/to/page')
assert r.status_code == 200

edited Oct 22 '15 at 17:04

answered Dec 03 '14 at 10:16

Dima Tisnek

11,241
4
68
120

2

Or to interact with Docker's socket (`/var/run/docker.sock`), use a URL like `http+unix://%2Fvar%2Frun%2Fdocker.sock/info` – Marc Abramowitz Feb 19 '17 at 03:15
1

By the way, there is also a plugin for [HTTPie](https://httpie.org/) to interact with unix domain sockets: https://github.com/msabramo/httpie-unixsocket – Marc Abramowitz Feb 19 '17 at 03:19
Could you please elaborate why we need to specify separators as `%2F` ? – farch Jun 08 '22 at 06:39
1

Because two things with slashes have to be stashed into the URL: the file path and the URL path. The file path is escaped and becomes the host part of the URL. – Dima Tisnek Jun 09 '22 at 11:19

score 15 · Answer 2 · answered Jan 04 '20 at 21:10

If you are looking for a minimalistic and clean approach to this in Python 3, here's a working example that will talk to Ubuntu's snapd on a unix domain socket.

import requests
import socket
import pprint

from urllib3.connection import HTTPConnection
from urllib3.connectionpool import HTTPConnectionPool
from requests.adapters import HTTPAdapter


class SnapdConnection(HTTPConnection):
    def __init__(self):
        super().__init__("localhost")

    def connect(self):
        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
        self.sock.connect("/run/snapd.socket")


class SnapdConnectionPool(HTTPConnectionPool):
    def __init__(self):
        super().__init__("localhost")

    def _new_conn(self):
        return SnapdConnection()


class SnapdAdapter(HTTPAdapter):
    def get_connection(self, url, proxies=None):
        return SnapdConnectionPool()


session = requests.Session()
session.mount("http://snapd/", SnapdAdapter())
response = session.get("http://snapd/v2/system-info")
pprint.pprint(response.json())

Hi, Nice code, would you please tell me how we can handle exceptions with your code — Umar Draz, Aug 10 '21 at 07:15

Maxim Egorushkin · Answer 3 · 2014-12-01T14:37:14.557

7

You can use socat to create a TCP to UNIX socket proxy, something like:

socat TCP-LISTEN:80,reuseaddr,fork UNIX-CLIENT:/tmp/foo.sock

And then send your http requests to that proxy. The server listening on UNIX socket /tmp/foo.sock still has to understand HTTP because socat does not do any message conversion.

edited Dec 01 '14 at 14:37

answered Dec 01 '14 at 13:54

Maxim Egorushkin

131,725
17
180
271

score 2 · Answer 4 · answered Dec 03 '14 at 06:05

2

requests has no implementation to work with unix sockets out-of-the-box.

But you can create custom adapter that will connect to unix socket, send request and read answer.

All methods you need to implement are .send() and .close(), that's easy and straightforward.

After registering the adapter in session object you can use requests machinery with UNIX transport.

answered Dec 03 '14 at 06:05

Andrew Svetlov

16,730
8
66
69

Also, in case of DIY, I think it's easier to subclass and add own connection pool. Only getting a new connection needs to be changed in the latter. – Dima Tisnek Dec 08 '14 at 12:51

score 0 · Answer 5 · answered Jul 01 '22 at 02:05

Another solution by https://stackoverflow.com/users/1105249/luis-masuelli is to use the httpx package instead,

>>> import httpx
>>> # Define a transporter
>>> transport = httpx.HTTPTransport(uds="/tmp/profilesvc.sock")
>>> client = httpx.Client(transport=transport)
>>> response = client.get('http://path/to/page')
>>> assert response.status_code == 200

What's the correct way to use a unix domain socket in requests framework?

5 Answers5

Linked