How to retain cookies for xmlrpc.client in Python 3?

Question

The default Python xmlrpc.client.Transport (can be used with xmlrpc.client.ServerProxy) does not retain cookies, which are sometimes needed for cookie based logins.

For example, the following proxy, when used with the TapaTalk API (for which the login method uses cookies for authentication), will give a permission error when trying to modify posts.

proxy = xmlrpc.client.ServerProxy(URL, xmlrpc.client.Transport())

There are some solutions for Python 2 on the net, but they aren't compatible with Python 3.

How can I use a Transport that retains cookies?

score 6 · Answer 1 · answered Nov 14 '17 at 17:22

Existing answer from GermainZ works only for HTTP. After a lot of time fighting with it, there is HTTPS adaptation. Note the context option which is crucial.

class CookiesTransport(xmlrpc.client.SafeTransport):
    """A SafeTransport (HTTPS) subclass that retains cookies over its lifetime."""

    # Note context option - it's required for success
    def __init__(self, context=None):
        super().__init__(context=context)
        self._cookies = []

    def send_headers(self, connection, headers):
        if self._cookies:
            connection.putheader("Cookie", "; ".join(self._cookies))
        super().send_headers(connection, headers)

    def parse_response(self, response):
        # This check is required if in some responses we receive no cookies at all
        if response.msg.get_all("Set-Cookie"):
            for header in response.msg.get_all("Set-Cookie"):
                cookie = header.split(";", 1)[0]
                self._cookies.append(cookie)
        return super().parse_response(response)

The reason for it is that ServerProxy doesn't respect context option related to transport, if transport is specified, so we need to use it directly in Transport constructor.

Usage:

import xmlrpc.client
import ssl


transport = CookiesTransport(context=ssl._create_unverified_context())
# Note the closing slash in address as well, very important
server = xmlrpc.client.ServerProxy("https://<api_link>/", transport=transport)

# do stuff with server
server.myApiFunc({'param1': 'x', 'param2': 'y'})

score 1 · Accepted Answer · answered Sep 16 '14 at 19:02

This is a simple Transport subclass that will retain all cookies:

class CookiesTransport(xmlrpc.client.Transport):
"""A Transport subclass that retains cookies over its lifetime."""

    def __init__(self):
        super().__init__()
        self._cookies = []

    def send_headers(self, connection, headers):
        if self._cookies:
            connection.putheader("Cookie", "; ".join(self._cookies))
        super().send_headers(connection, headers)

    def parse_response(self, response):
        for header in response.msg.get_all("Set-Cookie"):
            cookie = header.split(";", 1)[0]
            self._cookies.append(cookie)
        return super().parse_response(response)

Usage:

proxy = xmlrpc.client.ServerProxy(URL, CookiesTransport())

Since xmlrpc.client in Python 3 has better suited hooks for this, it's much simpler than an equivalent Python 2 version.

Note that this snippet does not work as is. Because if server doesn't send cookie for some requests (example - there are cookies in response only after first "login" call and other responses do not contain them), it will fail trying to get_all("Set-Cookie"). So additional check is required here — The Godfather, Nov 14 '17 at 14:32

How to retain cookies for xmlrpc.client in Python 3?

2 Answers2

Linked