Jackson JSON serialisation format from Python

Question

I'm trying to build a tool which needs to do admin operation on a Apache Pulsar service. For some reason they have decided to not use plain JSON for the parameters that is in the body of the REST API instead they seem to use Jackson JSON serialisation. The tool I develop is written in Python 3.6 and I'm looking for ways to encode simple data structures into the Jackson JSON serialisation format or even finding the spec for the serialised format. Does such documentation or Python code exist? Since the typical data structures that is needed to be serialised are simple like a Set<AuthActions>, with AuthActions being an enum, it would be feasible to hand-code such things directly into the serialised format, if known.

Edited with code example:

import asyncio
import aiohttp
import ssl
import os

async def go(loop):
    current_dir = os.path.abspath(os.path.dirname(__file__))
    sslcontext = ssl.create_default_context(cafile=os.path.join(current_dir, 'cacert.pem'))
    sslcontext.load_cert_chain(os.path.join(current_dir, 'super-cert.pem'),
                               os.path.join(current_dir, 'super-key.pem'))
    async with aiohttp.ClientSession(loop=loop) as session:
        async with session.post('https://localhost:8081/admin/namespaces/sample/standalone/ns1/permissions/testrole',
                                 json={'actions': [0, 1]}, ssl=sslcontext) as resp:
             print(resp.status)
             print(await resp.text())
        async with session.get('https://localhost:8081/admin/persistent/sample/standalone/ns1', ssl=sslcontext) as resp:
            print(resp.status)
            print(await resp.text())

loop = asyncio.get_event_loop()
loop.run_until_complete(go(loop))
loop.close()

Jackson JSON is just a JSON serializer for Java, not a special format - i.e. it serializes to and from a plain JSON so consuming and building such objects is perfectly suitable for the built-in Python `json` module. What can cause issues are data-binding libraries on top of that which might expect a specific structure but that depends on and should be explained by the service you're using. Since you've already mentioned Apache Pulsar, there is an official Python Pulsar client to save you the trouble: [`pulsar-client`](https://pulsar.incubator.apache.org/docs/v1.22.0-incubating/clients/Python/) — zwer, Apr 24 '18 at 10:51
Thanks, but the pulsar-client is only for produce and consume clients not admin clients. OK, then it is Jackson data-binding I have issues with since the server states: org.glassfish.jersey.server.ContainerException: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `java.util.HashSet` out of START_OBJECT token — Harald Gustafsson, Apr 24 '18 at 10:58
Can you post how and what are you sending to the Pulsar server? — zwer, Apr 24 '18 at 11:04
I have reported that the official documentation don't include the actions param to Apache Pulsar issue tracker on github. The pulsar admin client (written in Java) is including actions parameter in body. See https://github.com/apache/incubator-pulsar/issues/1633 — Harald Gustafsson, Apr 24 '18 at 11:21
Yeah, I've checked how `pulsar-admin` does it and saw that it still takes parameters (it wouldn't make much sense otherwise, anyway). Anyway, there is nothing special in the JSON they're sending as an example apart from indentation, so on the off-chance that indentation is the culprit, have you tried replacing `json={'actions': [0, 1]}` with: `data=json.dumps({"actions": [0, 1]}, indent=2)`? Also, you may want to try to pass your `0` and `1` as strings because, looking at their posted examples, it appears that integer values are ambiguous. — zwer, Apr 24 '18 at 11:26
I just realized that the java admin client does not take a map but rather just a list so when I replaced the above `json={"actions": [0, 1]}` with `json=[0, 1]` it worked! — Harald Gustafsson, Apr 24 '18 at 13:46
They sure could use some serious documentation work... I'm glad you've got it it sorted out. — zwer, Apr 24 '18 at 14:05

score 2 · Answer 1 · answered Apr 25 '18 at 07:18

It turns out that it was only lack of documentation in Apache Pulsar admin interface. Here is a working example:

import asyncio
import aiohttp
import ssl
import os

async def go(loop):
    current_dir = os.path.abspath(os.path.dirname(__file__))
    sslcontext = ssl.create_default_context(cafile=os.path.join(current_dir, 'cacert.pem'))
    sslcontext.load_cert_chain(os.path.join(current_dir, 'super-cert.pem'),
                               os.path.join(current_dir, 'super-key.pem'))
    async with aiohttp.ClientSession(loop=loop) as session:
        async with session.post('https://localhost:8081/admin/namespaces/sample/standalone/ns1/permissions/testrole',
                                 json=[0, 1], ssl=sslcontext) as resp:
             print(resp.status)
             print(await resp.text())
        async with session.get('https://localhost:8081/admin/persistent/sample/standalone/ns1', ssl=sslcontext) as resp:
            print(resp.status)
            print(await resp.text())

loop = asyncio.get_event_loop()
loop.run_until_complete(go(loop))
loop.close()

Jackson JSON serialisation format from Python

1 Answers1