PyBytes_FromString different endianness

Question

I have a python-wrapped C++ object whose underlying data is a container std::vector<T> that represents bits. I have a function that writes these bits to a PyBytes object. If the endianness is the same, then there is no issue. However if I wish to write the bytes in a different endianness, then I need to bitswap (or byteswap) each word.

Ideally, I could pass an output iterator to the PyBytes_FromString constructor, where the output operator just transforms the endianness of each word. This would be O(1) extra memory, which is the target.

Less ideally, I could somehow construct an empty PyBytes object, create the different-endianness char array manually and somehow assign that to the PyBytes object (basically reimplementing the PyBytes constructors). This would also be O(1) extra memory. Unfortunately, the way to do this would be to use _PyBytes_FromSize, but that's not available in the API.

The current way of doing this is to create an entire copy of the reversed words, just to then copy that representation over to the PyBytes objects representation.

I think the second option is the most practical way of doing this, but the only way I can see that working is by basically copying the _PyBytes_FromSize function into my source code which seems hacky. I'm new to the python-C api and am wondering if there's a cleaner way to do this.

Is `T` some type other than `std::byte`, `char`, or `unsigned char`? — user2357112, May 02 '20 at 21:48
It currently is not, but I'd like to keep the option open if possible — Throckmorton, May 18 '20 at 20:16

score 0 · Answer 1 · answered May 02 '20 at 21:44

PyBytes_FromStringAndSize lets you pass NULL as the first argument, in which case it returns an uninitialized bytes object (which you can edit). It's really just equivalent to _PyBytes_FromSize and would let you do your second option.

If you wanted to try your "output iterator" option instead, then the solution would be to call PyBytes_Type:

 PyObject *result = PyObject_CallFunctionObjArgs((PyObject*)&PyBytes_Type, your_iterable, NULL);

Any iterable that returns values between 0 and 255 should work. You can pick the PyObject_Call* that you find easiest to use.

I suspect writing the iterable in C/C++ will be more trouble than writing the loop though.

PyBytes_FromString different endianness

1 Answers1