Pythonic way to copy list to a sip.voidptr from QSharedMemory

Question

I am using PyQT5 and the QSharedMemory class. I am creating a shared memory which can hold up 6 1-byte elements. To copy these elments in the shared memory array I am looping over the elments from the python list like the following snippet:

f = shared_mem.data()
k = f.asarray()
memtocopy = [0,1,2,3,4,5]
for i in range(0,len(memtocopy)):
    k[i]  = memtocopy[i]
shared_mem.unlock()

Which seems very unpythonic and boilerplate-code like. I am wondering if there is a more suitable way of achieving the same result?

When using

k[:] = memtocopy

or:

k[:] = np.asarray(memtocopy,np.uint8)

It will fail with the error message:

TypeError: can only assign another array of unsigned char to the slice

The whole test code for reproducing looks like the following:

from PyQt5 import QtCore 

# Create shared memory and attach it
shared_mem = QtCore.QSharedMemory()
shared_mem.setNativeKey("test")
shared_mem.create(4*6)
shared_mem.attach()


# Fill in 
shared_mem.lock()
f = shared_mem.data()
k = f.asarray()

memtocopy = [0,1,2,3,4,5]

# Loop in question
for i in range(0,len(memtocopy)):
    k[i]  = memtocopy[i]

shared_mem.unlock()



# Read out
shared_mem.lock()
f1 = shared_mem.data()
k1 = f1.asarray()
shared_mem.unlock()

# Test results
if k1[0] == memtocopy[0]:
    print("success!")
else:
    print("fail!")

This will result in TypeError: can only assign another array of unsigned char to the slice, I have edited the question to show this case, too — Kev1n91, Jul 04 '18 at 09:06
Then it might be a matter of figuring out what array type it wants. I'd guess `ctypes`, but perhaps it supports the buffer protocol. [PyQt's documentation](http://pyqt.sourceforge.net/Docs/PyQt5/api/QtCore/qsharedmemory.html#PyQt5-QtCore-QSharedMemory) was no help. — Yann Vernier, Jul 04 '18 at 09:09
[`sip.voidptr.asarray`](http://pyqt.sourceforge.net/Docs/sip4/python_api.html#sip.voidptr.asarray) is new from version 4.16.5, which explains why I didn't have it in Debian Jessie. So the array type is [`sip.array`](http://pyqt.sourceforge.net/Docs/sip4/python_api.html#sip.array), which again happens to be documented only as far as existing. — Yann Vernier, Jul 04 '18 at 09:20
sip [should support the buffer protocol](http://pyqt.sourceforge.net/Docs/sip4/using.html#support-for-python-s-buffer-interface), so using a Python `array.array` or `bytearray` should really work. Lists of ints won't. — Yann Vernier, Jul 04 '18 at 09:25
I expect `k1` will be referencing the shared memory, so you should read it while you're holding the lock. — Yann Vernier, Jul 04 '18 at 12:55

ekhumoro · Accepted Answer · 2018-07-12T00:13:16.673

2

Here's a simpler approach using struct and memoryview that reads and writes the data with a couple of one-liners :

import struct
from PyQt5 import QtCore

shared_mem = QtCore.QSharedMemory()
shared_mem.setNativeKey("test")
shared_mem.create(4*6)
shared_mem.attach()

memtocopy = [0,1,2,3,4,5]

try:

    # Fill in 
    shared_mem.lock()
    shared_mem.data()[:] = memoryview(struct.pack('=6i', *memtocopy))
    shared_mem.unlock()

    # Read out
    shared_mem.lock()
    # python3
    k = memoryview(shared_mem.data()).cast('i')
    # python2
    # k = struct.unpack('=6i', memoryview(shared_mem.data()))
    shared_mem.unlock()

    if k[3] == memtocopy[3]:
        print("success!")
    else:
        print("fail!")

finally:

    shared_mem.detach()

edited Jul 12 '18 at 00:13

answered Jul 04 '18 at 21:13

ekhumoro

115,249
20
229
336

I am assumg '=6i' is for copying 6 integers? – Kev1n91 Jul 05 '18 at 07:35
1

@Kev1n91. Yes - the format is explained [here](https://docs.python.org/3/library/struct.html#struct-format-strings). The `=` should ensure the memoryview cast works the same way on all platforms. Of course, if you're only targeting one platform, you can just use `6i`. – ekhumoro Jul 05 '18 at 10:53
I am getting the error "memorieveiw" object has no attribute cast – Kev1n91 Jul 11 '18 at 13:40
@Kev1n91. That's because you've changed to python2. Obviously my answer uses python3, because that is what your original question uses. Anyway, I've added a python2 one-liner to my answer. – ekhumoro Jul 12 '18 at 00:10
I haven't noticed that, I am on a docker container which is not handled by me. Sorry about that mess up – Kev1n91 Jul 12 '18 at 08:07
@Kev1n91. Okay, no problem. Can you please now restore the accept on my answer? – ekhumoro Jul 12 '18 at 10:02

Yann Vernier · Answer 2 · 2018-07-04T12:49:33.800

1

After a bit of fiddling about, I managed to produce one combination that functioned for me:

a = array.array('i', range(6))
f[:] = buffer(a)
b = array.array('i')
b.fromstring(buffer(f))

This relies on the buffer protocol for reading both ways. It is likely you can use the array directly with your k, and fromstring has been renamed to frombytes in later versions.

In Python 3.4, this worked:

a = array.array('i', range(6))
f[:] = memoryview(a).cast('B')
b = array.array('i')
b.frombytes(memoryview(f))

memoryview replaced buffer, but knew the elements were 4 bytes large, so it required an additional cast.

edited Jul 04 '18 at 12:49

answered Jul 04 '18 at 09:50

Yann Vernier

15,414
2
28
26

This won't work in python3 (which the OP seems to be using), because there is no `buffer` function (and you cannot simply replace it with `memoryview`). – ekhumoro Jul 04 '18 at 11:36
I almost could - one cast was required. – Yann Vernier Jul 04 '18 at 12:50
what would happen if one uses integersr? frombytes wouldn't be usable at this point ? – Kev1n91 Jul 04 '18 at 12:59
This does use integers (`'i'` type in array). `frombytes` simply tells it to load from memory. – Yann Vernier Jul 04 '18 at 14:10

Pythonic way to copy list to a sip.voidptr from QSharedMemory

2 Answers2

Linked