`struct.unpack_from` doesn't work with `bytearray`?

Question

Unpacking from strings works:

>>> import struct
>>> struct.unpack('>h', 'ab')
(24930,)
>>> struct.unpack_from('>h', 'zabx', 1)
(24930,)

but if its a bytearray:

>>> struct.unpack_from('>h', bytearray('zabx'), 1)
Traceback (most recent call last):
  File "<ipython-input-4-d58338aafb82>", line 1, in <module>
    struct.unpack_from('>h', bytearray('zabx'), 1)
TypeError: unpack_from() argument 1 must be string or read-only buffer, not bytearray

Which seems a little odd. What am I actually supposed to do about it? obviously I could:

>>> struct.unpack_from('>h', str(bytearray('zabx')), 1)
(24930,)

But i'm explicitly trying to avoid copying possibly large amounts of memory around.

You can use the buffer type to refer to the string without taking up more memory, then pass that as an argument. — Asad Saeeduddin, Mar 17 '13 at 22:32

score 6 · Accepted Answer · answered Mar 17 '13 at 22:31

6

It looks like buffer() is the solution:

>>> struct.unpack_from('>h', buffer(bytearray('zabx')), 1)
(24930,)

buffer() is not a copy of the original, its a view:

>>> b0 = bytearray('xaby')
>>> b1 = buffer(b0)
>>> b1
<read-only buffer for ...>
>>> b1[1:3]
'ab'
>>> b0[1:3] = 'nu'
>>> b1[1:3]
'nu'

Alternitively, You(I?) can just use python 3; the limitation is lifted:

Python 3.2.3 (default, Jun  8 2012, 05:36:09) 
[GCC 4.7.0 20120507 (Red Hat 4.7.0-5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import struct
>>> struct.unpack_from('>h', b'xaby', 1)
(24930,)
>>> struct.unpack_from('>h', bytearray(b'xaby'), 1)
(24930,)
>>>

answered Mar 17 '13 at 22:31

SingleNegationElimination

151,563
33
264
304

Why not just use `buffer('xaby', index, length)`? That doesn't use up extra memory either. – Asad Saeeduddin Mar 17 '13 at 22:37
2

it happens that the `struct` bit is wrapping around a `bytearray`, chosen *because* it is mutable. Each time rather than maintain the read-only view of the primary, mutable data, I'd rather just recreate the bits in the middle to get the data from its primary source. This also goes for just keeping `(24930,)`. I really do want to unpack from a `bytearray`. – SingleNegationElimination Mar 17 '13 at 22:41
@Asad: I suspect creating a `buffer` object consumes at least a little memory. – martineau Mar 17 '13 at 22:41
@TokenMacGuy I see now; wasn't really paying attention to the bytearray modification in the last few lines. – Asad Saeeduddin Mar 17 '13 at 22:50

`struct.unpack_from` doesn't work with `bytearray`?

1 Answers1