Get Whole String From a Struct Pack in Python?

Question

What I am trying to do is to get a full length string from a Python struct. Here is my struct: struct.pack('6s','Hello!') Which is 'Hello!'. But, when I am doing struct.unpack('s','Hello!') I get an error: error: unpack requires a string argument of length 1. One answer would be to get the length of the input and put that in front of the 's', however in more complicated situations where you get structs that contain more than just one string. So what I would like to know is how to get a string from a struct that has many parts to it. For example, a struct contains int, string, unsigned short, string.

Tl;Dr: How would one get a full string out of a struct that has multiple integers and without knowledge of the length of the string?

A little more knowledge of the true structure:

Int, string (known size-is IP address of server (I am server)), Short, short, short, string (unknown size-is Username).

You need to know the size of the strings that go into a struct. — thebjorn, Feb 20 '15 at 19:40
Check this out: http://stackoverflow.com/questions/3753589/packing-and-unpacking-variable-length-array-string-using-the-struct-module-in-py — Savir, Feb 20 '15 at 19:41
Do you know if your string is null terminated or similar - otherwise, you do need to know the length... — Jon Clements, Feb 20 '15 at 20:00
Doesn't even seem to need a struct module. So you have Int = 4bytes, ip = 11?, 3x short = 6, string. Thus bytearray[21:].decode('utf-8')? — dgrigonis, Jun 20 '23 at 11:24

Tui Popenoe · Answer 1 · 2015-02-20T20:01:37.750

Use

string = 'Hello!'
struct_fmt = "{}s".format(len(string))
struct.pack(struct_fmt, string)
struct.unpack(struct_fmt, string)

You have to know about the data you are packing if you want to unpack it correctly. To unpack int, string, unsigned short, string multiple data types, your struct_fmt would look like:

struct_fmt = "I %ds H %ds" % (len(string1), len(string2))

stuct only supports fixed-length structures. For variable length strings, you can dynamically construct the format string by converting it to bytes before passing it to pack()/unpack()

string1 = bytes(string1, 'utf-8')
struct.pack("I%ds H %ds" % (len(string1),), len(string1), string1)

So for your example:

string1 = 'Hello!'
string2 = 'Goodbye!'
s = bytes(string1, 'utf-8')
s2 = bytes(string2, 'utf-8')
struct.pack("I I%ds H I%ds" % ((len(s),), len(s), s), (len(s2),), len(s2), s2)

Where the integer and the short have been omitted

The issue is that I don't know what the length of the string is: it would be raw data sent from a game and I am trying to interpret essentially this order: Int, string, unsinged short, string, long, string — Cold Diamondz, Feb 20 '15 at 19:47

score 0 · Answer 2 · edited Apr 12 '16 at 15:22

You can make this work with pascal strings. Pascal strings are prefixed with the length, but standard struct package doesn't support variable length pascal strings.

Here's some wrapper functions I wrote which help, they seem to work.

Here's the unpacking helper:

def unpack_from(fmt, data, offset = 0):
    (byte_order, fmt, args) = (fmt[0], fmt[1:], ()) if fmt and fmt[0] in ('@', '=', '<', '>', '!') else ('@', fmt, ())
    fmt = filter(None, re.sub("p", "\tp\t",  fmt).split('\t'))
    for sub_fmt in fmt:
        if sub_fmt == 'p':
            (str_len,) = struct.unpack_from('B', data, offset)
            sub_fmt = str(str_len + 1) + 'p'
            sub_size = str_len + 1
        else:
            sub_fmt = byte_order + sub_fmt
            sub_size = struct.calcsize(sub_fmt)
        args += struct.unpack_from(sub_fmt, data, offset)
        offset += sub_size
    return args

Here's the packing helper:

def pack(fmt, *args):
    (byte_order, fmt, data) = (fmt[0], fmt[1:], '') if fmt and fmt[0] in ('@', '=', '<', '>', '!') else ('@', fmt, '')
    fmt = filter(None, re.sub("p", "\tp\t",  fmt).split('\t'))
    for sub_fmt in fmt:
        if sub_fmt == 'p':
            (sub_args, args) = ((args[0],), args[1:]) if len(args) > 1 else ((args[0],), [])
            sub_fmt = str(len(sub_args[0]) + 1) + 'p'
        else:
            (sub_args, args) = (args[:len(sub_fmt)], args[len(sub_fmt):])
            sub_fmt = byte_order + sub_fmt
        data += struct.pack(sub_fmt, *sub_args)
    return data

Get Whole String From a Struct Pack in Python?

2 Answers2