Structures packing/unpacking class design

Question

I'm implementing binary file format (~15 different structures) reader/writer and i have a bit dilemma which design pattern i should use.

Each structure contains method for pack/unpack:

class Struct1:  
    def pack():
        struct.pack(self.foo)
        ...
    def unpack():  
        self.foo = struct.unpack()
        ...
class Struct2:  
    def pack():
        struct.pack(self.foo)
        ...
    def unpack():  
        self.foo = struct.unpack()
        ...
...

vs.

There is Reader/Writer class with read/write method for each structure:

class Reader:
    def read_struct1():
        s = Struct1()
        s.foo = struct.unpack()
        ...
        return s
    def read_struct2():
        s = Struct2()
        s.foo = struct.unpack()
        ...
        return s
    ...
class Writer:
    def write_struct1(s):
        struct.pack(s.foo)
        ...
    def write_struct2(s):
        struct.pack(s.foo)
        ...
    ...

score 2 · Accepted Answer · answered Dec 27 '12 at 14:59

The purpose of writing separate Reader/Writer classes is to support different output processing - write to console, write to bytes, write to a JSON string, whatever. The important thing is that each of the classes Struct1, Struct2, etc. knows what attributes are significant for saving their state. This is the philosophy used by the pickle module.

class Struct1(object):
    fields = ('a','b','c')
    ...

class Struct2(object):
    fields = ('foo', 'bar', 'baz')
    ...

Now various writer classes can be written using this metadata, without having to write class-specific code:

class StructWriter(object):
    packTypeMap = {int:'i', float:'f', str:'s'}

    def write(self, obj):
        fields = obj.fields
        packstring = ''.join(packTypeMap[type(f)] for f in fields)
        packargs = (getattr(obj,f) for f in fields)
        return struct.pack(packstring, *packargs)

class DictWriter(object):
    def write(self, obj):
        return dict((f, getattr(obj,f)) for f in obj.fields)

class JSONWriter(object):
    jsonTypeMap = {str:lambda s:"'"+s+"'"}
    defaultJsonFunc = lambda x:str(x)
    def write(self, obj):
        # not really recommended to roll your own strings, but for illustration...
        fields = obj.fields
        outargs = (getattr(obj,f) for f in fields)
        outvals = (jsonTypeMap.get(type(arg),defaultJsonFunc)(arg) 
                       for arg in outargs)
        return ('{' +
            ','.join("'%s':%s" % field_val for field_val in zip(fields, outvals))
            '}')

class ZipJSONWriter(JSONWriter):
    def write(self, obj):
        import zlib
        return zlib.compress(super(ZipJSONWriter,self).write(obj))        

class HTMLTableWriter(object):
    def write(self, obj):
        out = "<table>"
        for field in obj.fields:
            out += "<tr><td>%s</td><td>%s</td></tr>" % (field, getattr(obj,field))
        out += "</table>"
        return out

score 1 · Answer 2 · answered Dec 27 '12 at 14:02

1

The former seems to make a lot more sense - the structures are the things so it makes more sense to represent them as objects. I see no real advantage to the second method.

answered Dec 27 '12 at 14:02

Gareth Latty

86,389
17
178
183

score 0 · Answer 3 · answered Dec 27 '12 at 14:23

Do they have to be objects? For management purposes I would be tempted to consider just putting the pack/unpack strings into a dict (with a (named)tuple as value), and have that in a separate module...

some_file.py:

structs = {
    'struct1': ('unpack', 'pack'),
    'struct2': ('other unpack', 'other pack')
...
}

And if needs be have a PackerUnPacker class that takes and uses that...

score 0 · Answer 4 · answered Dec 27 '12 at 14:34

The point should be how can I make best use of class inheritance to avoid code duplication? I would try to define an abstract class for the common pack/unpack methods:

class PackUnpack(object):  
    def pack(self):
        struct.pack(self.foo)
        ...
    def unpack(self):  
        self.foo = struct.unpack()
        ...

class Struct1(object, PackUnpack):
    ...

class Struct2(object, PackUnpack):
    ...

But even if this is not possible (or too cumbersome to implement) the first choice seems to be more natural and somehow easier to maintain.

Structures packing/unpacking class design

4 Answers4