Space-efficient (de)serialise array of MyClass instances

Question

My object looks like this:

class Note(object):
    def __init__(self, note, vel, t, tOff=0):
        self.note = note  # ubyte
        self.vel = vel    # ubyte
        self.t = t        # float
        self.tOff = tOff  # float

(The type indications show the precision required for each field, rather than how Python is actually storing these!)

My program constructs an array of possibly several thousand Note-s.

I need to convert this array into a string, so that I can AJAX it to the server for storage (and subsequently retrieve and convert back to the original data structure).

I'm using Brython which implements Python's JSON library (I've tested: import json works. So I suspect JSON is my best bet.

But Brython is not a full CPython implementation, so I probably can't import non-core libraries. And it looks as though I can't do anything fancy like use slots to make for a storage-efficient class. (Brython maps Python constructs onto appropriate JavaScript constructs).

In theory I should be able to get each note down to 10 bytes, but I am aiming for lean code offering reasonably compact storage rather than ultimate compactness. I would however like to avoid massive inefficiency such as storing each note as a keyvalue pair -- i.e. the keys would be getting duplicated.

If I could see the range of solutions available to me, I could choose an appropriate complexity vs compactness trade-off. That is to say, I would be grateful for an answer anywhere on the continuum.

Answer 1

A quick test using struct seems to give you a possible length of 12 bytes as follows:

import struct

class Note(object):
    def __init__(self, note, vel, t, tOff=0):
        self.note = note  # ubyte
        self.vel = vel    # ubyte
        self.t = t        # float
        self.tOff = tOff  # float

    def pack(self):
        return struct.pack("BBff", self.note, self.vel, self.t, self.tOff)

    def unpack(self, packed):
        self.note, self.vel, self.t, self.tOff = struct.unpack("BBff", packed)

note = Note(10, 250, 2.9394286605624826e+32, 1.46971433028e+32)
packed = note.pack()
print "Packed length:", len(packed)

note2 = Note(0,0,0)
note2.unpack(packed)

print note2.note, note2.vel, note2.t, note2.tOff

This displays:

Packed length: 12
10 250 2.93942866056e+32 1.46971433028e+32

You might be able to further compact it depending on the type of floats you need, i.e. is some kind of fixed point possible?

To pack a list of notes, something like the following could be used:

notes = [1,2,3,4,5,6,7,8,9,10]
print struct.pack("{}B".format(len(notes)), *notes)

But the unpack needs to then be the same length. Or you could add a length byte to aid with unpacking:

notes = [1,2,3,4,5,6,7,8,9,10]
packed = struct.pack("B{}B".format(len(notes)), len(notes), *notes)

length = struct.unpack("B", packed[0])[0]
print struct.unpack("{}B".format(length), packed[1:])

This would display the correctly unpacked data:

(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)