ObjectStream: is there a way to read serialized object as properties map?

Question

Is there any standard way or any utility library to read/navigate through serialized (via ObjectOutputStream) object's properties?

The problem I'm trying to solve is to upgrade data which was serialized using ObjectOutputStream (legacy) and stored in database. In my case some internal fields were drastically changed and renamed. I cannot read object back using ObjectInputStream, as values of changed fields would be lost (set to null).

In particular there may be need to upgrade it again in future, so it would be better if I could replace old data stored this way with XML serialization. But the general way to accomplish this task would require to iterate through properties (their names, types and values). I wasn't able to find a standard way to read such metadata from serialized data (for example, jackson library could read JSON as an object or as a map of properties and maps, which you can easily manipulate).

Is there any low-level library to work with data, serialized with ObjectOutputStream? Resulting output looks like it contains information about serialized field names and their types. As a last resort I could sort out the format, but I've thought that someone could have done this already, yet I wasn't able to find any libraries myself.

For example, I had a class

public class TestCase implements Serializable
{
    int id;
    double doubleValue;
    String stringValue;

    public TestCase(int id, double doubleValue, String stringValue)
    {
        this.id = id;
        this.doubleValue = doubleValue;
        this.stringValue = stringValue;
    }
}

which was changed to

public class TestCase implements Serializable
{
    ComplexId id;
    double doubleValue;
    String stringValue;

    public TestCase(ComplexId id, double doubleValue, String stringValue)
    {
        this.id = id;
        this.doubleValue = doubleValue;
        this.stringValue = stringValue;
    }
}

class ComplexId implements Serializable
{
    int staticId;
    String uuid;

    public ComplexId(int staticId, String uuid)
    {
        this.staticId = staticId;
        this.uuid = uuid;
    }
}

It is not a problem to upgrade the value itself, I just don't know how to peek it and put back a new one (with a new type) without custom implementation of serialization/deserialization protocol (that's a last resort for me).

Answer 1

If you have version control system with original .java files compile them, read information using ObjectInputStream.

Another option is manually reading byte data according to Object Serialization Stream Protocol and Useful information about serialization.

I've written this sample to prove that deserialization without class file is possible. Inheritance isn't supported. It works only with primitive type fields and java.lang.String.

class CustomDeserialization {

    public static class A implements Serializable {
        private static final long serialVersionUID = 123124345135L;

        int foo = 1;
        String bar = "baz";
    }

    private byte[] bytes;
    private int cursor;

    CustomDeserialization(byte[] bytes) {
        this.bytes = bytes;
    }

    private List<List<Object>> parse() {
        cursor = 2; //skip STREAM_MAGIC
        short classNameLength = getShort();
        String className = getString(classNameLength);
        cursor += 9; //skip serialVersionUID and flag tells object supports serialization
        short numberOfFields = getShort();
        List<List<Object>> result = new ArrayList<>();
        List<Character> types = new ArrayList<>();
        List<Object> values = new ArrayList<>();
        List<String> classNames = new ArrayList<>();
        for (int fieldIndex = 0; fieldIndex < numberOfFields; fieldIndex++) {
            char c = getCharType();
            types.add(c);
            short fieldNameLength = getShort();
            String fieldName = getString(fieldNameLength);
            List<Object> objects = new ArrayList<>();
            if (c == 'L') {
                byte objectType = getByte();
                if (objectType == ObjectStreamConstants.TC_REFERENCE) {
                    getShort();
                    objects.add(classNames.get(getShort() - 1));
                } else {
                    short fieldClassNameLength = getShort();
                    String fieldClassName = getString(fieldClassNameLength);
                    classNames.add(fieldClassName);
                    objects.add(fieldClassName);
                }
            } else {
                Class clazz = getCorrectType(c);
                objects.add(clazz);
            }
            objects.add(fieldName);
            result.add(objects);
        }
        cursor += 2; //skip TC_ENDBLOCKDATA & TC_NULL
        for (int fieldIndex = 0; fieldIndex < numberOfFields; fieldIndex++) {
            result.get(fieldIndex).add(getValue(types.get(fieldIndex), values));
        }
        return result;
    }

    private String getString(int lengthOfClassName) {
        String s = new String(Arrays.copyOfRange(bytes, cursor, cursor + lengthOfClassName));
        cursor += lengthOfClassName;
        return s;
    }

    private char getCharType() {
        char c = (char) (bytes[cursor] & 0xFF);
        cursor++;
        return c;
    }

    private char getChar() {
        ByteBuffer bb = ByteBuffer.allocate(2);
        bb.order(ByteOrder.LITTLE_ENDIAN);
        bb.put(bytes[cursor + 1]);
        bb.put(bytes[cursor]);
        cursor += 2;
        return bb.getChar(0);
    }

    private short getShort() {
        ByteBuffer bb = ByteBuffer.allocate(2);
        bb.order(ByteOrder.LITTLE_ENDIAN);
        bb.put(bytes[cursor + 1]);
        bb.put(bytes[cursor]);
        cursor += 2;
        return bb.getShort(0);
    }

    private double getDouble() {
        ByteBuffer bb = ByteBuffer.allocate(8);
        bb.order(ByteOrder.LITTLE_ENDIAN);
        bb.put(bytes[cursor + 7]);
        bb.put(bytes[cursor + 6]);
        bb.put(bytes[cursor + 5]);
        bb.put(bytes[cursor + 4]);
        bb.put(bytes[cursor + 3]);
        bb.put(bytes[cursor + 2]);
        bb.put(bytes[cursor + 1]);
        bb.put(bytes[cursor]);
        cursor += 8;
        return bb.getDouble(0);
    }

    private long getLong() {
        ByteBuffer bb = ByteBuffer.allocate(8);
        bb.order(ByteOrder.LITTLE_ENDIAN);
        bb.put(bytes[cursor + 7]);
        bb.put(bytes[cursor + 6]);
        bb.put(bytes[cursor + 5]);
        bb.put(bytes[cursor + 4]);
        bb.put(bytes[cursor + 3]);
        bb.put(bytes[cursor + 2]);
        bb.put(bytes[cursor + 1]);
        bb.put(bytes[cursor]);
        cursor += 8;
        return bb.getLong(0);
    }

    private byte getByte() {
        byte b = bytes[cursor];
        cursor++;
        return b;
    }

    private int getInt() {
        ByteBuffer bb = ByteBuffer.allocate(4);
        bb.order(ByteOrder.LITTLE_ENDIAN);
        bb.put(bytes[cursor + 3]);
        bb.put(bytes[cursor + 2]);
        bb.put(bytes[cursor + 1]);
        bb.put(bytes[cursor]);
        cursor += 4;
        return bb.getInt(0);
    }

    private float getFloat() {
        ByteBuffer bb = ByteBuffer.allocate(4);
        bb.order(ByteOrder.LITTLE_ENDIAN);
        bb.put(bytes[cursor + 3]);
        bb.put(bytes[cursor + 2]);
        bb.put(bytes[cursor + 1]);
        bb.put(bytes[cursor]);
        cursor += 4;
        return bb.getFloat(0);
    }

    private boolean getBoolean() {
        boolean b = bytes[cursor] == 1;
        cursor++;
        return b;
    }

    private Class getCorrectType(char type) {
        switch (type) {
            case 'B':
                return byte.class;
            case 'C':
                return char.class;    // char
            case 'D':
                return double.class;    // double
            case 'F':
                return float.class;    // float
            case 'I':
                return int.class;    // integer
            case 'J':
                return long.class;    // long
            case 'S':
                return short.class;    // short
            case 'Z':
                return boolean.class;    // boolean
            case 'L':
                return Object.class;
        }
        throw new IllegalArgumentException();
    }

    private Object getValue(char type, List<Object> values) {
        switch (type) {
            case 'B':
                byte b = getByte();
                values.add(b);
                return b;
            case 'C':
                char c = getChar();
                values.add(c);
                return c;    // char
            case 'D':
                double d = getDouble();
                values.add(d);
                return d;    // double
            case 'F':
                float f = getFloat();
                values.add(f);
                return f;    // float
            case 'I':
                int i = getInt();
                values.add(i);
                return i;    // integer
            case 'J':
                long l = getLong();
                values.add(l);
                return l;    // long
            case 'S':
                short s = getShort();
                values.add(s);
                return s;    // short
            case 'Z':
                boolean b1 = getBoolean();
                values.add(b1);
                return b1;    // boolean
            case 'L':
                byte objectType = getByte();
                if (objectType == ObjectStreamConstants.TC_REFERENCE) {
                    getShort(); // skip 2 bytes
                    return values.get(getShort());
                } else {
                    short stringValueLength = getShort();
                    String string = getString(stringValueLength);
                    values.add(string);
                    return string;
                }
        }
        throw new IllegalArgumentException();
    }

    public static void main(String[] args) {
        A a = new A();
        try {
            File file = new File("temp.out");
            try (FileOutputStream fos = new FileOutputStream(file);
                 ObjectOutputStream oos = new ObjectOutputStream(fos);) {
                oos.writeObject(a);
                oos.flush();
                oos.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
            return;
        }
        try {
            try (FileInputStream fis = new FileInputStream("temp.out");
                 ObjectInputStream ois = new ObjectInputStream(fis);
                 ByteArrayOutputStream buffer = new ByteArrayOutputStream();) {
                int cursor;
                byte[] data = new byte[8192];
                while ((cursor = fis.read(data, 0, data.length)) != -1) {
                    buffer.write(data, 0, cursor);
                }
                byte[] bytes = buffer.toByteArray();

                List<List<Object>> result = new CustomDeserialization(bytes).parse();
                result.forEach(list -> {
                    list.forEach(o -> System.out.print(o + " "));
                    System.out.println();
                });
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Answer 2

I've compiled old version of needed classes and changed ClassLoader to load them in upgrade, read object with ObjectStream and serialized it using XML. Then I've added fix for XML structure.

I can add the code with ClassLoader hack if needed, but AFAIR it was somewhere on Stack Overflow.