Decoding Variant bytes from Godot to javascript object

Question

I'm trying to deserialize a byte array which I believe is a Uint8 array from Godot into a javascript object in a node process.

My current implementation is :

const message = Buffer.from(buffer.data, 'ascii');
console.log('message', message.toString('utf8'))

This is outputting stuff like : �a@

How can I deserialize this object without Godot's bytes2var in a node process?

Answer 1

var2bytes encodes type information with the value. If data in the PoolByteArray represents a utf8 string use String's method to_utf8() rather than var2bytes.

There is also the to_ascii() method on String.

var string = "hello"
var bytes = string.to_utf8()

If you don't have access to serialization process in Godot and the buffer payload is intended to contain ascii or utf8:

First var2bytes string encoding is straightforward. The first 4 bytes represent the variant type and encoding flags.

The second 4 bytes represent the length of the utf8 encoded string.

The remaining bytes are the utf8 encoded string. The buffer may be padded at 4 byte offsets.

const lengthOffset = 4
const messageOffset = lengthOffset * 2
const messageLength = buffer.readUInt32LE(lengthOffset)
const messageEndOffset = messageLength + messageOffset
const message = buffer.slice(messageOffset, messageEndOffset)
console.log(message.toString('utf8'))

---

Decoding Variant Arrays of Multiple Types

Decoding an array of variants is more involved. You will have to reimplement decode_variant for each supported type. I'll get you started with decoding an arbitrary sized array of ints, String's, and Vector3's.

Resources:

1. Marshalls decode_variant
2. Variant Encode Values
3. Variant Type Values

The following code assumes that buffer is valid.

// These values are from resource 2.
const variantEncodeMask = 0xFF
const variantEncodeFlag64 = 1 << 16

// These values are from resource 3.
const variantTypeInt = 2
const variantTypeString = 4
const variantTypeVector3 = 7
const variantTypeArray = 19

// Transcribed from resource 1.
function decodeVariant(buffer, _track = {offset: 0}) {
    const typeAndFlags = buffer.readUInt32LE(_track.offset)
    const type = typeAndFlags & variantEncodeMask
    _track.offset += 4

    // Buffer.slice does not copy memory. This makes the code a little easier to
    // read since every read* doesn't need the _track.offset parameter.
    const bufferSlice = buffer.slice(_track.offset)


    let result

    // The following switches on each type case. It's almost a one-to-one
    // implementation of the decode_variant function defined in marshalls.cpp.
    //
    // The main difference here is that we use _track to track the buffer offset
    // rather than using pointer arithmetic.
    //
    // Extend this by adding a case for the desired variant type. Follow godot's
    // implementation and make sure to increment _track.offset by the correct
    // byte count for every read* operation.
    switch (type) {
        case variantTypeInt:
            if (typeAndFlags & variantEncodeFlag64) {
                result = bufferSlice.readBigUInt64()
                _track.offset += 8
            } else {
                result = bufferSlice.readUInt32LE()
                _track.offset += 4
            }
            break

        case variantTypeVector3:
            result = {
                x: bufferSlice.readFloatLE(),
                y: bufferSlice.readFloatLE(4),
                z: bufferSlice.readFloatLE(8)
            }
            _track.offset += 4 * 3
            break

        case variantTypeString:
            const length = bufferSlice.readUInt32LE()
            const offset = 4
            const endOffset = length + offset
            const payload = bufferSlice.slice(offset, endOffset)
            let pad = 0
            if (length % 4) {
                pad = 4 - length % 4
            }
            result = payload.toString('utf8')

            _track.offset += endOffset + pad
            break

        case variantTypeArray:
            let count = bufferSlice.readUInt32LE()
            count &= 0x7FFFFFFF // Don't know why array needs this.
            _track.offset += 4

            result = new Array(count)
            for (let i = 0; i < count; i++) {
                result[i] = decodeVariant(buffer, _track)
            }
            break

    }

    return result
}