Convert java byte array to Go struct

Question

I have a system designed around a broker such that my producer is in Java and consumer in Go.

I am using apache-pulsar as my broker

Java - Producer

MessageJava class is converted to byte array before sending it to pulsar: An object of MessageJava class calls getBytes() method defined in same class to convert it to byte[] and then this array is sent to apache-pulsar

class MessageJava {
  String   id;
  int      entityId;
  Date     timestamp;

  public byte[] getBytes() throws Exception{
    ByteArrayOutputStream bos = new ByteArrayOutputStream();
    ObjectOutputStream oos = new ObjectOutputStream(bos);
    oos.writeObject(this);
    oos.flush();
    return bos.toByteArray();
  }
}

My consumer is written in Go.

Go - Consumer

byte array is read from pulsar and converted to MessageGo struct using ConvertAfterReceiving method [defined below], I am using gob for decoding:

type MessageGo struct {
    Id            string
    EntityId      int
    Timestamp     time.Time
}

func ConvertAfterReceiving(msg pulsar.Message) *MessageGo {
    payload := msg.Payload()
    messageBytes := bytes.NewBuffer(payload)
    dec := gob.NewDecoder(messageBytes)

    var message MessageGo
    err := dec.Decode(&message)
    if err != nil {
        logging.Log.Error("error occurred in consumer while decoding message:", err)
    }
    return &message
}

The issue is I am not able to decode byte[] and convert it to MessageGo struct. It shows error encoded unsigned integer out of range

I have tried changing MessageJava.entityId to short/long and MessageGo.EntityId to int8/int16/int32/int64/uint8/uint16/uint32/uint64 [all permutations], but all in vain.

There is no native Go parser for Java's Object stream - and even if there were, there is no `Date` type in Go to match up to Java's. Object stream is meant to be used within Java exclusively. You'll want to use some platform-independent format to pass data between Java and other languages, like JSON, BSON, XML, etc. — Adrian, Apr 10 '20 at 13:28
You should use either a binary byte-based protocol where both sides explicitly agree on the byte encoding (big-endian, how many bytes, etc.) or a text-based protocol where everything is in ASCII characters with lines terminated by specified characters. — NomadMaker, Apr 10 '20 at 15:59

ThisIsNoZaku · Answer 1 · 2020-04-10T15:29:13.680

A Java ObjectOutputStream and a Go Decoder do not speak the same language, even if at the base they're both made up of bytes; the same way that "these words" and "эти слова" are made up of lines yet knowing one doesn't let you know the other.

An ObjectOutputStream transforms objects into a form meant to be read by a Java ObjectInputStream, while a Go Decoder expects data in a format created by a Go Encoder.

What's needed is a language that they both speak like JSON, which both Java and Go know how to work with. Then instead of serializing the object straight into bytes, you transform it into a string representation, send the bytes of that string and convert that string in Go into the desired struct.

Convert java byte array to Go struct

1 Answers1