0

I have a C++ program and a Java program. On the C++ side, I have a vector having millions of entries. I need to transfer this data to my Java program. So far, I have tried the following:

  • Created a unix socket, converted the vector to a long string (Serialized) and send it through the unix socket

  • Created a thrift server/client model to transfer the data

Both approaches worked very well, but the performance I'm getting is quite low. I don't even see it using the full network bandwidth (in the case of thrift).

Also with the unix socket approach, Since I'm serializing it to String and then again converting this string back to a string array (received byte[] to String and split) on the Java side is very expensive operation.

What is the best way to transfer data faster from the C++ world to the Java world with lesser overhead on reconstructing/serializing the object?

AkhlD
  • 2,596
  • 2
  • 16
  • 15
  • Are both programs running on the same machine? – Andreas Fester Mar 08 '16 at 11:10
  • Turning it into one String and then sending the whole String at once cannot be the fastest. Writing data chunks directly taken from Array should be Taster, but you'll have to care about the nature of the represented data somehow. Hard to talk about without knowing details. – kratenko Mar 08 '16 at 11:16

1 Answers1

2

If both problems are on the same machine, I would use a shared memory mapped file.

This way both programs can access the data at full memory speed without serialization/deserialization esp if the values are int, long or double values.

You can place the file on a tmpfs, or ram drive to avoid hitting a hard drive.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130