-2

our HFT trading application is developed in Java. We need to convert 8 bytes to a double number but the conversion has to be extremely fast. Remember our latency is under 15 micro second, so conventional java approaches will probably not work for us. Any suggestion would be helpful. Thanks in advance

[edit]

We have already worked on few approaches like byetebuffer, nio classes, twiddling approach (using shifting) etc. However, I would like to re-iterate the need to have each operation as fast as possible. Using these forms of conversions are expensive (even though they may seem faster as a standalone) & we end up introducing latency into the application. I am seeking out-of-the-box ideas (& not limited to the APIs provided by Java) which any one may have had experience on using bytes in different formats (int, double, float, etc). A thought in that direction is probably caching the double value and compare with the bytes received hence eliminating the need to convert.

Willem Van Onsem
  • 443,496
  • 30
  • 428
  • 555
Neha Sharma
  • 153
  • 1
  • 5
  • Could you add a java approach for better understanding of the problem. – thepace Jan 07 '15 at 07:10
  • 3
    It's perfectly clear what OP is asking. "What is the fastest way to convert byte-by-byte representations of doubles into doubles in Java?" You might pick on him/her for not doing any research first, but you certainly can't for clarity. It's a common, specific question with an immediately clear "right answer" if you know the technologies involved. – BadZen Jan 07 '15 at 07:39
  • @BadZen Not really. Your answer is good (in that it is a non hacky standard Java solution), but considering the speed required with HFT, even your approach can be improved. However then you end up substituting cleanliness and reliability for speed. – Kayaman Jan 07 '15 at 07:46
  • I don't see how. Direct-allocated buffers are just about as fast as a native C cast plus the Java object overhead. If that latter time is killing you, you don't want to be using Java. What other faster solution do you propose that "substitutes cleanliness and reliability for speed"? – BadZen Jan 07 '15 at 08:40
  • @Neha - Can you describe exactly what you tried with nio? Of course if you do something like load all of your data into a ByteBuffer from a Java array, and then convert, and then load into a double array, that won't be any faster. You need to work with your channels, or use a specialized matrix library. Exactly how did you use nio? And how is your data stored? – BadZen Jan 07 '15 at 08:42
  • @BadZen - data is available in a byte[] out of which 8 bytes are relevant. These 8 bytes were fed into ByteBuffer & allowed it to convert it for us (asDoubleBuffer()). Something similar to the code snippet you mentioned below. Can you elaborate more on matrix library or any other unconventional methods? – Neha Sharma Jan 07 '15 at 10:48
  • The "conventional" method would be never to have your data in a byte[]. It's the loading into the buffer that is taking the time in your use case. To take advantage of nio, you need to always keep it in the buffer/channel. Or you could use something like a JNI interface to a native BLAS implementation (NIST has a decent one: http://math.nist.gov/javanumerics/blas.html) It depends what kind of math you're doing if it's worth it to do this latter thing. – BadZen Jan 07 '15 at 15:08
  • You probably want Double.doubleToRawLongBits() and Double.longBitsToDouble(). I do HFT work, and that is what I use when I have to, but mostly, I stay away from floating point and use fixed point decimal notation instead (eg "100230000" represents $100.23 with an understood 10^-6 in there). – JasonN Jan 10 '15 at 23:19

1 Answers1

1

You'll likely want to be using java.nio for IO, in which case you can convert between ByteBuffer and DoubleBuffer to accomplish what you need. This will be much faster than bit twiddling with the logic for the conversion yourself.

It's as easy as:

ByteBuffer buffer.allocateDirect(<size>);
... add to the buffer ...
DoubleBuffer db = buffer.asDoubleBuffer();

voila.

BadZen
  • 4,083
  • 2
  • 25
  • 48
  • You should probably already have all of your data in channels/buffers if you're working with that level of real-time requirement anyhow, as painful as it might be to refactor everything... – BadZen Jan 07 '15 at 07:15
  • Thanks for the quick response, we have updated more details about the problem. – Neha Sharma Jan 07 '15 at 08:06