5

I have Java application, which intensively working with 2D float arrays (float[][] arrays), which actually holding images on black background. Both dimensions are equals (square) and are power of 2 (mostly are 256, 512, 1024), so areas close to borders have zeroes in most cases.

Having sizes equals to power of 2 done for increasing performance (there is some FFT) and decreasing complexity on operations over those arrays like rotation, etc. Recently I faced lack of heap for this application on my machine with 6Gb. By my calculations - memory consumption for this application should be like up to 2-3Gb, while it reaches 4-5Gb (looking in Windows task manager). I used "YourKit" profiler and it shows that those floats arrays indeed takes most memory, however, total rough size for these floats arrays should be like 1.3Gb (well, I know that it's up to JVM to decide how to store data, but I was not expecting 2-3-times difference in memory consumption).

I was trying to compress/decompress data with Snappy compressor on the fly (and memory consumption drops to 3.5Gb), but performance drops several times, which is not very acceptable. Also, I was testing performance when replacing those floats[][] by BufferedImage, but performance was very poor.

So, there is 2 ways left which will work for me to decrease memory consumption: 1) write wrappers for float[][] array in order to save on "zeroes" elements (there are a lot "empty" rows and columns) 2) go away from "power of 2"

Both ways require quite a lot of coding/refactoring, so while I am thinking "to be or not to be" - may be you have better clue on this issue, guys?

Thanks!

Robin Green
  • 32,079
  • 16
  • 104
  • 187
Arsen
  • 153
  • 1
  • 12
  • 1
    What do the `float`s represent? Why can't you make them `int`s instead, for example? – Robin Green Nov 07 '13 at 20:03
  • I'd pay more careful attention to how the arrays are being used, and whether you're getting any temporaries introduced, which would significantly boost memory usage. There are also some Java classes for sparse arrays that you might want to investigate. – Eric Brown Nov 07 '13 at 20:04
  • You could do a heap dump and use [MAT](http://www.eclipse.org/mat/) to see where the memory usage is coming from. – Thomas Nov 07 '13 at 20:04
  • Robin, those arrays holding images in RGB-format on black background (note that there are a lot of black rows and columns due to fact that arrays resided up to nearest power of 2. Those arrays are also used for FFT operations to compute 2D correlation. And, in JVM both float and int should normally "eat" 32-bit – Arsen Nov 07 '13 at 20:13
  • Have you looked into memory leaks? [This](http://java.jiderhamn.se/category/classloader-leaks/) has good posts about working with MAT like @Thomas suggested. Could you tell us more about the nature of your program? Is it a batch converter that runs once or does it run continuously? – Leonard Brünings Nov 07 '13 at 20:14
  • Eric, do you mean something like Trie (I did not check this deeply yet). Implementations based on hash maps will decrease performance a lot, I guess. – Arsen Nov 07 '13 at 20:23
  • Thomas, I used YourKit profiler, most memory consumption comes from those arrays, this is matches my calculations – Arsen Nov 07 '13 at 20:24
  • Leonard, I was not diving into memory leaks to deep, from first glance - there is no memory leaks, application should really have a lot of float[][] data in memory, however, I was not expecting that this is will be 2-3 times more that I was expected – Arsen Nov 07 '13 at 20:27
  • The simplest solution is to add memory. You can't switch to a sparse data structure without incurring a performance penalty. A sparse data structure will require code to decide if the element being sought exists with a real value or has the default background value. To design the correct sparse data structure would require a good understanding of access patterns (row-oriented? column-oriented? iteration? random access?). – Jim Garrison Nov 07 '13 at 20:27
  • Jim, I am afraid that you are right... However I would like to investigate why those floats[][] eats 2-3 times more than I was expecting (note, that when I compressed data on fly by Snappy - memory consumption was 1-2Gb less). So, this is likely not "other" object leaking... – Arsen Nov 07 '13 at 20:31
  • Maybe you could give us some code on how you are creating the float arrays. – Leonard Brünings Nov 07 '13 at 20:37
  • Leonard, about nature of program - it comparing images one to each other, with some preprocessing work. – Arsen Nov 07 '13 at 20:39
  • Leonard, not sure that this will help. I am creating arrays like float[][] f = new float[size][size]; - nothing new as you see :) I think that, before adding memory - as last resort, I need to investigate for possible leaks of those arrays (to ensure that "processed" arrays are really garbage collected) – Arsen Nov 07 '13 at 20:47
  • 1
    @Arsen how do you store an RGB image in a float[][], I can't help but believe you have a 3rd dimension for the color channels. Show us the actual declaration of such an array and comment on how pixels are stored in it. – Durandal Nov 07 '13 at 21:01
  • @Durandal Each float can be converted to RGB with kind of such code 
      public static float[] f2rgb(float rgbValue) {
    Validate.isTrue(rgbValue >= 0);
    int rgbValueInt = (int) rgbValue;
    int rValue = (rgbValueInt & 0x00FF0000) >> 16;
    int gValue = (rgbValueInt & 0x0000FF00) >> 8;
    int bValue =  rgbValueInt & 0x000000FF;
    return new float[] {rValue, gValue, bValue};
  }
    btw, it's possible to have Alpha-channel, but I do not using it     – Arsen Nov 07 '13 at 21:24
  • sorry, missed how to do right code formatting in comments... – Arsen Nov 07 '13 at 21:38
  • You can save a little bit by linearizing the arrays. And do remember that a copying GC must have "headroom" for the largest object, over and above total object size, in order to do the copy operation. – Hot Licks Nov 07 '13 at 23:29
  • So you're saying that the inner arrays are really dimensioned `[3]`?? Is the array already linearized for the X-Y dimensions? – Hot Licks Nov 07 '13 at 23:34
  • 1
    @Arsen So your array *is* three dimensional? That would easily explain the unexpected memory consumption (java multidimensional arrays are implemented as array of arrays). Since each array has some overhead (about 12 bytes), which for the innermost float[3] amounts to as much memory as the floats stored within consume. That is what your code sample suggests anyway. I specifically asked about an actual declaration of such an array because that would unambigously clear up what memory layout you really use. Playing guessing games doesn't help anyone. Update your question with all relevant details – Durandal Nov 08 '13 at 18:21
  • @Hot, no, array is 2D. Float value depends only on "x" and "y" (there is no "z" variable). Consider this is like a function f(x,y). In other words - using such structure is possible to draw surface, but not sphere, for example. Not sure what do you mean about "linearized" ? Can you please explain more ? – Arsen Nov 08 '13 at 19:44
  • @Durandal, unfortunately, data is 2D. I creating array like `float[][] f = new float[size][size];` Btw, about array overhead : 12 bytes looks comparatively small to data stored even in one row (smallest array is 256x256, so 256 floats theoretically will "eat" 1024 bytes). About details - I wrote example how I am creating array, I would like to post some more details, but can you please suggest what will help you ? – Arsen Nov 08 '13 at 19:48
  • I'm just asking how you manage fit RGB info for one pixel into one `float` value. – Hot Licks Nov 08 '13 at 22:53
  • @Hot, float memory consumption is 32 bits, and RGB needed 24 bits. So, here is how I am doing it : `public static float[] f2rgb(float rgbValue) { Validate.isTrue(rgbValue >= 0); int rgbValueInt = (int) rgbValue; int rValue = (rgbValueInt & 0x00FF0000) >> 16; int gValue = (rgbValueInt & 0x0000FF00) >> 8; int bValue = rgbValueInt & 0x000000FF; return new float[] {rValue, gValue, bValue}; } ` – Arsen Nov 09 '13 at 09:21
  • That makes absolutely no sense. You could easily cram the 3 int values into an `int`. As it is, you just barely squeak by with the 23+1 bit precision of the float (probably losing a bit on the bValue at times), but I'm guessing it's dumb luck rather than planning -- having the value as float just adds overhead. – Hot Licks Nov 09 '13 at 13:53
  • @Hot Licks, no. I am holding RGB data for one pixel in one float value. I may use `int` for this case also (since both int and float are 32 bits), but I need float arrays also for FFT (above example not really good, since I am returning array of floats with 3 elements, but please, be sure, I am holding all necessary data for one pixel in one float value) – Arsen Nov 09 '13 at 19:47
  • You are ALMOST holding the data for one pixel in the float. And it makes little difference if you're using float arrays elsewhere -- allocating new arrays is cheap. – Hot Licks Nov 09 '13 at 22:13
  • 1
    @Arsen The problem hotlicks and me too have with your explanation is: You show code that create an array of *3 floats* per pixel. You *tell* us the array is 2D indexed by (x, y). That doesn't fit together at all, agreed? Then you say one float per pixel, in which case it makes *even less sense*, because of the precision issue hot pointed out. Then you say you need a float for FFT, but a float holding *R, G and B* together can hardly be suited for FFT processing (precision issue aside) since you would need to split the color channels *anyway before processing*. Try being less confusing/confused – Durandal Nov 10 '13 at 16:50
  • @Arsen -- Yes, perhaps you're not understanding. You cannot convert the colors into a float the way you are and then do an FFT on an array of such values -- this simply will not work (or at least it will not produce meaningful results -- you'll basically only process the red, with added noise). There's no advantage to storing your RGB data in a float like that, and several disadvantages. – Hot Licks Nov 10 '13 at 23:19
  • @Durandal, Hot Licks : before FFT I am converting image to luminance (gray image), so, relax guys :) The fact that I am returning array with three floats array do not prevent fact that float-type can easily fit RGB data. I am returning three floats, because it's omits further types conversions. Anyway, thanks for your help - but we went long away from original topic. I already found issue, it's explained below. Thanks again! – Arsen Nov 16 '13 at 15:55

2 Answers2

1

An FFT requires a complex array, which is double the size of the real data array, even if you convert from a real array at the input and back to a magnitude array at the end. That may account for 2X of the larger-than-expected memory usage.

A sparse array will not work for an FFT, since the intermediate steps in an FFT will almost always fill the entire complex array.

Many modern high-performance FFT libraries, such as ones based on FFTW, can very efficiently deal with FFT lengths other than just powers-of-2 (any length that is the product of just small primes can be FFT'd quite efficiently). This can save a lot of 2D padding for many sizes.

hotpaw2
  • 70,107
  • 14
  • 90
  • 153
  • true, before making FFT (using JTransfrom, also tried Cuda) - I am converting float[][] array to Re and Im values (so, total array length increased two times), but this done only one time and after computing correlation I am getting absolute and array back reduced to normal size. GC should collect those Re-Im arrays, I have no links on them anywhere. About beneficence of FFT libs, thanks will have in mind – Arsen Nov 07 '13 at 21:36
0

After more detailed investigation - it appeared that JVM was launching with "UnlockEperimentalFeatures" and "use GC1" flags. As a result - there were a quite a lot of NOT garbage collected "unreachable" BufferedImage rasters (which contains byte[] arrays). When invoking GC from "YourKit" priofiler - those objects removed from heap (this is of course not acceptable way for me, since I was expecting that JVM will manage heap by herself).

I want to say thanks to everybody who put his time helping me. Special thanks to Jim Garrison (it looks like I just postponed memory demand for some time removing flags mentioned above, but when more arrays will came to play - buying more memory will be most easy way to avoid performance penalty.

Arsen
  • 153
  • 1
  • 12