Calling an overloaded Java method from Jython

Question

I'm seeing some strange behavior I don't understand when I call an overloaded Java method from a Jython script.

Here is my Java class:

public class TestClass {
  public static float[][][] overloaded(float[][][] x) {
    return x;
  }
  public static float[][][][] overloaded(float[][][][] x) {
    return x;
  }
  public static float[][][] zeros(int n1, int n2, int n3) {
    return new float[n3][n2][n1];
  }
}

and here is my Jython script:

import time,TestClass
n1,n2,n3 = 250,250,250
z = TestClass.zeros(n1,n2,n3)
start = time.time()
TestClass.overloaded([z,z,z])
print 'time =',(time.time()-start)

This Jython script takes about 1 minute to run, but if I comment out the first method in TestClass the script takes almost no time at all. I'm confused why it takes so much longer when the method is overloaded. Am I missing something here?

Favonius · Answer 1 · 2012-03-02T11:46:50.650

Your Code !!

import time,TestClass
n1,n2,n3 = 250,250,250
z = TestClass.zeros(n1,n2,n3)
start = time.time()
TestClass.overloaded([z,z,z])
print 'time =',(time.time()-start)

Facts !!

Jython is Java based (well we already know that !!)
When you do n1,n2,n3 = 250,250,250 and say z = TestClass.zeros(n1,n2,n3) then essentially you are allocating 250x250x250x32 bytes that is around 500000000 bytes or 477 megabytes. Where 32 is the size of float in Java.
When you say TestClass.overloaded([z,z,z]) then you are always going to call the 4 dimensional overloaded method !! Try it if you don't believe me !!

My code works fine !!

I just changed the TestClass.overloaded([z,z,z]) to x = TestClass.overloaded([z,z,z]). And the execution was really fast. But on printing 'x' it still fails!! why ?!

The 'why' part !!

It fails because when you do TestClass.overloaded([z,z,z]) or when I print 'x' because python or rather jython needs to convert the object in string representation and that is where the problem lies. See the stacktrace below:

java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOfRange(Arrays.java:3209)
        at java.lang.String.<init>(String.java:215)
        at java.lang.StringBuilder.toString(StringBuilder.java:430)
        at org.python.core.PyList.list_toString(PyList.java:472)
        at org.python.core.PyList.toString(PyList.java:450)
        at org.python.core.PyArray.toString(PyArray.java:395)
        at org.python.core.PyObject.__repr__(PyObject.java:174)
        at org.python.core.PyList.list_toString(PyList.java:464)
        at org.python.core.PyList.toString(PyList.java:450)
        at org.python.core.PyArray.toString(PyArray.java:395)
        at org.python.core.PyObject.__repr__(PyObject.java:174)

See.. The JVM is bombed !!!! Its out of heap space... Even if you change the JVM argument for memory and bless this program with more, even then your talking about 478 MB !! (Well it is not just 478 MB because you are passing an array of 'z' and each one of them is 478 MB !!!) and that is just what you have allocated, besides that JVM will need memory for the StringBuilder to save the string representation and some other things!!

And believe me it will take time and a good amount of time.

Just to give you some feel !!

>>> n1,n2,n3 = 2,2,2
>>> z = TestClass.zeros(n1,n2,n3)
>>> x = TestClass.overloaded([z,z,z])
>>> x

Output:

array([[[F, [array([[F, [array([F, [array('f', [0.0, 0.0]), array('f', [0.0, 0.0
])]), array([F, [array('f', [0.0, 0.0]), array('f', [0.0, 0.0])])]), array([[F,
[array([F, [array('f', [0.0, 0.0]), array('f', [0.0, 0.0])]), array([F, [array('
f', [0.0, 0.0]), array('f', [0.0, 0.0])])]), array([[F, [array([F, [array('f', [
0.0, 0.0]), array('f', [0.0, 0.0])]), array([F, [array('f', [0.0, 0.0]), array('
f', [0.0, 0.0])])])])

See the size of string and it is just for 2x2x2x32 bytes of array !! Take the code I have used and then change all the 2's with 20's.

But why it is taking time when you don't uncomment the first method !!!

Remember in order to resolve to correct overloaded function jython needs to evaluate the [z,z,z] which is good amount of memory. And that is where you are seeing that delay. When you comment the first method then there is no confusion about the call and hence it return instantaneously. If I use your code then it first takes to resolve the above mentioned expression and then calculate the string representation of the object. Combined, it will take long to become responsive again. But, if I use the modified version of your code i.e. x = TestClass.overloaded([z,z,z]) then it becomes a bit faster but will still take time in printing 'x' or may result in Heap Exception !!

Have fun !!