I have a JNA application that needs to compute the sum of elements in a large 1D array (~1G). Using the JNA Resource.Rlimit class I am able to increase the resource limit as shown in the following snippet:
final Resource.Rlimit limit = new Resource.Rlimit();
limit.rlim_cur = 1024L * 1024L * 1024L;
limit.rlim_max = 2 * 1024L * 1024L * 1024L;
However, this only works when the stack size is set inside the method allocating the large array. Thus, when I run the below code, there is no out of memory exception.
package com.jnaapps;
import com.sun.jna.Library;
import com.sun.jna.Pointer;
import com.sun.jna.Native;
import com.sun.jna.Memory;
import com.sun.jna.platform.unix.LibCAPI;
import com.sun.jna.platform.unix.Resource;
public class Main {
public interface CLibrary extends LibCAPI, Library {
CLibrary INSTANCE = (CLibrary) Native.load("c", CLibrary.class);
}
public interface NativeClass extends Library {
NativeClass INSTANCE = (NativeClass)Native.load("nativemethods", NativeClass.class);
double sumOfArray1d(final Pointer values, long numberOfValues);
}
public static void main(String[] args) {
Test0();
}
public static void Test0() {
final Resource.Rlimit limit = new Resource.Rlimit();
limit.rlim_cur = 1024L * 1024L * 1024L;
limit.rlim_max = 2 * 1024L * 1024L * 1024L;
int err = CLibrary.INSTANCE.setrlimit(Resource.RLIMIT_STACK, limit);
if (err != 0) {
int lastError = Native.getLastError();
System.out.println("An exception occurred while setting RLimit: " +
lastError);
}
NativeClass clib = NativeClass.INSTANCE;
long dim = 10000L * 10000L;
final Pointer bigVector = new Memory(dim*Native.getNativeSize(Double.TYPE));
for (long i=0; i<dim; ++i) {
bigVector.setDouble(i*Native.getNativeSize(Double.TYPE), 3.5e-6);
}
double total = clib.sumOfArray1d(bigVector, dim);
System.out.println("Sum of elements in big vector = " + total);
}
public static void Test1() {
NativeClass clib = NativeClass.INSTANCE;
long dim = 10000L * 10000L;
final Pointer bigVector = new Memory(dim*Native.getNativeSize(Double.TYPE));
for (long i=0; i<dim; ++i) {
bigVector.setDouble(i*Native.getNativeSize(Double.TYPE), 3.5e-6);
}
double total = clib.sumOfArray1d(bigVector, dim);
System.out.println("Sum of elements in big vector = " + total);
}
}
But when the resource limits are set inside main() and method Test1() is called, the program crashes.
public static void main(String[] args) {
final Resource.Rlimit limit = new Resource.Rlimit();
limit.rlim_cur = 1024L * 1024L * 1024L;
limit.rlim_max = 2 * 1024L * 1024L * 1024L;
int err = CLibrary.INSTANCE.setrlimit(Resource.RLIMIT_STACK, limit);
if (err != 0) {
int lastError = Native.getLastError();
System.out.println("An exception occurred while setting RLimit: " +
lastError);
}
Test1();
}
Why can't I increase the stack size in main() before the call to Test1(). The project was written in IntelliJ Idea (Community Version 2019.3) on Ubuntu 16.04 amdx86-64 with 8GB RAM.