1

I'm working on a Halide project processing bursts of images. My initial data set is 9 bursts of images of 4208*3120 pixel, in format uint16_t. I got the constant_allocation_size error during run time:

"Error:

Total size for allocation layer_0 is constant but exceeds 2^31 - 1.
Aborted (core dumped)".

where layer_0 is a down-sample function scheduled with compute_root();

The sample code is like this:

Func my_process(Func input, ...){
  <do something here...> 
  output.compute_root().parallel(y).vectorize(x, 16);
}

Func raw2gray(Func input, std::string name){
   ...
   compute_root();
}

Func imgs_mirror = BoundaryConditions::mirror_interior(imgs, 0, imgs.width(), 0, imgs.height());
Func layer_0 = raw2gray(imgs_mirror, "layer_0");
Func out_imgs = my_process(layer_0, ...);

Does anyone know how to shrink my constant allocation size or how to increase this 2^31-1 upper limit?

Derrick
  • 3,669
  • 5
  • 35
  • 50
shuyun
  • 11
  • 4

2 Answers2

0

I figured it out myself. Here to post my solutions in case anyone else comes across a similar issue.

This is caused by my function boundary clarifications. Since I though it would be sufficient by putting BoundaryConditions to the input image, I did not care about function boundary problem in my latter process. However, since my latter function processes image hierarchically and manipulates several pixels' coordinates in-between, the boundary limit calculated back to the original image would be surprisingly large, which lead to the mirrored image to be as large as several 8GBs.

After adding a clamp each time manipulating the coordinates, this problem is solved.

shuyun
  • 11
  • 4
0

If you really do need more than 2^31-1 elements, you can add Target::LargeBuffers to the compilation target you use when compiling your pipeline (JIT or AOT); this increases the limit to 2^63-1 elements.

Steven Johnson
  • 266
  • 1
  • 4
  • Thanks a lot! Actually, I do not need more than 2^31 -1 elements. It was because I did not add boundary clamp in my latter function process, which in turn made the compiler infers that I need more than 2^31-1 elements. – shuyun Aug 27 '18 at 09:36