Bilinear Interpolation from C to Neon

Question

I'm trying to downsample an Image using Neon. So I tried to exercise neon by writing a function that subtracts two images using neon and I have succeeded. Now I came back to write the bilinear interpolation using neon intrinsics. Right now I have two problems, getting 4 pixels from one row and one column and also compute the interpolated value (gray) from 4 pixels or if it is possible from 8 pixels from one row and one column. I tried to think about it, but I think the algorithm should be rewritten at all ?

void resizeBilinearNeon( uint8_t *src, uint8_t *dest,  float srcWidth,  float srcHeight,  float destWidth,  float destHeight)
{

    int A, B, C, D, x, y, index;

       float x_ratio = ((float)(srcWidth-1))/destWidth ;
       float y_ratio = ((float)(srcHeight-1))/destHeight ;
       float x_diff, y_diff;

       for (int i=0;i<destHeight;i++) {
          for (int j=0;j<destWidth;j++) {
               x = (int)(x_ratio * j) ;
               y = (int)(y_ratio * i) ;
               x_diff = (x_ratio * j) - x ;
               y_diff = (y_ratio * i) - y ;
               index = y*srcWidth+x ;

               uint8x8_t pixels_r = vld1_u8 (src[index]);
               uint8x8_t pixels_c = vld1_u8 (src[index+srcWidth]);

               // Y = A(1-w)(1-h) + B(w)(1-h) + C(h)(1-w) + Dwh
               gray = (int)(
                           pixels_r[0]*(1-x_diff)*(1-y_diff) +  pixels_r[1]*(x_diff)*(1-y_diff) +
                           pixels_c[0]*(y_diff)*(1-x_diff)   +  pixels_c[1]*(x_diff*y_diff)
                           ) ;

               dest[i*w2 + j] = gray ;
           }
  }  

Answer 1

Neon will definitely help with downsampling in an arbitrary ratio using bilinear filtering. The key being clever use of vtbl.8 instruction, that is able to perform a parallel look-up-table for 8 consecutive destination pixels from pre-loaded array:

 d0 = a [b] c [d] e [f]  g  h, d1 =  i  j  k  l  m  n  o  p 
 d2 = q  r  s  t  u  v  [w] x, d3 = [y] z [A] B [C][D] E  F ...
 d4 = G  H  I  J  K  L   M  N, d5 =  O  P  Q  R  S  T  U  V ...

One can easily calculate the fractional positions for the pixels in brackets:

 [b] [d] [f] [w] [y] [A] [C] [D],  accessed with vtbl.8 d6, {d0,d1,d2,d3}
 The row below would be accessed with            vtbl.8 d7, {d2,d3,d4,d5} 

Incrementing vadd.8 d6, d30 ; with d30 = [1 1 1 1 1 ... 1] gives lookup indices for the pixels right of the origin etc.

There's no reason for getting the pixels from two rows other than illustrating it's possible and that the method can be used to implement also slight distortions if needed.

In real time applications using e.g. of lanzcos can be a bit overkill, but still feasible using NEON. Downsampling of larger factors require of course (heavy) filtering, but can be easily achieved with iteratively averaging and decimating by 2:1 and only at the end using fractional sampling.

For any 8 consecutive pixels to write, one can calculate the vector

  x_positions = (X + [0 1 2 3 4 5 6 7]) * source_width / target_width;
  y_positions = (Y + [0 0 0 0 0 0 0 0]) * source_height / target_height;

  ptr = to_int(x_positions) + y_positions * stride;
  x_position += (ptr & 7); // this pointer arithmetic goes only for 8-bit planar
  ptr &= ~7;               // this is to adjust read pointer to qword alignment

  vld1.8 {d0,d1}, [r0]
  vld1.8 {d2,d3], [r0], r2 // wasn't this possible? (use r2==stride)

  d4 = int_part_of (x_positions);
  d5 = d4 + 1;
  d6 = fract_part_of (x_positions);
  d7 = fract_part_of (y_positions);

  vtbl.8 d8,d4,{d0,d1}  // read top row
  vtbl.8 d9,d5,{d0,d1}  // read top row +1
  MIX(d8,d9,d6)             // horizontal mix of ptr[] & ptr[1]
  vtbl.8 d10,d4,{d2,d3} // read bottom row
  vtbl.8 d11,d5,{d2,d3} // read bottom row
  MIX(d10,d11,d6)           // horizontal mix of ptr[1024] & ptr[1025]
  MIX(d8,d10,d7)

  // MIX (dst, src, fract) is a macro that somehow does linear blending
  // should be doable with ~3-4 instructions

To calculate the integer parts, it's enough to use 8.8 bit resolution (one really doesn't have to calculate 666+[0 1 2 3 .. 7]) and keep all intermediate results in simd register.

Disclaimer -- this is conceptual pseudo c / vector code. In SIMD there are two parallel tasks to be optimized: what's the minimum amount of arithmetic operations needed and how to minimize unnecessary shuffling / copying of data. In this respect too NEON with three register approach is much better suited to serious DSP than SSE. The second respect is the amount of multiplication instruction and the third advantage the interleaving instructions.

Answer 2

@MarkRansom is not correct about nearest neighbor versus 2x2 bilinear interpolation; bilinear using 4 pixels will produce better output than nearest neighbor. He is correct that averaging the appropriate number of pixels (more than 4 if scaling by > 2:1) will produce better output still. However, NEON will not help with image downsampling unless the scaling is done by an integer ratio.

The maximum benefit of NEON and other SIMD instruction sets is to be able to process 8 or 16 pixels at once using the same operations. By accessing individual elements the way you are, you lose all the SIMD benefit. Another problem is that moving data from NEON to ARM registers is a slow operation. Downsampling images is best done by a GPU or optimized ARM instructions.