Does Mat in hls:Mat really represents a matrix?

Question

I am working on Vivado HLS. I am reading an image via stream and storing it in hls:mat. I want to perform an element-wise operation on this mat. Does mat really represent a matrix? Is there a way in which I can access it like a Matrix i.e. A[rows][columns]?

Method A.at<double>(0,0) is not working.

score 2 · Answer 1 · answered Jan 24 '18 at 07:44

2

No, according to Xilinx application note XAPP1167:

A second limitation is that the hls::Mat<> datatype used to model images is internally defined as a stream of pixels, using the hls::stream<> datatype, rather than as an array of pixels in external memory. As a result, random access is not supported on images, and the cv::Mat<>.at() method and cvGet2D() function have no corresponding equivalent function in the synthesizable library.

So you can only stream data to/from hls::Mat and you cannot access a random element.

answered Jan 24 '18 at 07:44

haggai_e

4,689
1
24
37

But I found something called as memory window. Memory Windows In video and image processing, a memory window is defined as a neighborhood of N pixels centered on pixel P. The memory window can also be viewed as a collection of shift registers, which forms a 2-dimensional data storage element. – A.k. Jan 24 '18 at 08:59
This kind of memory is usually implemented as flip-flops for these reasons: • It has fewer data elements. Only the pixels required to compute some characteristic of P are stored. An example of this is the 3 x 3 memory windows used in edge detection. • All pixels in the neighborhood must be simultaneously available when computing the value of P. The most basic definition of a memory window in C/C++ is a 2-D array. For example, a 3 x 3 memory window B can be defined as: `int B[3][3];` – A.k. Jan 24 '18 at 09:01
These memory windows sound like a useful construct, provided you can express your algorithm using them. – haggai_e Jan 24 '18 at 10:03
I want to find connected objects in a binary image, but not able to realize it. The algorithm is working fine on CPP but I am new to HLS so still struggling to implement it in HLS. – A.k. Jan 25 '18 at 10:11
You should expect HLS code to be different from general purpose C++. It is constrained due to synthesis requirements. I saw you posted [another question](https://stackoverflow.com/questions/48375912/how-to-find-connected-object-in-a-binary-image-in-vivado-hls) about connected components so perhaps you should discuss that specific problem there. – haggai_e Jan 25 '18 at 10:18

score 0 · Accepted Answer · answered Mar 08 '18 at 04:42

I found the answer using Sobel code (XAP1167)

void created_window(MY_IMAGE& src, MY_IMAGE& dst, int rows, int cols)
{
  MY_BUFFER buff_A;
  MY_WINDOW WINDOW_3x3;

  for(int row = 0; row < rows+1; row++){
    for(int col = 0; col < cols+1; col++){
#pragma HLS loop_flatten off
#pragma HLS dependence variable=&buff_A false
#pragma HLS PIPELINE II = 1

      // Temp values are used to reduce the number of memory reads
      unsigned char temp;
      MY_PIXEL tempx;

      //Line Buffer fill
      if(col < cols){
          buff_A.shift_down(col);
          temp = buff_A.getval(0,col);
      }

      //There is an offset to accommodate the active pixel region
      //There are only MAX_WIDTH and MAX_HEIGHT valid pixels in the image
      if(col < cols && row < rows){
          MY_PIXEL new_pix;
          src >> new_pix;
          tempx = new_pix;
          buff_A.insert_bottom(tempx.val[0],col);
      }

      //Shift the processing window to make room for the new column
      WINDOW_3x3.shift_right();

      //The processing window only needs to store luminance values
      //rgb2y function computes the luminance from the color pixel
      if(col < cols){
          WINDOW_3x3.insert(buff_A.getval(2,col),2,0);
          WINDOW_3x3.insert(temp,1,0);
          WINDOW_3x3.insert(tempx.val[0],0,0);
      }
      MY_PIXEL conn_obj;


      //The operator only works on the inner part of the image
      //This design assumes there are no connected objects on the boundary of the image


          conn_obj = find_conn(&WINDOW_3x3);


      //The output image is offset from the input to account for the line buffer
      if(row > 0 && col > 0) {
          dst << conn_obj;
      }
    }
  }
}






void create_window(AXI_STREAM& video_in, AXI_STREAM& video_out, int rows, int cols)
{
    //Create AXI streaming interfaces for the core
#pragma HLS INTERFACE axis port=video_in bundle=INPUT_STREAM
#pragma HLS INTERFACE axis port=video_out bundle=OUTPUT_STREAM

#pragma HLS INTERFACE s_axilite port=rows bundle=CONTROL_BUS offset=0x14
#pragma HLS INTERFACE s_axilite port=cols bundle=CONTROL_BUS offset=0x1C
#pragma HLS INTERFACE s_axilite port=return bundle=CONTROL_BUS

#pragma HLS INTERFACE ap_stable port=rows
#pragma HLS INTERFACE ap_stable port=cols


    MY_IMAGE img_0(rows, cols);
    MY_IMAGE img_1(rows, cols);

#pragma HLS dataflow
    hls::AXIvideo2Mat(video_in, img_0);
    created_window(img_0, img_1, rows, cols);
    hls::Mat2AXIvideo(img_0, video_out);
}

Does Mat in hls:Mat really represents a matrix?

2 Answers2