python opencv TypeError: Layout of the output array incompatible with cv::Mat

Question

I'm using the selective search here: http://koen.me/research/selectivesearch/ This gives possible regions of interest where an object might be. I want to do some processing and retain only some of the regions, and then remove duplicate bounding boxes to have a final neat collection of bounding boxes. To discard unwanted/duplicated bounding boxes regions, I'm using the grouprectangles function of opencv for pruning.

Once I get the interesting regions from Matlab from the "selective search algorithm" in the link above, I save the results in a .mat file and then retrieve them in a python program, like this:

 import scipy.io as sio
 inboxes = sio.loadmat('C:\\PATH_TO_MATFILE.mat')
 candidates = np.array(inboxes['boxes'])
 # candidates is 4 x N array with each row describing a bounding box like this: 
 # [rowBegin colBegin rowEnd colEnd]
 # Now I will process the candidates and retain only those regions that are interesting
 found = [] # This is the list in which I will retain what's interesting
 for win in candidates: 
     # doing some processing here, and if some condition is met, then retain it:
     found.append(win)

# Now I want to store only the interesting regions, stored in 'found', 
# and prune unnecessary bounding boxes

boxes = cv2.groupRectangles(found, 1, 2) # But I get an error here

The error is:

    boxes = cv2.groupRectangles(found, 1, 2)
TypeError: Layout of the output array rectList is incompatible with cv::Mat (step[ndims-1] != elemsize or step[1] != elemsize*nchannels)

What's wrong? I did something very similar in another piece of code which gave no errors. This was the error-free code:

inboxes = sio.loadmat('C:\\PATH_TO_MY_FILE\\boxes.mat')
boxes = np.array(inboxes['boxes'])
pruned_boxes = cv2.groupRectangles(boxes.tolist(), 100, 300)

The only difference I can see is that boxes was a numpy array which I then converted to a list. But in my problematic code, found is already a list.

Answer 1

My own solution was simply to ask a copy of original array...(god & gary bradski knows why...)

im = dbimg[i]
bb = boxes[i]  
m = im.transpose((1, 2, 0)).astype(np.uint8).copy() 
pt1 = (bb[0],bb[1])
pt2 = (bb[0]+bb[2],bb[1]+bb[3])  
cv2.rectangle(m,pt1,pt2,(0,255,0),2)  

Answer 2

Another reason may be that the array is not contiguous. Making it contiguous would also solve the issue

image = np.ascontiguousarray(image, dtype=np.uint8)

Answer 3

The solution was to convert found first to a numpy array, and then to recovert it into a list:

found = np.array(found)
boxes = cv2.groupRectangles(found.tolist(), 1, 2)

Answer 4

There are a lot of proposed solutions here, but the root cause is the memory layout of the array. For some reason (edit: see comment below), OpenCV requires its input to be in C order (row-major) and not F order (column-major), see here for details.

All of the proposed solutions here implicitly change the array to C order:

• using array.copy() does it because there is a default parameter of order='C'
• converting to a list and back to NumPy, since C order is the default
• array.astype() might do this (seems to depend on the original layout and dtype)
• np.ascontiguousarray() converts to C order.

Here's a little example that reproduces the issue, with cv2.line in this case.

import cv2
import numpy as np

img = np.zeros((200, 200, 3), dtype=np.uint8)

# Breaks
img = np.require(img, requirements=["F_CONTIGUOUS"])

# img = np.require(img, requirements=["C_CONTIGUOUS"])  # Fixes it
# img = img.copy()  # Also fixes it

cv2.line(img, (10, 10), (100, 100), (255,0,0), 5)

Answer 5

Opencv appears to have issues drawing to numpy arrays that have the data type np.int64, which is the default data type returned by methods such as np.array and np.full:

>>> canvas = np.full((256, 256, 3), 255)
>>> canvas
array([[255, 255, 255],
       [255, 255, 255],
       [255, 255, 255]])
>>> canvas.dtype
dtype('int64')
>>> cv2.rectangle(canvas, (0, 0), (2, 2), (0, 0, 0))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Layout of the output array img is incompatible with cv::Mat (step[ndims-1] != elemsize or step[1] != elemsize*nchannels)

The solution is to convert the array to np.int32 first:

>>> cv2.rectangle(canvas.astype(np.int32), (0, 0), (2, 2), (0, 0, 0))
array([[  0,   0,   0],
       [  0, 255,   0],
       [  0,   0,   0]], dtype=int32)

Answer 6

Had the same problem when running

image = cv2.putText(image, 'text', org, font, fontScale, color, thickness, cv2.LINE_AA)

This worked for me

image = cv2.putText(image.astype(np.uint8).copy(), 'text', org, font, fontScale, color, thickness, cv2.LINE_AA)

Answer 7

Just for the sake of completeness, it seems that many of us have used the solution of Etienne Perot above, minus .copy(). Converting the array type to int is enough. For example, when using the Hough transform:

    # Define the Hough transform parameters
    rho,theta,threshold,min,max = 1, np.pi/180, 30, 40, 60  

    image = ima.astype(np.uint8) # assuming that ima is an image.

    # Run Hough on edge detected image
    lines = cv2.HoughLinesP(sob, rho, theta, threshold, np.array([]), min, max)

    # Iterate over the output "lines" and draw lines on the blank 
    line_image = np.array([[0 for col in range(x)] for row in range(y)]).astype(np.uint8)

    for line in lines: # lines are series of (x,y) coordinates
        for x1,y1,x2,y2 in line:
            cv2.line(line_image, (x1,y1), (x2,y2), (255,0,0), 10)

Only then could the data be plotted out using plt.imshow()