Python OpenCV imshow Expected mat Argument

Question

I'm trying to write a program that shows a specific position on a given window. When I run the code OpenCV gives me an error message telling me unexpected type for argument mat. I have no idea what's wrong because when I use the position value in the following code it works fine. Please be nice as I'm fairly new to OpenCV.

Code:

#Imports
import numpy as np
import cv2
import pytesseract
from PIL import ImageGrab
import win32gui


#Assign Tesseract and other Variables
winList = []
toplist = []
pytesseract.pytesseract.tesseract_cmd = r'C:\\Program Files\\Tesseract-OCR\\tesseract.exe'
lH = 33
lW = 20

#Window Handle Object
def enum_win(hwnd, result):

    winText = win32gui.GetWindowText(hwnd)

    winList.append((hwnd, winText))

win32gui.EnumWindows(enum_win, toplist)


typeHwnd = 0

#Find Window Handle
for (hwnd, winText) in winList:

    if winText == "Program WinText Name Here":
        typeHwnd = hwnd
if typeHwnd == 0:
    print("Oops no Window Found")


while True:
    position = win32gui.GetWindowRect(typeHwnd)

    mid = (position[0] + (position[0] + position[2]))/2

    sPos = [int(mid)-267, position[1] + 295, lW, lH]

    screenshot = ImageGrab.grab(bbox=sPos)
    screenshot = np.array(screenshot)

    cv2.imshow("Screen", screenshot)
    key = cv2.waitKey(25)

print(position)
print(mid)

Error Message:

Traceback (most recent call last):
  File "C:/Users/MyName/PycharmProjects/Typing Bot/main.py", line 47, in <module>
    cv2.imshow("Screen", screenshot)
TypeError: Expected Ptr<cv::UMat> for argument 'mat'

System Specifications:

I am running Windows 10 64 bit and Python 3 with the most recent version of all the imported modules installed.

first check what you have in variable - `print(screenshot)`, `print(type(screenshot))` — furas, Feb 09 '21 at 00:46
That returns tuple. So instead of an array I need sPos to be a tuple. — Alex Rumer, Feb 09 '21 at 14:41
Even if I make sPos a tuple instead of an array it gives me that same error. — Alex Rumer, Feb 09 '21 at 14:46
I asked of `screenshot`, not `sPos`. `screenshot` in `imshow` has to be `numpy array` - and this is in your error message. And it doesn't mater if your `sPos` is tuple or list. — furas, Feb 09 '21 at 15:06
well, you may have problem with `sPos` in `grab` - it has to be `[x1, y1, x2, y2]` but it seems you use `[x, y, width, height]` — furas, Feb 09 '21 at 15:36

furas · Accepted Answer · 2021-02-09T17:17:06.817

Error shows problem with array in imshow but your problem starts with sPos in grab.

grab needs bbox which means [x1, y1, x2, y2] but you use [x, y, width, height] and this starts problem because grab returns image with size (0,0) and np.array() can't convert it so it returns pillow image instead of numpy array and then imshow gets pillow image instead of numpy array and it can't display it.

You have to use [x, y, x+width, y+height]

Minimal working example - with wrong and correct pos

import cv2
import numpy as np
import PIL.ImageGrab

x = 100
y = 100
w = 50 
h = 50 

pos = [x, y, w, h]      # wrong 
#pos = [x, y, x+w, y+h]  # correct 

print('bbox:', pos)
print('---')

img = PIL.ImageGrab.grab(pos)
print('img:', img)
print('type(img):', type(img))
print('img.size:', img.size)
print('---')

arr = np.array(img)
print('arr:', arr)
print('type(arr):', type(arr))
print('arr.size:', arr.size)
print('arr.shape:', arr.shape)
print('---')

cv2.imshow('test', arr)

print('Press ESC to exit')
while cv2.waitKey(100) != 27:
    pass

Python OpenCV imshow Expected mat Argument

1 Answers1