0

I'm trying to use cv2.imshow to display a specific section of my screen.

However, while running, I noticed the displayed window is smaller than expected. I've stepped through the code with the debugger and noticed that when cv2.waitKey(1) triggers the cv2.imshow window resizes itself and I'm not sure why.

If I comment out cv2.waitKey(1) the display window remains the expected size, the documentation that would suggest cv2.waitKey does anything other than wait for a specified amount of time 0 = infinite, x = time in milliseconds.

Why is waitKey resizing cv2.imshow and how do I fix this?

import cv2 as cv
import numpy as np
from mss import mss

sct = mss()

def screenshot(top, left, width, height):
    monitor = {'top': top, 'left': left, 'width': width, 'height': height}
    haystack = np.array(sct.grab(monitor))

    cv.imshow("Objects", haystack)
    cv.moveWindow("Objects",5000,50)
    cv.setWindowProperty('Objects', cv.WND_PROP_TOPMOST, 1)    
    cv.waitKey(1)
    return haystack
    
top = 350
left = 1000
width = 500
height = 800
    
while True:
    screenshot(top, left, width, height)

With cv.waitKey(1) commented out:

enter image description here

and with cv.waitKey(1) not commented out:

enter image description here

Contents of haystack saved to file using cv2.imwrite:

enter image description here

Dan Mašek
  • 17,852
  • 6
  • 57
  • 85
mak47
  • 303
  • 2
  • 14
  • "does anything other than wait for a specified amount" -- `waitKey` runs the event loop, which is what makes the GUI actually work. That's why, when you comment it out, you get a grey window -- no painting is being done. | By `im.show` you mean `cv2.imshow`? – Dan Mašek Dec 27 '22 at 14:49
  • 1
    Sorry yes, cv2.imshow or in the above example cv.imshow I realise why it's showing a grey window, but why is it showing a difference size of window? – mak47 Dec 27 '22 at 17:34
  • 1
    What is the actual size of `haystack`, prior to passing it to `imshow`? I can't reproduce the problem here, the window is the expected size. What version of OpenCV? I guess this is on Windows? | BTW, PIL seems useless here -- you can directly go to a numpy array: https://python-mss.readthedocs.io/examples.html#opencv-numpy – Dan Mašek Dec 27 '22 at 18:01
  • 1
    `haystack` is (800, 500, 3) although it doesn't matter, whatever I specify with top left width height it's always reset to the bottom image with the brown bars. opencv-python==4.6.0.66 and on windows yes – mak47 Dec 27 '22 at 18:43
  • 1
    When I step through with the debugger, `imshow` brings up a blank window named objects, move window and set property work correctly. The location changes, it moves to the front, still a grey empty window. As soon as step into waitKey the location resets (didn't notice that till now) and the window shrinks in size. When I check the return variable it still shows as shape: (800, 500,3) which is clearly wrong. – mak47 Dec 27 '22 at 18:48
  • 1
    Removed PIL from the larger script as well, no difference. Thank you! – mak47 Dec 27 '22 at 19:11
  • 1
    That's really strange (might as well [edit] your question and update the script to the non-PIL variant). Is the image with brown bars anything close to what you'd expect to be captured? What if you save `haystack` to file (say PNG) using `cv.imwrite`? Can you upload the result of that, please? | Does the same problem occur with a different version(s)/build(s) of OpenCV? – Dan Mašek Dec 27 '22 at 20:02
  • 1
    Updated the OP to the non-PIL variant. cv.imwrite gives the correct 800x500 (uploading to OP now). The colours have changed because I've not inverted them as I had with the PIL variant. So the blue is expected. So it's just `im.show` isn't showing correctly. – mak47 Dec 27 '22 at 21:28
  • OK, this doesn't look like something that should be happening. The suspicions bit is the OpenCV build that you're using. Does this reproduce on your machine with other versions of OpenCV installed using `pip`? What if you get a pre-built version from the official repository of opencv (e.g. https://github.com/opencv/opencv/releases/tag/4.6.0) and install it manually -- does the problem still occur? – Dan Mašek Dec 27 '22 at 21:39
  • I'll try that tomorrow morning and let you know. I've actually had it happen fairly frequently and I've always put it down to my own mistake somewhere. I've only ever installed opencv with pip install opencv-python from https://stackoverflow.com/questions/51853018/how-do-i-install-opencv-using-pip is that outdated/wrong? – mak47 Dec 27 '22 at 22:27
  • All of those commands (like imshow and movewindow) do not take action immediately. They all send window messages. Nothing actually happens until the UI framework gets back to its main loop, where messages are unqueued and dispatched. In your case, that happens in the `waitKey` call. – Tim Roberts Dec 27 '22 at 23:42
  • When I step through imshow immediately displays a window and movewindow moves the window, the window isn't populated until waitKey is called. – mak47 Dec 28 '22 at 09:55
  • @DanMašek opencv-python 4.6.0.66 is the most recent official version. https://pypi.org/project/opencv-python/ was officially adopted in 2021, and I'm not sure how to install it manually via the github link, I've downloaded the vc14_vc15 but there isn't an obvious windows setup.py – mak47 Dec 28 '22 at 11:07
  • it's all wrong. stop doing it the hard way. `pip install opencv-python` and that's it. and NEVER EVER omit the waitKey/pollKey. without that, the GUI simply DOES NOT WORK. -- the funny colors happen because the screenshot is RGB but OpenCV interprets it as BGR, so red and blue are swapped. the window size DOES NOT affect the data you passed to imshow. – Christoph Rackwitz Dec 28 '22 at 22:34
  • Yes, I got it working without fixing the image waitKey is displaying. It would still be nice to know why waitKey is changes the size of the imshow window. edit, I never did figure out how to manually install it, so I'm still using `pip install opencv-python` – mak47 Dec 30 '22 at 09:47

0 Answers0