0

I'm building a small image annotation app, but encountered some issues. In short, this app allows the user to free-hand-draw aliased pixel mask on an image.

For this, I create a

- QGraphicsView 
|-- QGraphicsScene # for drawing the overlaid mask
|-- QGraphicsPixmapItem  # for the underlying image

And I detect mousePressEvent, mouseMoveEvent, and mouseReleaseEvent to obtain the QPoints by event.scenePos() under the mouse and append them in a list. With this list, The mask is drew by doing:

for pts in This_list:
    painter.setPen(pen)
    painter.setBrush(Qt.green)
    painter.drawPoint(pts)

However, the Qt won't register every QPoint the mouse swept, if the mouse move faster, less points will be appended.

How can I get every single pixel that the mouse sweep over?

Edit 1: something like this enter image description here

Solution 1: here is a solution, inspired by this post:

  1. we need two QImage, first for the underlying image to annotate, second a transparent canvas where we free-hand draw
  2. the main idea of this solution is to retrieve the pixels that has the same QColor of our QPen
  3. use this solution to scan pixels of the QImage then using a pointer to the bytes data.

A short code is the following:

from PySide6.QtGui import *
from PySide6.QtCore import *
from PySide6.QtWidgets import *
import sys
import numpy as np


# <... other code here... MainWindow, mousePressEvent>
    def mouseReleaseEvent(self, event):
        if (event.button() == Qt.LeftButton) & self.drawing:
            # <... other code/algorithm here>

            # get all pixels under the drawn mask/path
            color_arr = self.getPixelColor(self.mask)
            idx = self.getColorCoordinate(color_arr, QColor(255, 0, 0, 30))

    # here is how to retrieve the QColor array of the self.mask
    def getPixelColor(self, qimage:QImage, readOnly=False):
        '''https://stackoverflow.com/questions/11360009/how-can-access-to-pixel-data-with-pyqt-qimage-scanline, this give a qcolor value array of type: [(255, 0, 0, 30), (255, 255, 255,0)...] 3+alpha color channel'''
        ptr = qimage.bits()

        if readOnly:
            ## get a read-only buffer to access the data
            buf = memoryview(ptr)

            ## view the data as a read-only numpy array
            arr = np.frombuffer(buf, dtype=np.ubyte).reshape(qimage.height(), qimage.width(), 4)
        else:
            ## view the data as a writable numpy array
            arr = np.asarray(ptr).reshape(qimage.height(), qimage.width(), 4)  # replace 4 by 3 if you have a such as FORMAT_RGB32 3 channel QImage
        return arr

    # simple way to find all the indexed of pixel that has QColor expected
    def getColorCoordinate(self, arr: np.array, qcolor: QColor):
        return np.where((arr[:, :, 0] == qcolor.toTuple()[0]) &
                        (arr[:, :, 1] == qcolor.toTuple()[1]) &
                        (arr[:, :, 2] == qcolor.toTuple()[2])  # Note: one can only search for the target label color by ignoring the transparency
                        # (arr[:, :, 3] == qcolor.toTuple()[3])  # Note: uncomment this line to consider the transparency
                        )

enter image description here

Zézouille
  • 503
  • 6
  • 21
  • please provide a [mre] – eyllanesc Oct 07 '21 at 18:49
  • 1
    Mouse events are not continuous, so you will always get points that are far away if the mouse is moved even a bit fast. This is not Qt, it's common for *every* OS. You need to clarify if you want to draw straight lines or try to approximate curves based on the path of those points, as there's a lot of difference in the implementation. – musicamante Oct 07 '21 at 19:07
  • @musicamante thanks, definitely not the straight lines, something like what I added in the picture. There is no such thing built-in in Qt? – Zézouille Oct 07 '21 at 19:23
  • @eyllanesc let me reduce my code – Zézouille Oct 07 '21 at 19:23
  • @Zézouille no, there's not, and I'd say *luckily*. There are a *lot* of ways for which discrete points can be "connected", and it's impossible to provide a predefined behavior that is valid for any situations, and that's valid especially for curved lines. That said, from your image it's still not clear what you want to do: is that image the result of the drawing? are those squared segments pixels? What was the mouse motion that caused it? – musicamante Oct 07 '21 at 22:16
  • @musicamante the image is the result of drawing. The behavior should be like the app 'paint' of Windows, when the mouse left-button is held, draw a continuous line, where the mouse sweeps. To note that, after the free-hand drawing, I would like to have all the coordinates under this mask – Zézouille Oct 08 '21 at 09:42
  • @Zézouille please answer all questions, as it's not clear if the image is maximised or you're using very large "points", and possibly provide an image with the desired result for the same behavior you're getting right now as a comparison. – musicamante Oct 08 '21 at 09:47
  • @musicamante, yes the image is zoomed. I'd like to **draw pixel accurately**. The image is a random 'dog.jpg' image – Zézouille Oct 08 '21 at 09:56
  • @Zézouille again, try to be more clear, what should "draw pixel accurately" mean? As already said, there are ***a lot*** of ways for which discrete points can be connected, we cannot cover them all. As said, a comparison with a possible expected result would really help. Please consider that even in the (not any more) *basic* Paint, drawing tools are actually complex and have different behaviors. – musicamante Oct 08 '21 at 10:06
  • @musicamante well, it is an annotation app, the mask I draw must be pixelized as the same resolution as the loaded image. So if I understand well your suggestion, I need to compute myself, how two points from `mouseEvent` are connected? Most importantly, I need to retrieve the pixel coordinates after the drawing. Do you have a suggestion for this? The path that I draw will be of complexe shape (freehand one). – Zézouille Oct 08 '21 at 14:59
  • here they used to select pixel by pixel with press event aftr moving mouse: https://stackoverflow.com/questions/3504522/pyqt-get-pixel-position-and-value-when-mouse-click-on-the-image . Guess it could work with super magnified images but it would by hand picking each single pixel , not surre it is what you were asking for ? – pippo1980 Oct 08 '21 at 17:18
  • 2
    https://www.pythonguis.com/tutorials/bitmap-graphics/ it says: the issue here is that when you move the mouse around quickly it jumps between locations on the screen, rather than moving smoothly from one place to next. The mouseMoveEventis fired for each location the mouse is in, but that's not enough to draw a continuous line, unless you move very slowly. The solution to this is to draw lines instead of points. On each event we simply draw a line from where we were (previous e.x() and e.y()) to where we are now (current e.x() and e.y()). We can do this by tracking last_x and last_y ourselves. – pippo1980 Oct 08 '21 at 17:23
  • Hi @pippo1980, thank you very much. Your second comment does help. However, it solves the half of my first question. I would like to get all the `pixel.pos()` between two points and the more tricky point will be the second question: if I thicken my line. – Zézouille Oct 08 '21 at 20:58
  • @Zézouille If you have another question then create another post, do not edit the current question. Please read [ask] – eyllanesc Oct 08 '21 at 20:59
  • @Zézouille, you state : the main idea of this solution is to retrieve the pixels that has the same QColor of our QPen, do you have a simple way to check wich is the first color that is not in our image to be able to set QPen to that color ??? – pippo1980 Oct 10 '21 at 15:37
  • @pippo1980 it is a tricker question. No, I don't need to because I have 2 QGraphicPixmapItem s, one over the other, and I draw on the upper QGPixItem'. So whatever QColor or overlaying drawing does not affect the underlying dog image's color. So I don't need to worry about the **color conflicts**. But to answer your origine question, sorry I don't have in mind an efficient way to do so. Maybe with a `np.unique()` function? – Zézouille Oct 10 '21 at 21:57

0 Answers0