1

I've created a basic tinker canva where the user is able to paint using a brush. I store all the x and y coordinates of the drawn lines in an array and then I resize them to 28x28 pixel (the size of dataset's images); somehow can I convert that array to a numpy array like the MNIST dataset (a tensorflow dataset) ?

This is an example of that dataset (every number represents the color on the rgb scale):

[[  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0  84 185 159 151  60  36   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0 222 254 254 254 254 241 198 198 198 198 198 198 198 198 170  52   0   0   0   0   0   0]
 [  0   0   0   0   0   0  67 114  72 114 163 227 254 225 254 254 254 250 229 254 254 140   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0  17  66  14  67  67  67  59  21 236 254 106   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  83 253 209  18   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  22 233 255  83   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 129 254 238  44   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  59 249 254  62   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 133 254 187   5   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   9 205 248  58   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 126 254 182   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0  75 251 240  57   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0  19 221 254 166   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   3 203 254 219  35   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0  38 254 254  77   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0  31 224 254 115   1   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0 133 254 254  52   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0  61 242 254 254  52   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0 121 254 254 219  40   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0 121 254 207  18   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]]

EDIT: I've found out that there's an cv2 for converting an image to that specific array that tensorflow requires, img_to_array(image) . So now I have to know how can "take a picture" of the drawn line by the user and then convert them to an array.

Basically the intent of the program is to use a classification neural network made with tensorflow to recognize the user's drawn digits.

SOLUTION:

1) save the image:

from win32 import win32gui
HWND = self.canvas.winfo_id()
rect = win32gui.GetWindowRect(HWND) 
img = ImageGrab.grab(rect)
img = img.resize((28, 28), PIL.Image.ANTIALIAS)

2) convert the image:

img = np.invert(img.convert('L')).ravel()
img = np.split(img, 28)
img = np.array(img)
print("Image:", img, img.shape)
img = img / 255
img = np.array(img)

3) predict the result:

prediction = self.model.predict(img.reshape(1,28,28))
Giacomo A.
  • 175
  • 1
  • 12

2 Answers2

0

If your array is yourarray, and if I understood your question, does not
arr = np.array(yourarray) suffice?

kklocker
  • 136
  • 1
  • 10
  • The problem is that I've saved the x and y coord of the drawn lines (ex. [[1,2] [2,3]]), the mnist dataset instead is based on the color not on the position of the lines (the example is that in the question). – Giacomo A. Feb 06 '20 at 20:47
  • Why do you want to use 3 channels (RGB) for a dataset like MNIST? Why not just have greyscale and a single channel? This way you can save data space from (28 x 28 x 3) down to (28 x 28 x 1) – Arun Feb 06 '20 at 20:58
0

This is how you can read and convert an image to numpy array using PIL (Python Imaging Library or Pillow)-

from PIL import Image
import numpy as np
import sys

try:
    # Absolute path to image-
    # 'Image.open()' method reads the image file. Pillow can read
    # over 30 different file formats
    pic = Image.open("/aboslute_path/pic.jpeg")

except IOError:
    print("\nCannot open/load file.\n")
    sys.exit(1)

# Pillow allows us to get some basic information about the image-
print("\nImage basic information:")
print("Format = {0}, Size = {1}, Mode = {2}".format(
    pic.format, pic.size, pic.mode))


# Convert PIL image to numpy array-
pic_np = np.array(pic.getdata()).reshape(pic.size[0], pic.size[1], 3)

Here, 3 signifies RGB or 3 channels. If your image is a single channel, then replace the 3 with 1.

Arun
  • 2,222
  • 7
  • 43
  • 78