1

I have the following situation, my in-memory image and what gets stored to disk and then read back do not equal, and I'd like to understand why and also how to 'fix' the difference.

If anyone wants to know why I have both (in-memory and local disk) its because I was using the stored images to fine tune my image matching/recognition using OpenCV's sliding window without having to constantly wait on a video stream to show what I was looking for.

The BGR frame comes from a video stream created through ffmpeg in bgr24 pixel format that then gets processed like this:

self.raw_image = self.pipe.stdout.read(self.byte_length * self.byte_width * self.byte_offset)
bgr_frame = np.frombuffer(self.raw_image, dtype=np.uint8).reshape((self.byte_width, self.byte_length, self.byte_offset))
cv2.imwrite("capture/rgb/Frame_%d.png", self.frame_num), bgr_frame, [cv2.IMWRITE_PNG_COMPRESSION, 0])

And now things get interesting, the bgr_frame is sent to another function that cuts out regions of interests and then tries to match each region against a lib_image from a library of images, like this

res = cv2.matchTemplate(region, lib_image, 'cv2.TM_CCOEFF_NORMED')

the value of res returned for the bgr_frame hovers around the .86 region which is .1 lower than if the same image is read back from disk like this:

image = cv2.imread("%s/%s" % (input_directory, file))
image = image[:,:,::-1] # Flip BGR to RGB

Pushing the image read from disk through the same function to match the region against the library results in matches that are in .96 range!

What am I missing? The OpenCV documentation doesn't mention doing anything to the image beyond encoding it in a specific format in my case PNG with 0 compression, and yet clearly what is in memory and what gets written/read to/from disk differs.

eyllanesc
  • 235,170
  • 19
  • 170
  • 241
Sauraus
  • 95
  • 13
  • Have you tried using `imshow` (either in `cv2` or in `matplotlib.pyplot`) to compare the two images visually? It would also be useful to have some example data to run through this. – ajrwhite Dec 14 '18 at 01:06
  • Just incase anyone wants to know how the frames are captured by `ffmpeg` and forwarded here is the code: `self.pipe = sp.Popen(['/usr/local/bin/ffmpeg', "-i", self.stream.url, "-r", "3", "-loglevel", "quiet", "-an", "-f", "image2pipe", "-pix_fmt", "bgr24", "-vcodec", "rawvideo", "-"], stdin=sp.PIPE, stdout=sp.PIPE)` The `-pix_fmt` is there because CV2 expects BGR and not RGB. – Sauraus Dec 14 '18 at 03:39
  • 1
    try this without any image manipulation after capturing: imwrite(image, "tmp.png") tmpImg = imread("tmp.png") diffimage = cv2.absdiff(image, tmpImage) diffimage should be 0,0,0 everywhere if the imwrite and imread do not manipulate the image. Fix the python syntax pleass, I don't know it well. – Micka Dec 15 '18 at 00:03

1 Answers1

1

CV2 imwrite does a conversion of the output from BGR2RGB when called like this:

cv2.imwrite("capture/rgb/Frame_%d.png", self.frame_num), bgr_frame, [cv2.IMWRITE_PNG_COMPRESSION, 0])

Which is why adding cv2.cvtColor(bgr_frame, cv2.COLOR_BGR2RGB) before passing the image on for analysis at the very least fixed the color issues I was seeing, this also explains why on the CV2 imread I had to flip the array to get the image gain back to RGB; there is probably a CV2.xxxxx parameter that I can pass to imread that will read the image as a RGB immediately with no need for any array manipulations.

image = cv2.imread("%s/%s" % (input_directory, file))
image = image[:,:,::-1] # Flip BGR to RGB

However this still hasn't fixed my poor matching from a live stream frame .86 vs .96 when saving and loading that very same frame as PNG.

Sauraus
  • 95
  • 13