Brightness/Histogram normalization for time lapse images

Question

I'm working in a laboratory and we often make time lapse series (image every hour) of stem cells. The current idea is to put all frames together and make a video showing this growing cells (similar to this youtube video). Which could done simple and cool using OpenCV + Python.

import numpy as np
import os
import cv2

fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('output.avi',fourcc, 20.0, (640,480))

timelapse_folder = '../myTimeLapse/'

for file in os.listdir(timelapse_folder):
    frame = cv2.imread(timelapse_folder+file, 0)
    out.write(frame)

out.release()

But we have the problem, that all images vary a little bit in brightness, so we get some flickering in our output video.

I'm not allowed to upload the videos but here are some simple examples generated with gimp to visualize the problem:

That's the video I get from the frames

and that's my desired video (it would be also great to minimize the flickering instead of removing it completely)

Is there a way to adjust the histogram or brightness over all images (or maybe between 2 images) to remove those flickering using OpenCV?

Thanks for every idea or hint!

Edit: The gif sequence produced by Andrew's idea (Answer below)

Presuming that all the images are 2D arrays of floats/ints, Then you could construct a 3D array of all the images, and then normalise against the brightest `(max(np.average(arr, axis=2)))`. Then you step through and write to the video? — Andrew, Sep 07 '16 at 07:08
Hey I tried `max_avg = np.max(np.average(images, axis=2))` and then in a for loop over each frame `frame = (frame/max_avg)*255` but it does not remove the flickering. Or do you mean another type of normalization? — Fabian, Sep 07 '16 at 11:16

score 1 · Accepted Answer · edited May 23 '17 at 12:08

1

If your data is in a 3D array, you shouldn't need to loop over it to do this. With 5 images of, say 256 x 256, you should be able to construct an array that is arr.shape == (256, 256, 5). My initial comment was a little off I think, but the below example should do it.

target_array = []

for file in os.listdir(timelapse_folder):
    frame = cv2.imread(timelapse_folder+file, 0)
    if target_array:#Not entirely happy with this, but it should work
        target_array = np.dstack((target_array, frame))
    elif not target_array:
        target_array = np.asarray(frame)
target_array = target_array / np.max(target_array)
#target_array *= 255 #If you want an intensity value with a more common range here  
for idx in xrange(target_array.shape[2]):
    out.write(target_array[:, :, idx])

EDIT: I used this page to iron out some kinks with addressing the 3D array

edited May 23 '17 at 12:08

Community

1
1

answered Sep 07 '16 at 13:05

Andrew

1,072
1
7
15

If you have issues with flickering, you could add an interpolated slice between each of your data slices which is mid-way between the two frames, that would reduce the apparent flicker effect. Not sure if a smoothing function would work, depending on how many frames there are. – Andrew Sep 07 '16 at 13:20
Hey I tried your solution doing that kind of normalization between the images and add transitions (interpolated slice) and it looks "Ok" now. The normalization removes the flickering a little bit but I have some problem with the interpolated slice because the cells are sometimes not close enough so that it looks really blurry. I will add a gif produced with the idea of your answer to my question, but I will wait with accept - maybe anyone has another idea. Otherwise I will accept in a couple of days. Thanks a lot. – Fabian Sep 07 '16 at 16:18
If smoothness is the priority, you can always add multiple interpolated slices, which can be done during the dstack, defining your number of interpolated slices and then creating that number of arrays. Then you can choose whether to step linearly or logarithmically (maybe a normal distribution with 11 slices would give optimal smoothness?). Obviously, your real data will be outnumbered by 11 : 1 or thereabouts, but as long as you're candid about it to any scientists who see it, I don't see any issue. – Andrew Sep 07 '16 at 16:36
if you have a white pixel (255) you will essentially be dividing and then multiplying by 255 - changing nothing at all... – Frank Musterman Feb 25 '20 at 14:23

score 0 · Answer 2 · answered Sep 07 '16 at 07:37

0

Are those images RGB or gray values? I would just perform a normalization in your loop after reading each frame:

frame = frame/np.max(frame)

In case of gray values each image should then have values between 0 and 1 but depending on how your images look like you could also try other normalizations, e.g. using the np.median or np.mean instead of np.max.

answered Sep 07 '16 at 07:37

a.smiet

1,727
3
21
36

I tried this but the brightness difference _between_ all images persists. I think I need to respect the average of all images like Andrew wrote in his comment, but I don't know how I can move the average for each frame to the average of all frames. Do you know what I mean? – Fabian Sep 07 '16 at 11:39
I think I get your problem. Do you have some kind of objects that you can separate from the background in your frames (e.g. using a simple threshold)? That way you could try to adjust only the background of all frames. For example: Take the mean of the background of an arbitrary frame as reference and shift all other frames such that the averages of their backgrounds match the reference. – a.smiet Sep 07 '16 at 12:39
Is there a simple numpy or cv2 method to shift those frame values? The cells are not completely white, so I think if I only adjust the background I will get flickering cells (foreground). In later images with higher confluency (many cells) I can not separate the background, because there are only flickering cells really close together. – Fabian Sep 07 '16 at 12:48
Hmm... If a simple normalization does not work it seems that you have "non-constant" differences between cells and background in each frame, i.e. you will always have a flickering, either in the foreground or background... Maybe you could play around with histogram equalization and find out whether this will help in some way? See http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_histograms/py_histogram_equalization/py_histogram_equalization.html – a.smiet Sep 07 '16 at 13:08

Brightness/Histogram normalization for time lapse images

2 Answers2