8

I know this thread about converting black color to white and white to black simultaneously. I would like to convert only black to white. I know this thread about doing this what I am asking but I do not understand what goes wrong.

Picture

enter image description here

Code

rgbImage = imread('ecg.png');
grayImage = rgb2gray(rgbImage); % for non-indexed images
level = graythresh(grayImage); % threshold for converting image to binary, 
binaryImage = im2bw(grayImage, level); 
% Extract the individual red, green, and blue color channels.
redChannel = rgbImage(:, :, 1);
greenChannel = rgbImage(:, :, 2);
blueChannel = rgbImage(:, :, 3);
% Make the black parts pure red.
redChannel(~binaryImage) = 255;
greenChannel(~binaryImage) = 0;
blueChannel(~binaryImage) = 0;
% Now recombine to form the output image.
rgbImageOut = cat(3, redChannel, greenChannel, blueChannel);
imshow(rgbImageOut);

Which gives

enter image description here

Where seems to be something wrong in red color channel. The Black color is just (0,0,0) in RGB so its removal should mean to turn every (0,0,0) pixel to white (255,255,255). Doing this idea with

redChannel(~binaryImage) = 255;
greenChannel(~binaryImage) = 255;
blueChannel(~binaryImage) = 255;

Gives

enter image description here

So I must have misunderstood something in Matlab. The blue color should not have any black. So this last image is strange.

How can you turn only black color to white? I want to keep the blue color of the ECG.

Community
  • 1
  • 1
Léo Léopold Hertz 준영
  • 134,464
  • 179
  • 445
  • 697

4 Answers4

7

If I understand you properly, you want to extract out the blue ECG plot while removing the text and axes. The best way to do that would be to examine the HSV colour space of the image. The HSV colour space is great for discerning colours just like the way humans do. We can clearly see that there are two distinct colours in the image.

We can convert the image to HSV using rgb2hsv and we can examine the components separately. The hue component represents the dominant colour of the pixel, the saturation denotes the purity or how much white light there is in the pixel and the value represents the intensity or strength of the pixel.

Try visualizing each channel doing:

im = imread('https://i.stack.imgur.com/cFOSp.png'); %// Read in your image
hsv = rgb2hsv(im);
figure;
subplot(1,3,1); imshow(hsv(:,:,1)); title('Hue');
subplot(1,3,2); imshow(hsv(:,:,2)); title('Saturation');
subplot(1,3,3); imshow(hsv(:,:,3)); title('Value');

enter image description here

Hmm... well the hue and saturation don't help us at all. It's telling us the dominant colour and saturation are the same... but what sets them apart is the value. If you take a look at the image on the right, we can tell them apart by the strength of the colour itself. So what it's telling us is that the "black" pixels are actually blue but with almost no strength associated to it.

We can actually use this to our advantage. Any pixels whose values are above a certain value are the values we want to keep.

Try setting a threshold... something like 0.75. MATLAB's dynamic range of the HSV values are from [0-1], so:

mask = hsv(:,:,3) > 0.75;

When we threshold the value component, this is what we get:

enter image description here

There's obviously a bit of quantization noise... especially around the axes and font. What I'm going to do next is perform a morphological erosion so that I can eliminate the quantization noise that's around each of the numbers and the axes. I'm going to make it the mask a bit large to ensure that I remove this noise. Using the image processing toolbox:

se = strel('square', 5);
mask_erode = imerode(mask, se);

We get this:

enter image description here

Great, so what I'm going to do now is make a copy of your original image, then set any pixel that is black from the mask I derived (above) to white in the final image. All of the other pixels should remain intact. This way, we can remove any text and the axes seen in your image:

im_final = im;
mask_final = repmat(mask_erode, [1 1 3]);
im_final(~mask_final) = 255;

I need to replicate the mask in the third dimension because this is a colour image and I need to set each channel to 255 simultaneously in the same spatial locations.

When I do that, this is what I get:

enter image description here

Now you'll notice that there are gaps in the graph.... which is to be expected due to quantization noise. We can do something further by converting this image to grayscale and thresholding the image, then filling joining the edges together by a morphological dilation. This is safe because we have already eliminated the axies and text. We can then use this as a mask to index into the original image to obtain our final graph.

Something like this:

im2 = rgb2gray(im_final);
thresh = im2 < 200;
se = strel('line', 10, 90);
im_dilate = imdilate(thresh, se);
mask2 = repmat(im_dilate, [1 1 3]);
im_final_final = 255*ones(size(im), class(im));
im_final_final(mask2) = im(mask2);

I threshold the previous image that we got without the text and axes after I convert it to grayscale, and then I perform dilation with a line structuring element that is 90 degrees in order to connect those lines that were originally disconnected. This thresholded image will contain the pixels that we ultimately need to sample from the original image so that we can get the graph data we need.

I then take this mask, replicate it, make a completely white image and then sample from the original image and place the locations we want from the original image in the white image.

This is our final image:

enter image description here

Very nice! I had to do all of that image processing because your image basically has quantization noise to begin with, so it's going to be a bit harder to get the graph entirely. Ander Biguri in his answer explained in more detail about colour quantization noise so certainly check out his post for more details.

However, as a qualitative measure, we can subtract this image from the original image and see what is remaining:

imshow(rgb2gray(abs(double(im) - double(im_final_final))));

We get:

enter image description here

So it looks like the axes and text are removed fine, but there are some traces in the graph that we didn't capture from the original image and that makes sense. It all has to do with the proper thresholds you want to select in order to get the graph data. There are some trouble spots near the beginning of the graph, and that's probably due to the morphological processing that I did. This image you provided is quite tricky with the quantization noise, so it's going to be very difficult to get a perfect result. Also, these thresholds unfortunately are all heuristic, so play around with the thresholds until you get something that agrees with you.

Good luck!

rayryeng
  • 102,964
  • 22
  • 184
  • 193
  • Can you approximate how much information is lost from the signal with any selection of threshold? Th-0.60 and Th-0.75 seems to be about similar. How many points have you lost more from the former than the latter? – Léo Léopold Hertz 준영 Apr 22 '15 at 15:34
  • @Masi - That's a hard question to answer. Your image was already subjected to quantization noise so I can't tell how much of your original signal was there before the noise and how much of the data is present due to quantization noise. If you are trying to use this graph for accuracy, then you probably shouldn't be doing that. – rayryeng Apr 22 '15 at 15:41
  • 1
    @Masi - However, if you want to determine how many pixels are different in the mask between `0.60` and `0.75`, simply threshold both of the images with that amount and store them in separate images... called `I1` and `I2`, then do `abs(nnz(I1) - nnz(I2))` to determine how many pixels are different between both images. From there you can determine how much percentage change was experienced. – rayryeng Apr 22 '15 at 15:44
  • How is variable `im_dilate` defined? – Léo Léopold Hertz 준영 Apr 22 '15 at 15:45
  • 1
    @Masi - oops. sorry hold on – rayryeng Apr 22 '15 at 15:46
  • 1
    @Masi - There we go. Sorry. I forgot to actually perform the dilation on the image. – rayryeng Apr 22 '15 at 15:47
  • Can you estimate how much these crossings of the axes with the graph affect your final result? https://www.dropbox.com/s/9injflk0iti6m3u/ECGexample1.pdf%20copy.png?dl=0 – Léo Léopold Hertz 준영 Apr 22 '15 at 15:54
  • 1
    @Masi - That's the beauty of the HSV colour space. The crossings have a different value components than the graph data itself, so when I thresholded the value plane, I managed to remove those axes. Take a look at my edit where I actually showed the difference between the final image with the original. – rayryeng Apr 22 '15 at 15:57
  • There is quite much lost signal and I want to understand why. The different selection of the threshold did not affect it much, which I do not understand. The original data is got from this PDF file: https://www.dropbox.com/s/dvc2vz8k2r0th6n/ECGexample1.pdf?dl=0 It should be exactly the same image that is in my question. – Léo Léopold Hertz 준영 Apr 22 '15 at 16:04
  • The lost of those peaks (shown white) in the ECG are very critical for the application. Why do they get lost in the processing? Can we change something such that they do not get lost? I think the line selection of 90 degree can be the reason. Can it be set to 89 degree or something else. Peaks must be allowed in the ECG signal. – Léo Léopold Hertz 준영 Apr 22 '15 at 16:06
  • Hmm... wouldn't it be more prudent to just **crop** the image? Could you just specify a region of interest and just remove out everything else but the data itself? – rayryeng Apr 22 '15 at 16:07
  • @Masi - I'll try another structuring element. Give me a moment. – rayryeng Apr 22 '15 at 16:08
  • I started to think a new colormap in making the mask for the quantization noise here http://stackoverflow.com/q/29806687/54964 – Léo Léopold Hertz 준영 Apr 22 '15 at 19:07
  • 2
    I am going to finish editing my question because it looks like (I am super excited) my answer performs a little better than what the amazing rayryeng does (OMGOMGOMG, this is not usual!). Check it out @Masi – Ander Biguri Apr 23 '15 at 08:31
  • 2
    @AnderBiguri - lmao. I welcome to be proven wrong from time to time. Please let me see your results :) – rayryeng Apr 23 '15 at 08:33
  • 2
    @rayryeng sorry took a bit long posting them. There they are! – Ander Biguri Apr 23 '15 at 08:53
  • 1
    @rayryeng: I'm late to this, but in another question from OP, I posted a similar solution with improved results. Basically I apply two morphological closing operations using horizontal and vertical lines as structuring element. Check out the "extract blue ECG signal" section from the second part of [this answer](http://stackoverflow.com/a/29952648/97160). – Amro Apr 29 '15 at 22:44
5

What's the problem?

You want to detect all black parts of the image, but they are not really black

Example:

enter image description here

Your idea (or your code):

You first binarize the image, selecting the pixels that ARE something against the pixels that are not. In short, you do: if pixel>level; pixel is something

Therefore there is a small misconception you have here! when you write

% Make the black parts pure red.

it should read

% Make every pixel that is something (not background) pure red.

Therefore, when you do

redChannel(~binaryImage) = 255;
greenChannel(~binaryImage) = 255;
blueChannel(~binaryImage) = 255;

You are doing

% Make every pixel that is something (not background) white 
% (or what it is the same in this case, delete them).

Therefore what you should get is a completely white image. The image is not completely white because there has been some pixels that were labelled as "not something, part of the background" by the value of level, in case of your image around 0.6.

A solution that one could think of is manually setting the level to 0.05 or similar, so only black pixels will be selected in the gray to binary threholding. But this will not work 100%, as you can see, the numbers have some very "no-black" values.


How would I try to solve the problem:

I would try to find the colour you want, extract just that colour from the image, and then delete outliers.

Extract blue using HSV (I believe I answered you somewhere else how to use HSV).

rgbImage = imread('ecg.png');
hsvImage=rgb2hsv(rgbImage);
I=rgbImage;
R=I(:,:,1);
G=I(:,:,2);
B=I(:,:,3);
th=0.1;
R((hsvImage(:,:,1)>(280/360))|(hsvImage(:,:,1)<(200/360)))=255;
G((hsvImage(:,:,1)>(280/360))|(hsvImage(:,:,1)<(200/360)))=255;
B((hsvImage(:,:,1)>(280/360))|(hsvImage(:,:,1)<(200/360)))=255;
I2= cat(3, R, G, B);

imshow(I2)

enter image description here

Once here we would like to get the biggest blue part, and that would be our signal. Therefore the best approach seems to first binarize the image taking all blue pixels

% Binarize image, getting all the pixels that are "blue"
bw=im2bw(rgb2gray(I2),0.9999);

And then using bwlabel, label all the independent pixel "islands".

% Label each "blob"
lbl=bwlabel(~bw);

The label most repeated will be the signal. So we find it and separate the background from the signal using that label.

% Find the blob with the highes amount of data. That  will be your signal.
r=histc(lbl(:),1:max(lbl(:)));
[~,idxmax]=max(r);
% Profit!
signal=rgbImage;
signal(repmat((lbl~=idxmax),[1 1 3]))=255;
background=rgbImage;
background(repmat((lbl==idxmax),[1 1 3]))=255;

Here there is a plot with the signal, background and difference (using the same equation as @rayryang used)

enter image description here

Ander Biguri
  • 35,140
  • 11
  • 74
  • 120
  • 2
    Very nice. Pretty much the same approach I took almost lol. +1. – rayryeng Apr 22 '15 at 15:08
  • This sounds good *A solution that one could think of is manually setting the level to 0.05 or similar, so only black pixels will be selected in the gray to binary threholding.* You mean with binary-Threshold-0.60 and level 0.05. It would be nice to be able to approximate what is the uncertainty or amount of lost signal with Threshold-0.60 and level 0.05. And compare this to Threshold-0.75 with 0.05 level. I think 0.05 level is a good choice. I think the changeable parameter is Threshold. – Léo Léopold Hertz 준영 Apr 22 '15 at 15:39
  • 1
    @Masi Level==gray2bw treshold in this case. However check my updated answer to see a way of extracting the signal from the plot – Ander Biguri Apr 23 '15 at 08:48
  • The last image is got by `figure; imshow(rgb2gray(abs(double(rgbImage) - double(signal))));` here as described in rayryeng's answer so removing the obtained signal from the original RGB figure. – Léo Léopold Hertz 준영 Apr 23 '15 at 12:31
  • This seems to be a better result than Rayryeng's result by inspection. Now, I am thinking to qualitatively develop some indicator to estimate how good your result is, particularly near the axes. It seems that there is no false positive removals in the center, but may be near the corners. – Léo Léopold Hertz 준영 Apr 23 '15 at 12:39
  • Excellent result! I went through false positive removals. I did not find them. There are 2042 pixel removals `size(find(comp_image(25:446, 41:574) > 0))` totally inside the white box, given 1 black pixel margin around it. There is `2042 (= (422-1)*2 + (533-1)*2 + 5*9*2 + 7*4 + 5*3 + 4*2 + 3*1 - 7 - 1 )` white pixels in axes only. Nothing removed from the actual ECG signal. Can you confirm? – Léo Léopold Hertz 준영 Apr 23 '15 at 13:55
  • @Masi Thanks. I don't really understand what you mean. There are really few false positives (pixels that are selected as signal when they are axes/background). These are the pixels that are missing in the left and right axes around 0-0.1. It looks they are around10 pixels or so, but the only way of counting them is literally counting them! – Ander Biguri Apr 23 '15 at 14:40
  • 1
    @AnderBiguri Yes, I did it. I also did a computation of pixels in the center but none found. I counted all white pixels in the picture without text. – Léo Léopold Hertz 준영 Apr 23 '15 at 15:11
  • Can you, please, explain me why you use this array `[1 1 3]` here `signal(repmat((lbl~=idxmax),[1 1 3]))=255;`? Why 3 lastly? How do read the whole line? You are mapping those blobs white such that they are not visible in the signal. – Léo Léopold Hertz 준영 Apr 28 '15 at 21:06
  • 1
    It is an indexing problem. Whenever a pixel is wanted white you need to set the 3 chanels (RGB) to 255, therefore the 3. What does " How do read the whole line?" means?. I am indeed selecting the blobs that are not the biggest ones and setting their colour to white. @Masi – Ander Biguri Apr 28 '15 at 21:17
  • I extended this challenge to a problem with colors here http://stackoverflow.com/q/29990133/54964 Note also that there are many pikes and horizontal lines in the data which also makes the problem more difficult. Your solution works best from all answers here with this kind of data but not well enough. – Léo Léopold Hertz 준영 May 01 '15 at 16:12
  • @Masi ill have a look. However you shoukd choose an answer as accepted here! – Ander Biguri May 02 '15 at 17:00
3

Here is a variation on @rayryeng's solution to extract the blue signal:

%// retrieve picture
imgRGB = imread('https://i.stack.imgur.com/cFOSp.png');

%// detect axis lines and labels
imgHSV = rgb2hsv(imgRGB);
BW = (imgHSV(:,:,3) < 1);
BW = imclose(imclose(BW, strel('line',40,0)), strel('line',10,90));

%// clear those masked pixels by setting them to background white color
imgRGB2 = imgRGB;
imgRGB2(repmat(BW,[1 1 3])) = 255;

%// show extracted signal
imshow(imgRGB2)

extracted_signal

To get a better view, here is the detected mask overlayed on top of the original image (I'm using imoverlay function from the File Exchange):

figure
imshow(imoverlay(imgRGB, BW, uint8([255,0,0])))

overlayed_mask

Community
  • 1
  • 1
Amro
  • 123,847
  • 25
  • 243
  • 454
  • I went through a set of ECG signals and their Time-Frequency correspondences where also the strength of the signal is visualized by colors. This method breaks in this TF space. However, it seems to work well in uniform ECG signals. I will try to provide later some data about these special cases. I am now testing if this method works in all variations of ECG. It should whenever the colorcoding is uniform. – Léo Léopold Hertz 준영 May 01 '15 at 06:22
1

Here is a code for this:

rgbImage = imread('ecg.png');

redChannel = rgbImage(:, :, 1);
greenChannel = rgbImage(:, :, 2);
blueChannel = rgbImage(:, :, 3);

black = ~redChannel&~greenChannel&~blueChannel;

redChannel(black) = 255;
greenChannel(black) = 255;
blueChannel(black) = 255;

rgbImageOut = cat(3, redChannel, greenChannel, blueChannel);

imshow(rgbImageOut);

black is the area containing the black pixels. These pixels are set to white in each color channel.

In your code you use a threshold and a grayscale image so of course you have much bigger area of pixels that is set to white resp. red color. In this code only pixel that contain absolutly no red, green and blue are set to white.

The following code does the same with a threshold for each color channel:

rgbImage = imread('ecg.png');

redChannel = rgbImage(:, :, 1);
greenChannel = rgbImage(:, :, 2);
blueChannel = rgbImage(:, :, 3);

black = (redChannel<150)&(greenChannel<150)&(blueChannel<150);

redChannel(black) = 255;
greenChannel(black) = 255;
blueChannel(black) = 255;

rgbImageOut = cat(3, redChannel, greenChannel, blueChannel);

imshow(rgbImageOut);
Thomas Sablik
  • 16,127
  • 7
  • 34
  • 62
  • Can you approximate how much you lose data if you have threshold 240? The challenge is to find sufficient good data with a selection of threshold. – Léo Léopold Hertz 준영 Apr 22 '15 at 15:13
  • The 240th picture is this one https://www.dropbox.com/s/pxl99xd8zd64xxt/Screen%20Shot%202015-04-22%20at%2018.12.33.png?dl=0 – Léo Léopold Hertz 준영 Apr 22 '15 at 15:14
  • What do you mean by lost data? Changed pixels of the blue graph? – Thomas Sablik Apr 22 '15 at 15:27
  • I tried this with a treshold for blue 250 and the red and green channels are deleted. No pixel of the graph has changed. No lose of data. – Thomas Sablik Apr 22 '15 at 15:33
  • Actually, there is lost data because of the threshold change of blue color caused by the noise quantization from text and axis as described in rayryeng's answer. You can see some gaps in your blue graph. – Léo Léopold Hertz 준영 Apr 22 '15 at 15:47
  • 1
    You are right. I didn't compared this part of the graph. I only compared the image section (32:438,48:565,:). In this section is no difference. Strictly speaking the graph didn't change because at this point was no graph. – Thomas Sablik Apr 22 '15 at 15:55