specific outliers on a heat map- matplotlib

Question

I am generating a heat map with data that has a fixed outlier number and I need to show these outliers as a colour out of the colour palette of the cmap I use which is "hot". With the use of cmap.set_bad('green') and np.ma.masked_values(data, outlier), I get a plot which looks right but the color bar is not getting synced with the data properly even if I use cmap.set_over('green'). Here is the code I have been trying:

plt.xlim(0,35)
plt.ylim(0,35)
img=plt.imshow(data, interpolation='none',norm=norm, cmap=cmap,vmax=outlier)

cb_ax=fig.add_axes([0.85, 0.1, 0.03, 0.8])

cb=mpl.colorbar.ColorbarBase(cb_ax,cmap=cmap,norm=norm,extend='both',spacing='uniform')
cmap.set_over('green')
cmap.set_under('green')

Here is the data (outlier is 1.69 obviously):

Data;A;B;C;D;E;F;G;H;I;J;K    
A;1.2;0;0;0;0;1.69;0;0;1.69;1.69;0    
B;0;0;0;0;0;1.69;0;0;1.69;1.69;0    
C;0;0;0;0;0;1.69;0;0.45;1.69;1.69;0.92    
D;1;0;-0.7;-1.2;0;1.69;0;0;1.69;1.69;0    
E;0;0;0;0;0;1.69;0;0;1.69;1.69;0    
F;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69    
G;0;0;0;0;0;1.69;0;0;1.69;1.69;0    
H;0;0;0;0;0;1.69;0;0;1.69;1.69;0    
I;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69
J;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69
K;0;0;0;0;0;1.69;0;0;1.69;1.69;0

Appreciate any help

Joe's answer explains what you need to do. I recommend following it. — Paul H, Feb 27 '14 at 23:04

Joe Kington · Accepted Answer · 2014-02-27T22:10:16.363

What's happening is that you're using a masked array where the outliers are masked.

Therefore, they don't show up on the colorbar as being "over". (i.e. as far as matplotlib is concerned, the masked values are invalid, not over the threshold)

As a stand-alone example to reproduce your problem:

import numpy as np
import matplotlib.pyplot as plt

threshold = 0.8
data = np.random.random((10,10))
data = np.ma.masked_greater(data, threshold)

fig, ax = plt.subplots()
im = ax.imshow(data, cmap=plt.cm.hot, interpolation='none')
cbar = fig.colorbar(im, extend='max')
cbar.cmap.set_over('green')

plt.show()

enter image description here

If we simply don't make this a masked array, and instead specify the vmax kwarg to imshow:

import numpy as np
import matplotlib.pyplot as plt

threshold = 0.8
data = np.random.random((10,10))

fig, ax = plt.subplots()
im = ax.imshow(data, cmap=plt.cm.hot, interpolation='none', vmax=threshold)
cbar = fig.colorbar(im, extend='max')
cbar.cmap.set_over('green')

plt.show()

enter image description here

Basically, this is the difference between set_over (or under) and set_bad.

If you did still want to use a masked array, you could just call cbar.cmap.set_bad('green') as well as set_over, and you'd get the effect you want (though all "bad" values, not just ones over the threshold, would be green). If you take that route, you'll need to manually specify the vmax. Otherwise it will be taken as the maximum of the unmasked portions of the array.

I used the vmax as the outlier value but its still the same output. I have edited my question to include the code I am running now and also data that I working with!! — user2998764, Feb 27 '14 at 22:49

Paul H · Answer 2 · 2014-02-27T23:02:44.160

1

I think you need to set extend to "both" and feed in a Normalize object:

import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
import pandas

from io import StringIO # python 3
#from StringIO import StringIO # python 2

datastring = StringIO("""\
Data;A;B;C;D;E;F;G;H;I;J;K
A;1.2;0;0;0;0;1.69;0;0;1.69;1.69;0
B;0;0;0;0;0;1.69;0;0;1.69;1.69;0
C;0;0;0;0;0;1.69;0;0.45;1.69;1.69;0.92
D;1;0;-0.7;-1.2;0;1.69;0;0;1.69;1.69;0
E;0;0;0;0;0;1.69;0;0;1.69;1.69;0
F;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69
G;0;0;0;0;0;1.69;0;0;1.69;1.69;0
H;0;0;0;0;0;1.69;0;0;1.69;1.69;0
I;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69
J;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69
K;0;0;0;0;0;1.69;0;0;1.69;1.69;0
""")

threshold = 1.68
data = pandas.read_table(datastring, sep=';', index_col='Data')
cmap = mpl.cm.coolwarm
norm = mpl.colors.Normalize(vmin=-1 * threshold, vmax=threshold)
cmap.set_over('slategray')
cmap.set_under('forestgreen')

fig, ax = plt.subplots()
ax.set_aspect('equal')
cb_ax=fig.add_axes([0.85, 0.1, 0.03, 0.8])
img = ax.imshow(data, cmap=cmap, norm=norm, interpolation='none')
cb = mpl.colorbar.ColorbarBase(cb_ax, cmap=cmap, norm=norm, extend='both')

Gives me: enter image description here

edited Feb 27 '14 at 23:02

answered Feb 27 '14 at 21:18

Paul H

65,268
20
159
136

I added the normalize object: norm = mpl.colors.Normalize(vmin=negative_outlier,vmax=outlier) cmap.set_over('green') img=plt.imshow(data, interpolation='none', cmap=cmap,norm=norm) cb=mpl.colorbar.ColorbarBase(cb_ax,cmap=cmap,norm=norm,extend='both',spacing='uniform') But its still the same result... white space instead of green – user2998764 Feb 27 '14 at 21:39
NO it didn't, its still giving me white spaces for all the outliers instead of green. The colour bar is fixed with norm object but not the heat map. I still have to use set_bad and masking properties to make it work. Don't know what is wrong as the map you generated incorporates these outliers without set_bad..(I have edited my post to include the changes you mentioned) – user2998764 Feb 27 '14 at 21:50
@user2998764 well i've taken you about as far as i can without seeing your data. edit your question to include it. – Paul H Feb 27 '14 at 21:56
@user2998764 see my modified response – Paul H Feb 27 '14 at 23:03

specific outliers on a heat map- matplotlib

2 Answers2

Linked