Inconsistency between gaussian_kde and density integral sum

Question

Can one explain why after estimation of kernel density

d = gaussian_kde(g[:,1])

And calculation of integral sum of it:

x = np.linspace(0, g[:,1].max(), 1500)
integral = np.trapz(d(x), x)

I got resulting integral sum completely different to 1:

print integral
Out: 0.55618

Also the estimated Guassian kernal will extend past `g[:,1].max()` as the Gaussian distribution has an infinite support. Try extending both boundaries of `x` by the range of the data in `g[:,1]`. — cr1msonB1ade, Jun 22 '15 at 15:22
with x = np.linspace(g[:,1].min(), g[:,1].max(), 1500) the result has changed unsignificantly: 0.55642 — Ivan Sudos, Jun 22 '15 at 15:29
Try: `rang=g[:,1].max() - g[:,1].min()`, `x = np.linspace(g[:,1].min() - rang, g[:,1].max() + rang, 1500)` — cr1msonB1ade, Jun 22 '15 at 15:32
That's helped. After density estimation the support range occurs to be larger than initial data range. — Ivan Sudos, Jun 22 '15 at 15:41

score 3 · Accepted Answer · answered Jun 22 '15 at 15:47

As stated in my comment, this is an issue with kernel density support. The Gaussian kernel has infinite support. Even fit on data with a specific range the range of the Gaussian kernel will be from negative to positive infinity. That being said the large majority of the density will lie within a range reasonably around the range of the fitted data.

If you would like a Gaussian kernel fitted within the range of your original data, you can fit a truncated Gaussian kernel by truncating the kernel and re-normalizing the truncated portion to integrate to 1, but I am not sure if that is what you want here. You can also truncate to a non-negative Gaussian kernel with similar logic.

Inconsistency between gaussian_kde and density integral sum

1 Answers1