3

I'm using SciPy's stats.gaussian_kde function to generate a kernel density estimate (kde) function from a data set of x,y points.

This is a simple MWE of my code:

import numpy as np
from scipy import stats

def random_data(N):
    # Generate some random data.
    return np.random.uniform(0., 10., N)

# Data lists.
x_data = random_data(100)
y_data = random_data(100)

# Obtain the gaussian kernel.
kernel = stats.gaussian_kde(np.vstack([x_data, y_data]))

Since I'm not setting a bandwidth manually (via the bw_method key), the function defaults to using Scott's rule (see function's description). What I need is to obtain this bandwidth value set automatically by the stats.gaussian_kde function.

I've tried using:

print kernel.set_bandwidth()

but it always returns None instead of a float.

Gabriel
  • 40,504
  • 73
  • 230
  • 404

3 Answers3

10

Short answer

The bandwidth is kernel.covariance_factor() multiplied by the std of the sample that you are using.

(This is in the case of 1D sample and it is computed using Scott's rule of thumb in the default case).

Example:

from scipy.stats import gaussian_kde
sample = np.random.normal(0., 2., 100)
kde = gaussian_kde(sample)
f = kde.covariance_factor()
bw = f * sample.std()

The pdf that you get is this:

from pylab import plot
x_grid = np.linspace(-6, 6, 200)
plot(x_grid, kde.evaluate(x_grid))

enter image description here

You can check it this way, If you use a new function to create a kde using, say, sklearn:

from sklearn.neighbors import KernelDensity
def kde_sklearn(x, x_grid, bandwidth):
    kde_skl = KernelDensity(bandwidth=bandwidth)
    kde_skl.fit(x[:, np.newaxis])
    # score_samples() returns the log-likelihood of the samples
    log_pdf = kde_skl.score_samples(x_grid[:, np.newaxis])
    pdf = np.exp(log_pdf)
    return pdf

Now using the same code from above you get:

plot(x_grid, kde_sklearn(sample, x_grid, f))

enter image description here

plot(x_grid, kde_sklearn(sample, x_grid, bw))

enter image description here

nivniv
  • 3,421
  • 5
  • 33
  • 40
  • see my updated comment. what you are using is not the bandwidth, it is the factor to be multiplied with the std of the sample – nivniv Mar 08 '16 at 18:34
  • Are you aware that the arguments `bw_method` in `scipy.stats.gaussian_kde` and `bandwidth` in `sklearn.neighbors.KernelDensity` do not represent the same thing? Please see this question: http://stackoverflow.com/q/21000140/1391441 – Gabriel Mar 08 '16 at 19:05
  • You were asking for the bandwidth, weren't you? covariance_factor is not the bandwidth – nivniv Mar 08 '16 at 19:10
  • I call it 'bandwidth' because the docs call `bw_method` "The method used to calculate the estimator bandwidth". You are probably right statistically speaking, although I'm not sure what your answer is supposed to do. Are you saying my accepted answer is not correct? – Gabriel Mar 08 '16 at 19:13
  • 2
    Well it depends what you ask. If you ask how to get the factor, then your answer is right. But if you want to know what the "bandwidth" you'll have to use what I wrote. "Bandwidth" is a technical term for a parameter used by kernel-density-estimators (you can check it here https://en.wikipedia.org/wiki/Kernel_density_estimation). If you give bw-method a scalar, it is used to calculate the bandwidth in the manner written above (it is multiplied by the std), but it is not the bandwidth itself. – nivniv Mar 08 '16 at 19:27
1

I've got it, the line is:

kernel.covariance_factor()

From scipy.stats.gaussian_kde.covariance_factor:

Computes the coefficient (kde.factor) that multiplies the data covariance matrix to obtain the kernel covariance matrix. The default is scotts_factor. A subclass can overwrite this method to provide a different method, or set it through a call to kde.set_bandwidth.

One can check that the resulting kernel using this bandwidth value is equivalent to the kernel generated using the default bandwidth. To do this obtain a new kernel with the bandwidth given by covariance_factor(), and compare its value on a random point with the original kernel:

kernel = stats.gaussian_kde(np.vstack([x_data, y_data]))
print kernel([0.5, 1.3])

bw = kernel.covariance_factor()    
kernel2 = stats.gaussian_kde(np.vstack([x_data, y_data]), bw_method=bw)
print kernel2([0.5, 1.3])
Gabriel
  • 40,504
  • 73
  • 230
  • 404
  • 1
    bw_method parameter is used as the factor, therefore setting the factor will give you the same results. but this is not the "bandwidth". I added an answer – nivniv Mar 08 '16 at 18:33
0

I came across this old question since I was also interested in knowing what was the bandwidth used for gaussian_kde from Scipy. I would like to add/amend previous answers that the covariance factor is used in kde.py code from Scipy as: self.covariance = self._data_covariance * self.factor**2

Therefore the full kernel covariance is the sample covariance times the square of the so-called covariance factor (Scott factor) which can be retrieved by kde.factor or kde.covariance_factor().

Manuel
  • 1