scipy.stats.anderson test critical values

Question

I used scipy.stats.anderson() to test for normal distribution. my test distribution was not normal distributed, therefore teststatistic > critical value. However, when checking all calculated critical values I observed, that for decreasing p-values critical values are increasing. That means, the more critical the test (smaller p-values), the closer gets the critical value to the test statistic. Which should be the other way, in my opinion. Anyone familiar with Anderson Test and its implementation in Scipy?

Answer 1

The last time I did some spot checking, scipy.stats.anderson worked correctly. For Anderson-Darling test against the normal distribution, statsmodels has a function that returns the p-value http://statsmodels.sourceforge.net/devel/generated/statsmodels.stats.diagnostic.normal_ad.html

The critical values are for a given significance level. When we want a smaller significance level, then we have to increase the critical values, assuming we are in the right, upper tail of the distribution.

http://en.wikipedia.org/wiki/Statistical_significance

For example for a z-test, the one-sided upper tail critical values based on the normal distribution are:

>>> from scipy import stats
>>> stats.norm.ppf([0.9, 0.95, 0.975])
array([ 1.28155157,  1.64485363,  1.95996398])

P-values, in contrast, calculate the tail probability at the given observed value, the larger the observed value, the smaller is the p-value, again in the case of the right, upper tail.

In your example, if the observed value is above the critical value of the 10% level, then it could still be below the critical value of the 5% level. In this case we would reject the null hypothesis at the 10% but not at the 5% level.