-1

Hi I'm new to statistics and just wanted some clarifications on p-values. So far I've learned that if we use a 5% significance level then we reject the null hypothesis and accept the alternative hypothesis if the p-value is less than 0.05.

If the p-value is greater than 0.05 then we say there is insufficient evidence and we can't reject the null hypothesis. I've learned that we can't accept the null hypothesis if p-value is greater than 0.05 but at the same time if we have a strong p-value we can't ignore it

So my question it what is considered a high p-value where I should consider accepting the null hypothesis, like where should I cut off at 0.7 and higher? 0.8? 0.9?

statsnewbie
  • 3
  • 2
  • 1
    Consider reading [this statement by the ASA on p-values](http://amstat.tandfonline.com/doi/pdf/10.1080/00031305.2016.1154108?needAccess=true), or check out a [shorter version](http://www.amstat.org/asa/files/pdfs/P-ValueStatement.pdf) – bouncyball Mar 20 '17 at 18:06
  • I'm voting to close this question as off-topic because it is about statistics and [math.se] instead of programming or software development. – Pang Mar 22 '17 at 04:59
  • It might be a better fit for CrossValidated... but then its definitely got the right tags on it, and those tags must exist on SO for a reason... – 4Oh4 Mar 26 '17 at 00:33

1 Answers1

1

Can't argue with the link to the ASA statement.

An example that helped me with this: If you are working to a 5% significance level (alpha=0.05), and calculate a p-value of 0.5, your data does not provide sufficient evidence to reject the null hypothesis.

There are two possible scenarios here:

  1. The null hypothesis is indeed true
  2. The null hypothesis is actually false (Type II error, false negative)

Once that point has been reached, you shouldn't do much more with the p-value. It is tempting to try to justify inconvenient results by saying that (for example) a p-value of 0.07 is quite close to 0.05, so there is some evidence to support the alternative hypothesis, but that is not a very robust approach. It is good practice to set your significance level in advance, and stick to it.

As a side-note, significance levels are an expression of how much uncertainty in your results you are willing to accept. A value of 5% indicates that you are willing (on average, over a large number of experiments) to be wrong about 5% of the time, or 1 in 20 experiments. In this case, by 'wrong' we mean falsely reject a true null hypothesis in favour of the alternative hypothesis (that is not true). By increasing the significance level we are saying we are willing to be wrong more often (with the trade-off of having to gather less data).

4Oh4
  • 2,031
  • 1
  • 18
  • 33