3

As you probably know, the James Randi educational foundation offers 1 000 000$ for "any psychic, supernatural, or paranormal ability under satisfactory observation".

But sometimes such claimed ability is said to be "not working all the time". Like in, say, dowsing or astrology -- it just increases the probability (compared to random guessing) of getting something right.

Still, such claims can be tested using statistical significance testing techniques. The "confidence" in the results of such statistical tests is called "the statistical significance" -- a probability that the claimant does not have any supernatural abilities and just guessed the right answers.

The probability that one have to beat in the "one million challenge" seems to be equal to 1 in 100000.

But some opponents of James Randi say that in order to start using the supernatural effects in practice ("for the benefit of Mankind" so to say) such significance is too high. "Some researchers", they say, "are using significances of 5%. And still making progress."

I'm working in particle physics, and in my field in order to establish a discovery we require "5 sigma", which amounts to significance of about 1 in two million. So, for me Randi's requirement is too soft.

But what about other fields? Like medicine or biology? Or even sociology and psychology? Is there any overview of the significances for these disciplines and what is considered "a discovery" for them?

Sklivvz
  • 78,578
  • 29
  • 321
  • 428
Kostya
  • 637
  • 6
  • 9
  • 2
    [Questions *about* skepticism are not on topic](http://skeptics.stackexchange.com/help/on-topic). This site is a place to *practice* scientific skepticism. However, you have enough reputation to discuss this interesting topic at [chat]. –  Sep 07 '13 at 18:01
  • @Sancho I don't feel like I'm asking about general skepticism -- It can definitely be answered with concrete facts and references. I'd agree that it is rather broad, but I have no idea how can I narrow it down. – Kostya Sep 07 '13 at 21:12
  • 1
    Well, at minimum, you need to specify what the notable claim is that you want us to skeptically examine. Provide a quote of the claim, and a link to where you found it. –  Sep 07 '13 at 21:47
  • You want Sellke, Thomas; Bayarri, M. J.; Berger, James O. (2001). "Calibration of p Values for Testing Precise Null Hypotheses". But the short form: Natural sciences use very very low p values, Humanities use 5%, psychology and medicine are all over the place (There might not be enough patients for any given syndrome to establish a good P value even with very rigorous experimental procedures and very clear evidence.) – shieldfoss Sep 07 '13 at 23:45
  • 1
    I is also true that the 5 sigma requirement in particle physics is an artifact of the bump-hunting era and some people (Murray Goodman among them) are arguing for it to be apply to situations like bump-hunting where you don't know what you are looking for but not when looking for a *particular* signal (there are good reasons for this, the statistics of the two situations are very different). – dmckee --- ex-moderator kitten Sep 08 '13 at 16:35

0 Answers0