12

An organization called Freedom Health Canada makes an extraordinary claim on their website, which can be found here.

“Scientifically controlled clinical trials have shown that only 10 -20 % of all procedures used in present-day medical practices are of benefit to patients. It was concluded that the vast majority of medical procedures now being utilized routinely by physicians are "unproven" when subjected to the same rigid standards these same orthodox physicians are demanding of alternative, nutritional practitioners.” - U.S. Office of Technology Assessment

Although the claim is certainly questionable in my opinion, they do quote the U.S. Office of Technology Assessment, which is legitimate.

My question is: Was this claim ever made by the U.S. Office of Technology Assessment, and could the claim be true?

Christian
  • 33,271
  • 15
  • 112
  • 266
LanceLafontaine
  • 1,012
  • 9
  • 23
  • The complete archives of the OTA can be found online [here](http://www.fas.org/ota/) so someone with the patience could search. It also means that Freedom Health Canada could tell you which report the statement was in, if they actually wanted you to follow up on it. Draw your own conclusions. – DJClayworth May 28 '12 at 16:46
  • 3
    [Here](http://www.fas.org/ota/2010/05/12/how-scientific-is-modern-medicine/) is an OTA article, and [here](http://www.fas.org/ota/reports/7805.pdf) is the orignal report (which is from 1978). – Oliver_C May 28 '12 at 16:57
  • @Oliver_C You should make that an answer. It's better than mine. – DJClayworth May 28 '12 at 17:16
  • Should this have the united-states tag on it? I'm looking at the answers, and they seem specific to the USA, even though Freedom Health Canada is making the claim. – Oddthinking May 31 '12 at 01:08
  • @Oddthinking, I noticed the same; I've added the tag. – LanceLafontaine May 31 '12 at 01:31
  • There's a question (and answer) on this in the FAQ for The Cochrane Collaboration - http://www.cochrane.org/faq/general#t86n547 – Tom77 May 31 '12 at 09:46
  • @Christian What is [tag:philosophy-of-ebm]? That tag has no description. – Reinstate Monica -- notmaynard Apr 08 '16 at 13:33
  • @iamnotmaynard : I added a tag description but it's not approved at this point in time. EBM is Evidence-Based Medicine. This question is about the extend to which evidence-based medicine is practiced and thus I find the tag applicable. – Christian Apr 08 '16 at 13:37

2 Answers2

22

The quote attributed to the OTA is untrue, both in its exact words and in its intended meaning.

With thanks to Oliver C, the quote that they are referring to is from this report. The full quote is:

It has been estimated that only 10 to 20 percent of all procedures currently used in medical practice have been shown to be efficacious by controlled trial.

In short that does not mean "we did controlled tests on all medical procedures, and only 10-20% were found to be effective", it means "we have only done enough controlled tests to show that 10-20% of procedures were effective". That is very different from what FreedomHealthCanada is claiming.

Note first that this is from 1978, and things have changed a lot since then. Second the OTA did NOT think this was a good state of affairs, even then, and wanted more clinical trials. Quite a lot of procedures are of course known to be effective without the benefit of controlled trials. There are also a large number of procedures where it would be unethical to do a controlled trial - such as performing CPR on only half the patients with a stopped heart to see how many recovered.

DJClayworth
  • 57,419
  • 26
  • 209
  • 195
6

While the specific claim is exaggerated and doesn't support the argument for alternative medicine, there is evidence that a disturbing proportion of conventional medicine is unproven and wasteful

When I first saw the headline question I thought it raised an interesting and challenging issue for modern medicine. Then I read the Freedom Health Canada argument which seems to want to make a case for applying "alternative" medicine. So while they raise interesting criticisms, the argument that conventional medicine hasn't been proved to work so you should be prepared to try other forms of medicine that haven't been proved to work, isn't coherent.

But there is legitimate and skeptical criticism of much of what is done in the western medical establishment which is sometimes shockingly complacent when it comes to demonstrating the need and value of the treatments it offers. Some examples are given below.

A good review of some of the key issues is found in the recent books Testing Treatments and Better Doctors, Better Patients, Better Decisions. The first chapter of the second book starts by quoting two extraordinary examples where the US health system overuses screening in expensive and damaging ways:

Almost ten million U.S. women have had unnecessary Pap smears to screen for cervical cancer--unnecessary because, having already undergone complete hysterectomies, these women no longer have a cervix. [Original JAMA paper here]

Every year, one million U.S. children have unnecessary CT scans. An unnecessary CT scan equates to more than a waste of money: an estimated 29,000 cancers result from the approximately 70 million CT scans performed annually in the United States. [NEJM reference here]

While we tend to casually assume that medical treatment is good when it happens and when it doesn't happen it is because of rationing, that idea can be tested and that testing provides an excellent test of whether the treatment benefits patients in the first place. Natural experiments exist because the amount of specific types of activity varies widely in different regions of most countries. For example, the number of hip replacements per 1,000 people varies by a factor of more than 5 in different parts of england.

The most thorough analysis of such variation and the best analysis of the resulting cost and health gain for patients is found in the ongoing work of the Dartmouth Atlas Project for US healthcare (website here). Their work has uncovered vast degrees of variation in what activity gets done and has demonstrated that more healthcare is often worse for patients. In their own words (but my emphasis to show how their work is relevant to the question):

Supply-sensitive care refers to services where the supply of a specific resource has a major influence on utilization rates. The frequency of use of supply-sensitive care is not determined by well-articulated medical theory, much less by scientific evidence; rather, it is largely due to differences in local capacity, and a payment system that ensures that existing capacity remains fully deployed. Simply put, in regions where there are more hospital beds per capita, patients will be more likely to be admitted to the hospital. In regions where there are more intensive care unit beds, more patients will be cared for in the ICU. More specialists will result in more visits to specialists. And the more CT scanners are available, the more CT scans patients will receive. The Dartmouth Atlas has consistently demonstrated these relationships.

In regions where there are relatively fewer medical resources, patients get less care; however, there is no evidence that these patients are worse off than their counterparts in high-resourced, high-spending regions. Patients do not experience improved survival or better quality of life if they live in regions with more care. In fact, the care they receive appears to be worse. They report being less satisfied with their care than patients in regions that spend less, and having more trouble getting in to see their physicians. Most studies have found that mortality is no better in higher in high-spending regions, almost certainly because the benefits to some patients are counterbalanced by the harms to others. Hospitals can be dangerous places, where patients face the risk of medical error, adverse events, and hospital-acquired antibiotic-resistant infections. As more physicians get involved in a patient’s care, it becomes less and less clear who is responsible, and miscommunication and mistakes become more likely. Greater use of diagnostic tests increases the risk of finding -- and being treated for -- abnormalities that are unlikely to have caused the patient any problem. Patients who receive care for conditions that would have never caused a problem can only experience the risk of the intervention.

These examples show that much activity in conventional medicine cannot be justified by evidence. More detailed evidence could show that whole classes of activity are hard to justify, but the above examples are simpler.

So, though the original claim exaggeration is a failed attempt to advocate alternative medicine, there are legitimate and important reasons to be skeptical of the evidence base about much conventional medical activity.

matt_black
  • 56,186
  • 16
  • 175
  • 373
  • 2
    This is quite an essay, and would be good if it were answering the question. – DJClayworth May 31 '12 at 15:16
  • @DJClayworth I plead guilty. I think I'm really answering a broader question. The **headline** points towards my answer but the **content** of the question is much more specific. – matt_black May 31 '12 at 15:23
  • 2
    With respect, this is not a debate site. What's to stop someone else posting a long essay answer giving all the good reasons why orthodox medecine does in fact work? Are you going to post another answer rebutting them? Are they going to past an answer rebutting yours? If you can find a specific claim that your post is an answer to then you might think about posting it as a question. – DJClayworth May 31 '12 at 15:29