When reliable evidence about long established treatments is gathered it often overturns the conventional wisdom; this is clear evidence that evidence based medicine is more effective than ignorance based medicine.
The trouble with the question is that it is, to some extent, self-referential. The point of evidence based medicine is about challenging what standard of evidence we should accept. The question asks us to apply the same standard to the whole approach. But the very act of comparing not using evidence to using evidence seems to accept the premises of the evidence based approach.
But there is one way we can get something like a satisfactory answer: we can look at areas where a common medical practice has been accepted for a long time but where a properly controlled study has eventually been conducted. The book Testing Treatments is a valuebale source of material on the subject and much of the material below is derived from it (the full text is online). But first a quick review of what counts as reliable evidence.
What does reliable evidence look like?
The Gold standard for evidence about treatments is the double-blind randomised trial. This means that a treatment or a comparison (a placebo or an alternative treatment) is given to two groups of patients, selected randomly without either the patients or the trial managers knowing who gets which. The blinding is necessary because of the placebo effect and the potential for investigators to be biased in how they assess the benefits if they know which group is being treated (even unconscious bias can be very significant). Randomisation is necessary to ensure that any observed differences between the groups are due to the differences in treatment not some other characteristic of the patients.
For more detail see this section in Testing Treatments.
Why does anyone resist the application of EBM?
Many people would look at the basic idea of EBM and wonder why it isn't just obvious. But many have opposed it and it is worth asking why.
One explanation which applies particularly to medicine (though it is also often seen in politicians) is what has been called The God Complex. Tim Harford described it in his Ted Talk (it can also be found in his book Adapt) when telling the story of Archie Cochrane (first quote is from a transcript of the talk the second from the book):
Archie Cochrane all his life fought against a terrible affliction and he realized it was debilitating to individuals and it was corrosive to societies and he had a name for it he called it the God Complex. Now I can describe the symptoms of the God Complex very easily. The symptoms of the God Complex are: no matter how complicated the problem you have an absolutely overwhelming belief that you are infallibly right in your solution. Now Archie was a doctor so he hung around the doctors a lot and the doctors suffered from the God Complex a lot...
Cochrane complained about the 'God Complex' of doctors who didn't need to carry out trials because they knew the correct course of treatment - even when other doctors were issuing contradictory advice with equal confidence.
You have to be humble to recognise the need for reliable evidence and doctors are not trained to be humble.
So what sort of practices have been overturned by evidence?
Babies should sleep on their front
From the 1950s to the 1970s the top authority in childcare, Dr Benjamin Spock, made the following recommendation about how babies should sleep:
There are two disadvantages to a baby’s sleeping on his back. If he vomits he’s more likely to choke on the vomitus. Also he tends to keep his head turned towards the same side . . . this may flatten the side of the head . . . I think it is preferable to accustom a baby to sleeping on his stomach from the start.
The argument has both authority and logic in its favour. But it is wrong.
When proper trials were done and the evidence reviewed the following conclusion emerged:
Advice to put infants to sleep on the front for nearly a half century was contrary to evidence available from 1970 that this was likely to be harmful. Systematic review of preventable risk factors for SIDS from 1970 would have led to earlier recognition of the risks of sleeping on the front and might have prevented over 10 000 infant deaths in the UK and at least 50 000 in Europe, the USA, and Australasia.
The earlier you detect something the better the outcome
It seems intuitively obvious that detecting cancers early should improve the outcomes. They should be easy to treat in healthier patients where the disease has not progressed so much thereby improving survival. While the issue applies to many cancers and progressive diseases, a solid case study is provided by our experience with neurblastoma.
As testing Treatments summarises:
Neuroblastoma was a tempting target for screening for four reasons: (1) children who are diagnosed before the age of one year are known to have a better outlook than those who are diagnosed later; (2) children with advanced disease fare much worse than those with early disease; (3) there is a simple and cheap screening test that can be carried out by blotting wet nappies and measuring substances in the urine; and (4) the test detects nine out of ten children with neuroblastoma.
The logic of early screening seemed so good that Japan introduced screening for 6 month old children across the country in 1985. But:
But 20 years later there was no evidence that neuroblastoma screening had reduced the number of children dying from this cancer. How could that be?
There were several problems. One was that trial outcome was judged by counting survival from time of diagnosis. But this is a biased metric: earlier diagnosis makes it look better even if the treatment does nothing to change the date of death or the progression of the disease (survival should have been measured from birth, which is not biased). There are also several types of neuroblastoma: some regress naturally, other progress rapidly. Screening can miss the fast developing type but will often spot the slow and regressing type even though the disease may have disappeared naturally without intervention. But those patients who may not have suffered from the disease at all will suffer the costs of intervention, which are not negligible.
When reliable evidence was collected, the following results emerged:
By contrast, when unbiased evidence was obtained from clinical trials done in Canada and Germany, involving about three million children in all, researchers were unable to detect any benefit from screening, but there were obvious harms. These included unjustified surgery and chemotherapy, both of which can have serious unwanted effects. In the light of this evidence, infant screening for neuroblastoma in Japan was stopped in 2004.
The intuition that early detection must be good is very deep seated and hard to overturn. The controversy affects other areas of screening such as screening for prostate cancer (see Does screening for prostate cancer save lives? ) and breast cancer screening (see the sometimes heated arguments made here Is routine screening for breast cancer for asymptomatic women worthwhile? ).
Steroids will reduce brain swelling in traumatic head injury
Until about ten years ago if you turned up in a hospital emergency department with a traumatic head injury you would probably have been given steroids. The logic is clear: steroids are great drugs for reducing swelling in most parts of the body and brain swelling is serious; surely anything likely to help must be good for the patient?
Sadly, reality again trumps logic. The CRASH trial came to a different conclusion (this summary is from Margaret Mcartney's book The Patient Paradox):
When the results came in, it was found that steroids were neither effective nor neutral. They were actively harmful to the extent that, if you were given them, you were more likely to die. It was calculated that around 10,000 patients would not have died had this research been done sooner and the use of steroids stopped in these circumstances.
One of the criticisms made in the comments to the original answer was that the examples given above involve studies that would take a long time to produce answers. Part of the answer is that, when there is no reliable evidence, we should start systematically collecting information on outcomes as early as possible. Then we will have the results as fast as possible. We don't stop treating in the meantime but we know that the information that will help us improve will eventually emerge. Shockingly this isn't standard medical practice and the idea of collecting evidence in a systematic way is often opposed by the medical community.
But the benefits can be enormous. Atul Gawande tells a story about military medics in chapter 3 of his magisterial book on medicine, Better: a surgeon's notes on performance. He noted that americas military medics had an unusual habit of diligence about recording what they did and what subsequently happened with military casualties. Unusual because record keeping is not the first thing you think of in the high pressure, under-resourced field hospital when injured soldiers arrive. But systematic recording of injury, treatment and outcomes enabled them to half the rate of death in less than a decade with no new technology for treatment. The simple process of observing which treatments were used and analysing their success rates led to a major improvement in outcomes. Gawande remarks towards the end of the chapter:
We do little tracking like this here at home. Ask a typical American hospital what its death and complication rates for surgery were during the last six months and it cannot tell you.
The problem isn't how much effort it takes, the problem is some people just don't want to do it. Those who want to collect reliable evidence often face deep seated opposition. One of Archie Cochrane's early triumphs (See Tim Harford's TED talk or book) was based on the question of whether patients recovering from a heart attack would survive better if kept in a hospital bed or if they were sent home quickly. Conventional medical opinion was so strongly in favour of keeping them in convalescent beds that they opposed the idea of doing a trial as unethical and looked likely to challenge any statistics the might emerge if a trial were ever conducted. Somehow Cochrane managed to start a trial and get some early results. Then he presented the results to the established doctors showing that survival was clearly better in the group staying in bed. The established experts were so convinced by the statistics that they argued the trial should be stopped on ethical grounds. Only then did Cochrane reveal that he had swapped the results in his presentation and it was actually the patients going home who had better outcomes. The God Complex isn't open to evidence and isn't good for patients.
Moreover, evidence can sometimes be collected quickly. And sometimes even simple ideas can yield big improvements.
Checklists for the insertion of central lines in ICUs
Very ill patients are often cared for in Intensive Care Units (ICUs). They don't usually stay there for long periods of time, but the care they receive is intense. Many need central venous catheters inserted. And they run some risk of harm. As Gigerenzer and Gray describe in Better Doctors, Better Patients, Better Decisions:
...each year, central venous catheters cause an estimated 80,000 bloodstream infections and, as a result, up to 28,000 deaths in intensive care units (ICU) in U.S. hospitals. Total cost of these infections are estimated at US$2.3 billion annually. To save lives Peter Pronovost developed a simple checklist of five steps (including hand washing and cleaning the skin with chlorhexidine) for ICU doctors to follow before inserting an IV line to prevent the introduction of bacteria. The checklist reduced the infection rate to almost zero at some one hundred ICUs in hospitals in Michigan....Yet most ICU physicians do not use them.
The original Pronovost study is reported in the NEJM here. Gawande has a popular account in the New Yorker.
The key lessons are that even simple changes in practice (in this case using a simple checklist to ensure higher compliance with known good practice) can yield big improvement and do so quickly. We don't need to wait for large randomised trials. In this trial the benefits are obvious within weeks. But this is still not standard practice in most hospitals.
Checklists to avoid complications in surgery
Surgeons are particularly prone the the God Complex. So they tend to resist interventions that imply they are not omnicompetent. Such as the systematic use of checklists in operating theatres before surgery. The story of checklists is told by Gawande in his book The Checklist Manifesto
Again a simple intervention was tested and resulted in a large and significant improvement in results for patients and did so quickly. The clinical report of the original work is in the NEJM here. The essential idea is a 19 point checklist that is designed to ensure key elements of good practice are not missed by the team in the operating theatre (such as confirmation of the actual procedure, or ensuring adequate supplies of the right type of blood are on hand in case of emergencies).
As the original paper reports:
The rate of death was 1.5% before the checklist was introduced and declined to 0.8% afterward (P=0.003). Inpatient complications occurred in 11.0% of patients at baseline and in 7.0% after introduction of the checklist (P<0.001)
Those are significant improvements and didn't take that long to see. So again EBM doesn't have to wait forever to get results nor are the improvements achievable small.
Summary and conclusions
There are plenty more examples similar to the above. All demonstrate that reliable evidence often overturns accepted medical practice and persuasive logic. That is the point of evidence based medicine: we should not accept conventional wisdom or standard practice. The only way to know for sure what works and what doesn't work is to use high quality, fair, trials. And the profession of medicine should become less of a craft and more of a science where everyone's duty is to collect the data to measure outcomes and continually improve medical practice.
There are enough examples of where such evidence has reversed accepted practice for us to trust evidence based medicine rather than the alternative, ignorance based medicine.
