55

I was curious about this thrice-upvoted comment, which says: Surgeons are also known for refusing to operate in order to maintain their ratios. So I Googled it and it seems surgeons can refuse to operate, but it's not immediately obvious they do so to "maintain their ratios". I recall something along these lines in the movie Doctor Strange:

Billy: I have a 68-year-old female with an advanced brain stem glioma.
Doctor Strange: Yeah, you want me to screw up my perfect record? Definitely not.
Doctor Strange quote.

This makes me suspect it might be an urban legend. A bit more digging yields a published claim along these lines:

Usually, most experienced surgeons operate in the most challenging cases, thereby distorting their mortality statistics. For patients, these statistics represent mainly a surgeon’s skills and competence. Therefore, “gaming” occurs: Surgeons may refuse to operate on high-risk patients or may pass the difficult cases to colleagues to keep their own statistics clean196,197.
Johanna Ruohoalho, Complications and Their Registration in Otorhinolaryngology – Head and Neck Surgery, PhD Thesis, University of Helsinki, 2018 (pdf, p.56)

Unfortunately, I don't have free access to either of the references above (one is a book, and one is behind a paywall).

Question: Do surgeons refuse to operate "to keep their own statistics clean"?

Rebecca J. Stones
  • 7,948
  • 6
  • 37
  • 68
  • Yes, see [this New York Times article](https://www.nytimes.com/2003/03/16/magazine/when-doctor-s-slam-the-door.html) discussing this at length. “_[…] The researchers concluded that surgical report cards in New York and Pennsylvania led to substantial selection bias by surgeons […]_” – Dan Romik Jan 06 '22 at 09:01
  • 24
    This may be something that doesn't *generalise*. Some health systems may have incentives that drive this behaviour, but there are many systems and some may have no such incentives. So the answer might be "it happens in some systems but not in others". – matt_black Jan 06 '22 at 09:46
  • 21
    It may be worth pointing out that Dr. Stephen Strange at the beginning of the movie is presented as an unlikeable character. It's possible that rejecting patients so he looks better is another facet of his narcissistic personality. – Engineer Toast Jan 06 '22 at 15:49
  • 8
    This question is framed as if the practice in question is a bad thing. If your surgeon isn't confident that they'll be successful, don't you *want* them to decline and pass you to someone more experienced? They have an obligation to do no harm, so it seems natural that they'd pass on cases where they're more likely to harm than heal. This only really becomes a problem if *nobody* will do the surgery, but that's not really something you can track using per-surgeon statistics. – bta Jan 06 '22 at 16:17
  • Depending on the organisation of a countries health system a surgeon might not even have the opportunity to choose their patients. They are simply on duty and operate whatever patients is due at that time. – quarague Jan 06 '22 at 16:51
  • Just to clarify, how many instances do we need to say yes? Just one? – tuskiomi Jan 06 '22 at 17:28
  • 6
    @bta: I would want them to decline for someone more experienced. I believe this question is more about _experienced_ surgeons declining risky surgeries and then _inexperienced_ surgeons taking them on. – Mooing Duck Jan 06 '22 at 18:31
  • 1
    @MooingDuck there's a similar logic even if you've got the best surgeon ever though. Surgical results are a game of odds, not one of guarantees. If the best surgeon in the world says "no, the risk doesn't outweigh the benefits", are they "padding their stats", or are they protecting the patient from harm? Both at the same time? Because's that really what the statistic is... "patients harmed by surgery"... – mbrig Jan 06 '22 at 21:11
  • So the "real question" becomes "are surgeons acting conservatively when they shouldn't, because of the optics of failure as opposed to the true expected/utilitarian outcomes?" but that's a nightmare of motivations, philosophy, and personal views/knowledge to untangle. How many successful high risk surgeries balances out the patient that doesn't wake up? – mbrig Jan 06 '22 at 21:17
  • 3
    This seems very much related to a recent question whether 32% more women end up dead after surgery by a male doctor - different degrees of risk aversion between female and male doctors would be a possible explanation. – gnasher729 Jan 07 '22 at 10:10
  • 5
    @mbrig What if the surgeon says "Without surgery, the patient will die. With surgery, they have a 50% chance to die or survive. I won't perform the surgery to keep my statistics clean", that's one thing. If the surgeon says "You are at high risk of complications, and giving you these (totally unneccessary) breast implants has a ten percent chance of killing you, I won't do it" that's a completely different and very reasonable and commendable thing to say. – gnasher729 Jan 07 '22 at 10:14
  • This very point came up in an episode of "Scrubs" - although that is far from a documentary they did often reflect some of the real situations encountered in medicine. – Grimm The Opiner Jan 07 '22 at 16:23
  • @gnasher729 well yes, if they made it blatantly obvious thanks to an unrealistic contrived scenario, then it would be easy to tell they were doing it, but that's not generally what happens in the real world. But not doing it in the most obvious of cases doesn't mean they aren't doing it in the non-obvious cases. – mbrig Jan 07 '22 at 20:32
  • Can you Post any evidence, or not? – Robbie Goodwin Jan 08 '22 at 00:28
  • 1
    A de-paywalled copy of the second reference you couldn't access is available [here](https://sci-hub.se/10.1111/codi.13433). – Vikki Jan 08 '22 at 01:36
  • @mbrig What is contrived about my example? One is where surgery is objectively better for the patient, and a surgeon might refuse because he is worried about his statistics. The other is where surgery is objectively not recommendable for the patient, and a surgeon might refuse because he is worried about his statistics _AND_ about causing damage to the patient. – gnasher729 Jan 10 '22 at 12:35
  • 1
    @gnasher729 A dangerous surgery needed to prevent imminent death is not a common scenario outside of maybe trauma surgery, and that's done in a setting where surgeons aren't refusing patients. For everything else, there's a tradeoff of "die on the table right now" vs "live some amount of time with the condition". And again, examining the most egregious possible scenarios is not likely to give a useful answer to OP's question: just because surgeons aren't doing something blatantly unethical to protect their stats doesn't mean they aren't doing it in subtle ways. – mbrig Jan 10 '22 at 17:27

1 Answers1

65

Risk aversion in surgeons is widely believed to be true but the evidence suggests it isn't universal

The potential for surgeons to behave in risk averse ways occurs when surgeon level outcome data is published. This clearly creates an incentive to select patients whose risk of complications is low so that the published statistics for the surgeon look better.

But–especially when the evidence is handled carefully including proper risk-adjustment and careful presentation–it is less clear that the incentive to be risk-averse is clear and simple. If risk-adjustment can be done well and presented well so even the public can understand the results, there may be little real incentive to avoid risky patients.

The belief that open publication of outcome data is a problem is clearly widespread. This 2014 article in the BMJ is one of a series arguing against surgeon-level outcome data for related reasons (note the date for context later).

The UK experience of using surgeon-level data is a useful story. In the 1990s a whistleblower exposed a serious problem with heart surgery in children. The Bristol Heart scandal as it is known exposed serious deficiencies in surgical competence in one department that had been hidden for some time because there was no open publication of surgeon-level outcome data. One outcome of the eventual enquiry prompted the NHS to promote open publication of risk-adjusted surgical outcomes for all cardiac surgeons. The profession resisted this at the time (largely because of the supposed incentives for surgeons to "game" their numbers by risk-averse selection of patients) but they were browbeaten into accepting the proposal by the NHS Medical Director, Bruce Keogh and public pressure from the Bristol scandal.

But the early results of the use of surgeon-level data did not create the expected effects. One BMJ report on the results in 2009 "Survival after heart surgery continues to improve after publication of mortality data" reports:

Outcomes for adult cardiac patients have improved substantially in the past five years, even though more elderly and high risk patients are now being treated. Mortality rates after coronary artery surgery have fallen by 21% and for isolated valves by a third, a study by the Society for Cardiothoracic Surgery of Great Britain and Ireland has found...

Ben Bridgewater, consultant cardiac surgeon at the University Hospital of South Manchester and author of the report, said, “One of the benefits we are now seeing from public reporting of outcomes is not just about bringing poor performers ‘into the pack’ but improving the performance of the pack as a whole. The very act of auditing services brings about improvements as centres learn from one another.”

And these results were so convincing that the Royal College of Surgeons argued that other specialties should also openly publish their data (my emphasis):

The results have prompted the Royal College of Surgeons of England to urge all surgical specialties to follow the lead as soon as possible and publish mortality data.

The college said that although critics expected that publishing mortality data would lead to risk averse behaviour from surgeons, with the most sick and elderly patients denied surgery for fear of worsening the statistics, in practice, the opposite has turned out to be true.

Note that this report was published 5 years before the 2014 article arguing the opposite. As one critic responding to the 2014 argument summarised well (my highlighting):

The article by Steve Westaby dealing with publication of surgical mortality data is at best a superficial examination of the issue and at worst misleading. The evidence has been irrefutable for many years that the collection and feedback of risk-adjusted mortality data will reduce mortality rates by between 40% and 24% in cardiac surgery and other specialties. The issue of ‘gaming’ cardiac surgery outcomes by avoiding high risk patients was proposed and investigated at the time of the initial publication of the results and no evidence was found to support the assertion. More recently it has been suggested that without the knowledge of their own mortality or complication rates and that of their colleagues it is not possible for surgeons to obtain full informed consent from their patients. Thus complication and mortality rates become a tool for continuous quality improvement, patient information and informed consent rather than a stick with which to beat the surgeon.

The UK data and discussion illustrates two key things. One is that many in the profession still believe that open data promotes risk-aversion. But the second is that this is refuted by the actual experience of publishing the data.

Whether this is universal in different health systems is unclear. The NHS has few if any direct financial incentives for its surgeons but the US system has strong financial incentives which might promote the pursuit of volume rather than quality. It is clear, though, from the UK data that the expected risk-aversion does not always occur when data is published and that publication can lead to systematic improvement over time.

matt_black
  • 56,186
  • 16
  • 175
  • 373
  • Worth noting also that the GMC hates doctors and even trivial complaints to them can tie a doctor up for years and hundreds of thousands of GBP in costs, so while there isnt a direct financial incentive for NHS doctors theres both an indirect one and a licensing one (the GMC is the UK licensing body, and its quick to revoke and slow to reinstate). – Moo Jan 06 '22 at 20:59
  • 26
    This may or may not be apropos, but there was a study some time ago, in the US, that clearly showed that a great deal of the success/failure rates of a surgeon was actually due to the presence/absence of a _stable surgical team_ that worked together _frequently_. Practice does indeed make perfect (or as close as you can come in an operating room). Hospitals that did frequent heart (whatever) transplants and had a stable team that did them week after week had much better results than those where the same operations were less frequent, or the team changed. So is rating only the surgeon ok? – davidbak Jan 07 '22 at 02:22
  • 1
    Do you think a random patient has enough numeracy to understand the mortality data? I highly doubt that. The common-folk can barely wrap their heads around correlation vs causation. – Nelson Jan 07 '22 at 03:36
  • 5
    @davidbak The issue of whether the team was more important than the surgeon was a big argument used by opponents of rating in the UK. "It's so unfair" they argued. But who is in charge of the team? Part of the reason this lost in the UK was that teams and hospitals had conspired to hide the incompetence of some surgeons in the Bristol scandal partly because the strong leaders had a huge influence on their teams. Transparency incentivises the leader to ensure they have a competent team. – matt_black Jan 07 '22 at 09:25
  • 5
    @nelson Yes patients can understand the data, if it is presented to them in careful, well-designed ways. This was carefully done in the UK with clear, simple visuals and robust analysis not raw data. Some argued that only the professionals were equipped to interpret mortality data. But the Bristol scandal showed that the reaction of some professionals was to hide the data they understood rather than acting on it, which somewhat undermined their argument. – matt_black Jan 07 '22 at 09:29
  • Seems odd to rely on success rates of operations when determining whether surgeons are finding ways to NOT operate. But isn't that exactly what all the citations here are doing? – Indigenuity Jan 07 '22 at 19:36
  • 1
    @Indigenuity Nope. Studies of the mix of patients operated on over time can shed light on whether there has been a change in risk aversion by the surgeons. That is part of some studies. – matt_black Jan 08 '22 at 01:10
  • 1
    @Indigenuity One of the quotes notes that more elderly and high risk patients are being treated. A statement which could be made clearer by noting if it accounts for population changes (are you treating more because there are more in the first place, etc.), but implicitly it means to suggest that you would expect there to be fewer patients in these categories if you had encouraged surgeons to avoid them. – zibadawa timmy Oct 01 '22 at 07:13