90

There is this new study that came out, alleging, that COVID-19 vaccines cause two deaths for every three deaths they prevent, which sounds pretty bad. I was wondering how valid this analysis is and what other view points exist on the topic.

Quote:

Abstract: Background: COVID-19 vaccines have had expedited reviews without sufficient safety data. We wanted to compare risks and benefits. Method: We calculated the number needed to vaccinate (NNTV) from a large Israeli field study to prevent one death. We accessed the Adverse Drug Reactions (ADR) database of the European Medicines Agency and of the Dutch National Register (lareb.nl) to extract the number of cases reporting severe side effects and the number of cases with fatal side effects. Result: The NNTV is between 200–700 to prevent one case of COVID-19 for the mRNA vaccine marketed by Pfizer, while the NNTV to prevent one death is between 9000 and 50,000 (95% confidence interval), with 16,000 as a point estimate. The number of cases experiencing adverse reactions has been reported to be 700 per 100,000 vaccinations. Currently, we see 16 serious side effects per 100,000 vaccinations, and the number of fatal side effects is at 4.11/100,000 vaccinations. For three deaths prevented by vaccination we have to accept two inflicted by vaccination. Conclusions: This lack of clear benefit should cause governments to rethink their vaccination policy.

Source:https://www.mdpi.com/2076-393X/9/7/693/htm

Now obviously this all hinges on the deaths reported in the adverse reaction data base to be causal, but even if they aren't, doesn't this put these vaccines in comparison to other vaccines in a bad light, or is this completely normal for most vaccines?

Peter Mortensen
  • 190
  • 7
user1721135
  • 2,706
  • 4
  • 17
  • 24
  • Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/127182/discussion-on-question-by-user1721135-do-covid-19-vaccines-cause-two-deaths-for). Please do NOT start discussion up again here - take it to chat. – Oddthinking Jul 04 '21 at 21:08
  • 4
    Currently in too much pain and not enough energy to make a full answer, but the paper making this claim has been retracted: https://retractionwatch.com/2021/07/02/journal-retracts-paper-claiming-two-deaths-from-covid-19-vaccination-for-every-three-prevented-cases/ – Shadur Jul 05 '21 at 14:37
  • 3
    (Secifically, the journal found some serious flaws in its methodology, asked the authors to clarify and found the subsequent explanation lacking) – Shadur Jul 05 '21 at 14:50
  • 2
    Somewhat disturbingly *Science* recounts (in article published yesterday) how that paper sailed through peer review... https://science.sciencemag.org/content/373/6551/147 – Fizz Jul 10 '21 at 08:31

4 Answers4

192

A series of comments on the article's PubPeer appears to entirely invalidate the paper's conclusion; the paper commits several classic errors which have been previously discussed on skeptics.se.

First is misuse of adverse event databases, exactly as discussed here. These sorts of adverse event databases do not establish causality; it is simply a list of all the times people have reported to have been affected shortly after getting a vaccine. The database used by the authors specifically states that it cannot be used this way.

Second, a risk-reward comparison for vaccination invariably suffers from its own success.

As an absurd example, before SARS-CoV-2 had been transmitted to the U.S., the US had a 0% mortality rate from SARS-CoV-2. If the entire U.S. was then vaccinated at a 0.01% rate of serious adverse events, some 32,800 people would die and practically none from the virus, for a risk-reward ratio of infinity. This ignores the 619,000 people who would otherwise have died.

(Note that it is possible to produce a risk-reward for a certain snapshot in time. Here's one example.)

0xDBFB7
  • 1,934
  • 1
  • 12
  • 20
  • 1
    This is a lousy answer, but it's what I've got for now. I recall seeing a much better example of a correctly produced risk-reward analysis somewhere regarding blood clots and age groups, but I cannot find it offhand. Other possible improvements might include the types of statistical mining that are done to establish causality, maybe an example of the number of deaths expected after a certain time period based on chance (Poisson process?), a better rebuke of the 95% underreporting figure. – 0xDBFB7 Jun 27 '21 at 23:03
  • Just to note in connection with "The database used by the authors specifically states that it cannot be used this way." The authors didn't use the CDC VAERS database, though the database does say reports are not necessarily caused by a vaccine. – Barry Harrison Jun 28 '21 at 04:15
  • 3
    We have the same uncertainty with COVID deaths (at least in Germany), the statistics state they died "from or with" COVID. Some people just happen to test positive, even when they died from cancer or road accident. – user24582 Jun 28 '21 at 07:25
  • 2
    @user24582 That's correct but the base rates are vastly different, making these measures not at all comparable. This may be surprising but it's really a result of basic statistics, and well known in epidemiology. – Konrad Rudolph Jun 28 '21 at 13:15
  • 17
    @CJR minor correction: Both the mRNA vaccines and the adenoviral vector vaccines have been linked to a VERY rare but also very serious form of blood clotting that has in some rare cases caused the death of a vaccinated person. However, we're talking about a VERY low death rate here: in the entirety of the USA, after 300M doses, there have been 3 deaths from J&J due to these clots, and 11 cases of thrombocytopenia (including deaths). For comparison: in the entirety of 2020, the same amount of people have been killed by shark attacks in the USA. – Nzall Jun 28 '21 at 13:23
  • 4
    @Nzall That's an excellent point; [the viral vector vaccines have been linked to deaths](https://www.nejm.org/doi/full/10.1056/NEJMoa2104882). [No equivalent events have occurred for the mRNA vaccines](https://www.nature.com/articles/s41591-021-01408-4), although there is likely a very low risk of myocarditis/pericarditis in some people that hasn't resulted in any deaths as of yet (to the best of my knowledge). – CJR Jun 28 '21 at 13:31
  • 1
    The 2nd half seems to fall under "back-of-the-envelope", which isn't allowed. But the first half is perfect -- the paper is based on raw VEARS-like data, which by now we know is only used by liars; and the OP didn't do their research (since the paper's comments disprove it just fine w/o our help). – Owen Reynolds Jun 28 '21 at 14:14
  • 1
    Honestly, given the amount of publicity the paper and this answer is getting, I really wish it was better. It's correct, but the rhetoric seems... *unconvincing* and hostile for its target audience. The paper is thoroughly bunk from tip to stern, and deserves a point-by-point rebuke. I really like Barry's answer. – 0xDBFB7 Jun 29 '21 at 18:33
  • 1
    Beyond anything else, the limited few-week timespan considered really deserves a mention, since the vaccines will work for years at least. – 0xDBFB7 Jun 29 '21 at 18:35
  • Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/127271/discussion-on-answer-by-0xdbfb7-do-covid-19-vaccines-cause-two-deaths-for-every). –  Jul 07 '21 at 04:48
135

Update (7/9/21): The paper was retracted on July 2, 2021. The retraction cited the same concern that not all deaths reported to national government systems are necessarily caused by the vaccine, while the paper incorrectly assumed that all such reported deaths were caused by the vaccine.

Do COVID-19 vaccines cause 2 deaths for every 3 deaths they prevent?

No. The paper is flawed and is contradicted by other reports.

Is the paper scientifically rigorous?

The paper quoted in the question is flawed. The paper searched the European equivalent of VAERS, a vaccine adverse event reporting database, compared the numbers among various countries, and then used the Dutch numbers in their analysis because the Netherlands had the most reports per vaccination. (Because of this, the authors' analysis cannot be generalized to countries beyond the Netherlands. i.e. death rates from COVID-19 may be, and in some instances are, much higher in other countries.) Additionally, the paper only tracked the number of deaths prevented for "three to four weeks" after vaccination. It's likely the vaccine will avert even more deaths from COVID-19 beyond that "three to four week" period. Therefore, the analysis in this paper is flawed. This is in addition to the fact, also noted in the other answer, that reports to these systems do not necessarily represent an event caused by the vaccine.

VAERS accepts and analyzes reports of possible health problems—also called “adverse events”—after vaccination. As an early warning system, VAERS cannot prove that a vaccine caused a problem. Specifically, a report to VAERS does not mean that a vaccine caused an adverse event. (emphasis from the original CDC source)

What do other studies say?

Britain

Luckily for us, this topic has interested the UK government. The UK government has studied the exact question of how many deaths COVID-19 vaccines have prevented. On March 25, 2021, Public Health England released a report on the benefits of COVID-19 vaccination.

Public Health England (PHE) analysis indicates that the COVID-19 vaccination programme prevented 6,100 deaths in those aged 70 and older in England up to the end of February. (Press release on Gov.UK)

As of March 14, 2021, the UK government reported 594 events with a fatal outcome after receiving a COVID-19 vaccine. (Again, reports do not indicate an event was caused by a vaccine.)

The MHRA has received 259 UK reports of suspected ADRs to the Pfizer/BioNTech vaccine in which the patient died shortly after vaccination, 326 reports for the Oxford University/AstraZeneca vaccine and 9 where the brand of vaccine was unspecified. The majority of these reports were in elderly people or people with underlying illness. Usage of the AstraZeneca has increased rapidly and as such, so has reporting of fatal events with a temporal association with vaccination however, this does not indicate a link between vaccination and the fatalities reported. Review of individual reports and patterns of reporting does not suggest the vaccine played a role in the death.

Needless to say, 6,100 deaths averted outnumbers 600 deaths reported to the UK-equivalent of VAERS by slightly over 10 times, again refuting the paper's conclusion.

Those figures have held up even to today. More recently, on May 14, Public Health England concluded the COVID-19 vaccines together prevented 14,000 deaths.

PHE estimates that 14,000 deaths have now been prevented in people aged 60 years or older in England up to 30 May 2021 (11,800 deaths in individuals aged 80 years and older, 1,800 in individuals aged 70 to 79 and 400 in individuals aged 60 to 69 years).

Estimates also indicate that the vaccination programme has prevented around 42,000 hospitalisations in those aged 65 years and over in England (approximately 5,400 admissions in those aged 65 to 74, 16,300 in those aged 75 to 84 and 20,300 in those aged 85 and over).

As of June 16, the most recent date with data available, 1,356 deaths after vaccination have been reported to the UK government. Again, deaths averted exceed deaths reported to the government by over 10 times. And the numbers above only include deaths prevented in older people.

Other studies:

US

The CDC has not yet released numbers on how many lives COVID-19 vaccines have saved. A May 6 study from The Journal of Infectious Diseases predicted that each 1% increase in coverage from 40% to 50% prevents 6,660 deaths. As of May 22, the US adult vaccine coverage is 57% (CDC source). This also questions the paper in the question as exactly 4,001 deaths have so far been reported to VAERS (see VAERS database). (Again, reports do not indicate an event was caused by a vaccine.)

Israel

Israel is a well-known case study in vaccine efficacy due to their vaccine rollout so far. Though I have not found studies of how many deaths were prevented, the number of COVID-19 cases (and, presumably, deaths) has steadily dropped with increased vaccinations (e.g. paper published by the Israel Ministry of Health in The Lancet).

Two doses of [the Pfizer vaccine] are highly effective across all age groups (≥16 years, including older adults aged ≥85 years) in preventing symptomatic and asymptomatic SARS-CoV-2 infections and COVID-19-related hospitalisations, severe disease, and death, including those caused by the B.1.1.7 SARS-CoV-2 variant. There were marked and sustained declines in SARS-CoV-2 incidence corresponding to increasing vaccine coverage. These findings suggest that COVID-19 vaccination can help to control the pandemic.

Barry Harrison
  • 14,093
  • 4
  • 68
  • 101
  • 22
    *"In any case, 260,000 deaths averted >> 4,001 deaths reported to VAERS"* - I think you could also add that we have no idea how many of those 4,001 deaths were due to the vaccine as opposed to some other cause. (and in the similar argument later). Your point of course is that even if *all* the deaths reported to VAERS were due to the vaccine (which is highly unlikely), it *still* would be better to vaccinate than not. – abligh Jun 28 '21 at 06:06
  • 1
    @abligh Thanks for your comment and for reading through this answer. Your point is excellent. I do include earlier on, in bold, "Specifically, a report to VAERS does not mean that a vaccine caused an adverse event." That said, I will make this edit to make things clearer. Thanks again! – Barry Harrison Jun 28 '21 at 06:07
  • 3
    @abligh You mean the right thing but just to be clear: it's incorrect to say that “we have no idea”. On the contrary, based on other data and prior knowledge we have *a pretty good idea* that few if any of these deaths are caused by the vaccines (*probably* >0, but even that is actually not yet firmly established). – Konrad Rudolph Jun 28 '21 at 13:19
  • 1
    @KonradRudolph indeed; my language was sloppy. By *"we have no idea how many"* I meant *"the reference paper gives us no idea how many ..."*. You are entirely correct that there is other data / studies that do give us a pretty good idea. – abligh Jun 28 '21 at 13:49
  • @KonradRudolph and abligh Both of you are correct of course. I could go into how many VAERS-reported deaths are to be expected just from deaths naturally occurring but I think that'd be too-"original research" and frowned on. – Barry Harrison Jun 28 '21 at 14:41
  • 1
    @barry Well, UK has a weekly death rate of about 0.01%. In the first report, 25 million shots where given, with a significant bias towards the older (and hence more likely to die) population. Even ignoring that, 25 million people randomly picked in the UK would 2500 deaths in the next week. 594 is rather low. (I suppose a good percent of people who die spend the period prior in intensive care already, and maybe we aren't given them vaccines) – Yakk Jun 28 '21 at 19:41
  • 5
    600 deaths for 6000 saved, thats pretty bad actually. I don't think any other vaccine has had such a horrible track record. Id love to see a comparison to say the flu vaccine which is targeted to similar demographics and is said to have lots of side effects. – user1721135 Jun 28 '21 at 22:35
  • 8
    @user1721135 Quoting the report with those 594 adverse reactions: "The majority of these reports were in elderly people or people with underlying illness. Usage of the AstraZeneca has increased rapidly and as such, so has reporting of fatal events with a temporal association with vaccination however, this does not indicate a link between vaccination and the fatalities reported. Review of individual reports and patterns of reporting does not suggest the vaccine played a role in the death." – MT0 Jun 28 '21 at 23:21
  • 2
    "Additionally, the paper only tracked the number of deaths prevented for "three to four weeks" after vaccination. It's likely the vaccine will avert even more deaths from COVID-19 beyond that "three to four week" period." I'm not clear on why the *ratio* would be different on a different time scale. Are they restricting the time frame for COVID deaths preventions, but not for deaths allegedly caused by the vaccine? – Acccumulation Jun 29 '21 at 02:13
  • @Yakk I agree with your points. I'm guessing that some obvious cases where death was from another cause (e.g. cancer) weren't reported. – Barry Harrison Jun 29 '21 at 03:46
  • @user1721135 MT0's comment is a good point. I should've quoted more of the report. Thanks MT0! – Barry Harrison Jun 29 '21 at 03:47
  • 1
    @Acccumulation Vaccine side effects tend to occur in the first 2-3 weeks after a vaccination. Whereas the benefits are much more lasting (typically years for a vaccine). This is why the ratio would be different on a different time scale. – Barry Harrison Jun 29 '21 at 03:48
  • @Acccumulation So to answer your question, yes that is essentially restricting the time frame for which COVID-19 deaths would be prevented, while not restricting the time frame for any deaths the vaccine may cause. Good question, thanks for asking. I will edit the answer to make this clearer. – Barry Harrison Jun 29 '21 at 03:51
  • 5
    @user1721135, "600 deaths for 6000 saved" is only something it's reasonable to say if those deaths were _caused by_ the vaccine, but the reporting explicitly does not track causality (and with the first people to be vaccinated being those most at risk, whether from old age or other health problems that had potential to be fatal in combination with a case of COVID, that rate is not something to discard). – Charles Duffy Jun 29 '21 at 15:40
  • 3
    It should also perhaps be noted that one of the reasons the Dutch COVID death rate was a lot lower than elsewhere was because we started taking emergency measures almost immediately specifically aimed toward making sure that our hospitalization rate would not exceed our hospitals' capability. So everyone who got ill enough to require hospitalization got hospitalized, which significantly upped the survival rate. – Shadur Jul 05 '21 at 14:54
  • 1
    @Shadur Great point, thanks for your comment! I did not know about that. – Barry Harrison Jul 05 '21 at 22:08
  • 3
    You don't need permission from anyone to improve your answer, that is, feel free to add that the paper has been retracted. –  Jul 07 '21 at 05:02
  • 1
    @fredsbend Ok, thanks! – Barry Harrison Jul 09 '21 at 15:48
6

For three deaths prevented by vaccination we have to accept two inflicted by vaccination.

This statement seems to imply that vaccines have a 2/3 mortality rate of Covid cases. If that is what it is asserting, then that can be easily refuted with a simple rough order of magnitude estimate using stats from the US. (all numbers are rough).

In order to gauge a ROM of "preventable deaths" we can take the 3 month period before the vaccines, Oct-Dec 2020, there were roughly 14M Covid cases resulting in 175K deaths. Excess mortality rates seem to confirm the 175K deaths from Covid.

In a 3 month period, March-May of 2021, about 100M people were fully vaccinated. That is 7 times (100M/14M) the number of Covid cases above.

With a 2/3 mortality rate, that would predict 4.7 times the number of deaths from vaccines in March-May of 2021 as compared Covid deaths Oct-Dec 2020.

4.7 x 175K = 833K deaths from vaccines.

I think if 833K had died from vaccines in US the past 3 months (over double the expected death rate), I think we would know about it. At least for March 2021, there has been no excess deaths over usual.

Anthony Corriveau
  • 69
  • 2
  • 7
    [Welcome to Skeptics!](http://meta.skeptics.stackexchange.com/questions/1505/welcome-to-new-users) It is so tempting to think "Oh, I have a figure from this source, and a figure from that source, I can just divide them or multiply them to get the figure I need", but that calculation itself needs a reference to justify it. In this case, you seem to be jumping between CFR and IFR, and misnaming it "mortality". Find someone who has been peer-reviewed or is at least a recognised expert who has made the calculations rather than asking us to trust you. – Oddthinking Jun 30 '21 at 13:16
  • 1
    I don't think I need to be an expert to do simple math. The title of the question is "Do covid vaccines cause 2 deaths for every 3 deaths they prevent?". I was doing some simple back of the envelope calculations based on that premise. Specifically what numbers do you have a problem with? If you have better sources please provide. If my logic is flawed, please elaborate. – Anthony Corriveau Jun 30 '21 at 13:28
  • I agree that calculating mortality is difficult, because it is impossible to know how many people actually contracted the Covid. But your cited number of 8.2/1000000 above cannot be applicable to the US. Even if the entire 331M population of the US contracted Covid (which is clearly not the case) that would be only 2700 deaths, instead of the documented 600,000. – Anthony Corriveau Jun 30 '21 at 13:44
  • 2
    You don't need to be an expert to do simple math, but you need to be an expert to understand why your math does not make much sense. You are for example using the US death rates compared to vaccine death rates in the article. Also, that very article predicts around 4110 vaccine deaths for 100 million vaccinated, not 883000 wherever that came from. This is simply just a random person on the internet trying to juggle numbers and won't convince anyone. – pipe Jun 30 '21 at 13:47
  • 1
    OK then lets work backwards, 4110 vaccine deaths per 100 million. We had 100 million vaccines administered in the US. With a 2/3 ratio, that implies that the Covid death rate is 6165 per 100M. If EVERYONE in the US has had Covid that is 20,000 deaths. Does that seem correct to you? – Anthony Corriveau Jun 30 '21 at 13:52
  • 1
    *"I think we would know about it."* I think without PCR we wouldn't know about the pandemic at all... Several hundreds of thousands die every day, and I think even doubling that wouldn't be noticed until months down the line. My point is not to argue this, just that "I think people would notice" is based on feelings not facts. –  Jun 30 '21 at 14:11
  • The claim is that vaccines cause 2 deaths for every 3 deaths that they prevent. I'm unclear on how you are calculating the number of deaths they prevented *during the March-May period* (I don't see the relevance of the Oct-Dec period - it might be mortality differed due to different treatment available etc.). Moreover I don't see why "deaths caused by vaccine" would need to be excess deaths - playing Devil's Advocate, perhaps a vaccine might "cause the death" of old and frail people who might have died a month later anyway. FAOD I don't believe the claim, but your answer doesn't disprove it. – abligh Jun 30 '21 at 14:17
  • 3
    This answer isn't amazing and uses back-of-the-envelope math that I wouldn't use (in the US, we measured how many people have died from the vaccine, and we measured how many people have died of COVID - just compare two numbers you've measured, not two numbers you speculate about). That said it looks brilliant compared to some of these comments (the piles of bodies would be unnoticeable without PCR???) so I guess good job. – CJR Jun 30 '21 at 14:42
  • there is no way to know deaths that were prevented by the vaccine. but you can take get a rough estimate of the number of Covid deaths WITHOUT the vaccine, and the nearest period was Oct-Dec 2020. And excess deaths is really the only way to measure a significant effect. If everyone who died from the vaccine would have died shortly after anyway, then it is not significant, is it? – Anthony Corriveau Jun 30 '21 at 14:48
  • 1
    And yet the case control study exists to do the very thing you think cannot be done. https://www.nejm.org/doi/full/10.1056/NEJMoa2101765 – CJR Jun 30 '21 at 15:03
  • @fredsbend "I think people would notice" isn't based on reality. As statisticians concluded (for quite a few places): ["One can clearly see that 2020 *was not* a conspicuous year in terms of excess mortality."](https://www.covid19.statistik.uni-muenchen.de/pdfs/codag_bericht_8.pdf) This is not only dependent on 'test,test,test' but "please count following broken-by-design definitions, count *everything* in this most irresponsible way and *report daily* how baaaad it is with this tunnel focus." – LangLаngС Jul 06 '21 at 09:20
  • If we "back of the envelope" this, then issue I have with the numbers, since you asked for specifics - is not everyone will be exposed to and catch covid, and of those that do, then a fraction of those would be serious enough to be a reported case, **THEN**, of those, just under 2% would die, so the "prevented death" number would be 2% of a fraction of a fraction of the number of vaccinations, not the full number of vaccinated people. – PoloHoleSet Jul 08 '21 at 13:39
  • 3
    This is a positive example of critical thinking, a relatively inexpensive and rapid check for _plausibility_ of a claim, without spending the time and resources required for an in-depth investigation and analysis. It does not _replace_ in-depth work, however it can be _very_ useful when attempting to deal with on-the-fly discussions or many issues in a limited time. – Technophile Sep 30 '21 at 18:09
1

The rate of COVID mentioned above is inconsistent with other sources.

For example the article below I found from a quick search and it gives a mortality rate of 8.2 per million. This is much lower than the one mentioned above. Without doing a whole review paper of the published statistics, other sources are often lower than this value.

pubmed.ncbi.nlm.nih.gov/34055843 Front Med (Lausanne) . 2021 May 14;8:670370. doi: 10.3389/fmed.2021.670370. eCollection 2021. Mortality Rate and Characteristics of Deaths Following COVID-19 Vaccination Gang Lv 1 , Jing Yuan 2 , Xiaomo Xiong 3 , Minghui Li 4

Oddthinking
  • 140,378
  • 46
  • 548
  • 638
Tman
  • 79
  • 1
  • 3
    [Welcome to Skeptics!](http://meta.skeptics.stackexchange.com/questions/1505/welcome-to-new-users) Please read this link for more info about requiring references. – Oddthinking Jun 30 '21 at 10:31