28

I came across a study that makes the following claim (emphasis mine):

Two doses (300 μg/kg/dose in a gap of 72 hours) of ivermectin chemoprophylaxis reduced COVID-19 infection by 83% among HCWs for one month. Ivermectin is a safe and effective strategy to prevent COVID-19, in the containment of pandemic alongside vaccine. Further research is required to guide the frequency of chemoprevention, acceptability, and cost-effectiveness in the community setting.

Is this a robust study and are the conclusions supported by the data?

F1Krazy
  • 3,520
  • 5
  • 20
  • 29
John
  • 397
  • 1
  • 3
  • 4
  • 24
    This seems to have no blinding, such that the subjects actually choose whether to receive none, 1 or 2 doses. I think this qualifies it as not robust. – User65535 Feb 01 '22 at 12:55
  • 20
    Cureus seems to be well-regarded by some as a publication where peer review and publication are quick and trouble free for publishers. However, the peer review process has been compared to "crowdsourcing," with a window of a few days where readers comment on it (anyone can join in). That's not particularly robust. It has an impact rating of 1.15, on a scale where 10 is excellent and 3 is considered adequate. – Cristobol Polychronopolis Feb 01 '22 at 15:38
  • 5
    @User65535: I'm going to tell you a story about blinding. It became necessary to test whether or not the measles vaccine caused autism. This was done without blinding on purpose. The test was done by gathering the entire data on the population of Wales for who was vaccinated and who wasn't and who got autism and found and publish a small negative correlation: if anything, the measles vaccine prevents autism. Why not blind? Easy. You chose to participate in the control group by not getting the measles vaccine. They didn't sample. They used the whole population. – Joshua Feb 02 '22 at 01:09
  • 15
    @Joshua That tells you that the the measles vaccine and autism have a correlation, it tells you nothing about the direction of causality. It could be that the measles vaccine prevents autism, or that autistic people are more likely to avoid the vaccine, or that there is a genetic component and parents of children who become autistic are less likely to get their children vaccinated. – User65535 Feb 02 '22 at 09:11
  • 4
    We can only judge whether ivermectin has a notable effect by looking at **all** the studies and carefully considering the detailed methods and results. Whether *this* study says ivermectin works is almost irrelevant. – matt_black Feb 02 '22 at 12:59
  • 11
    There is also the theory floating around that all the studies that showed positive effects are in places where parasites are somewhat common, so that effect could be attributed to the Ivermectin clearing out parasites that the patients unknowingly had, which would reduce the stress on their immune system and lead to a better outcome. – Simon Richter Feb 02 '22 at 16:26

3 Answers3

55

Summary: The conclusions are not generally applicable because the study did not isolate the role of ivermectin in the outcomes, and the process was not a randomized controlled trial.

First, this August 2021 report of a September-to-November 2020 study the sort of study that suggests "this may be promising, more research is needed." As the authors note, "Further research is required to guide the frequency of chemoprevention, acceptability, and cost-effectiveness in the community setting." (Result so far have been dubious: see Meta-Analyses Do Not Establish Improved Mortality With Ivermectin Use in COVID-19 in the Jan.-Feb. 2022 issue of American Journal of Therapeutics)

This study of 3,532 doctors, nurses, students, administrators and other staff at All India Institute of Medical Sciences (AIIMS) Bhubaneswar has some shortcomings that undercut its conclusiveness, some as noted in the comments above:

  • Administration of the ivermectin was not double-blind and was not randomized. Drug studies typically compare the results of two randomly-assigned populations, one taking the drug, the other taking a placebo or other control, with neither the researchers nor the participants knowing which they were getting. In this case, not only did the participants know they were taking the drug, they were also the people who chose to take the drug. For example, the authors note that "Ivermectin prophylaxis uptake was better with increasing age and among males."
  • We don't know how the behavior of the subject group differed from the general population given that they were people who chose to take ivermectin, or how their behavior might have changed knowing they were taking ivermectin or by participating in the study. They may have taken more or fewer risks to avoid infection, or otherwise altered their activities, their diet, or their other medication. The authors suggest "we had a strong institutional policy in place related to COVID-19-appropriate behavior in the workplace, which may have avoided the possible bias," but we don't know if that policy was enforced; there have been many studies about doctors misestimating how often they wash their hands, for example. There were also reports of ivermectin side effects such as nausea and headache among 42 participants; while side effects are commonly over-reported, we don't know if those drug-takers were so sick they missed work and had reduced exposure, for example.
  • We don't have conclusive evidence about how many health care workers (HCWs) in the overall study were infected. The authors note: "The major limitation is that we only tested HCWs who either developed symptoms or who were direct or high-risk contacts of positive patients. This was done in keeping with the Government strategy for COVID-19 testing in India. However, this precludes us from including the HCWs who may have been asymptomatic or mildly symptomatic and chose not to get tested."
  • Some unknown number of the hospital employees had already contracted covid in September 2020, which is what prompted the study. "We noticed an increasing number of HCWs getting infected with SARS-CoV-2 infection in early September 2020 at our hospital, which was negatively impacting the healthcare services we had to provide." We don't know how many of those employees were back at the hospital or how their participation in the study, other behavior, or chance of getting reinfected might have been affected by their personal experiences.
jeffronicus
  • 5,412
  • 3
  • 28
  • 27
  • 53
    "Later higher-quality studies and meta-analyses indeed suggested little to no benefit from taking ivermectin, so I won't rehash that information." I think it would be worth rehashing this because a lot of people believe it anyway, for other reasons. – Laurel Feb 01 '22 at 17:40
  • 5
    @Laurel Adding a link to a fairly critical new overview of the current set of meta-analyses of ivermectin. – jeffronicus Feb 01 '22 at 18:06
  • 2
    Thanks, @jeffronicus Appreciate you taking the time to do this. – John Feb 01 '22 at 20:33
  • 14
    Also worth emphasizing, the employees who took ivermectin did so as part of the hospital's response to rising covid infections; it seems likely that the people willing to take an experimental treatment to avoid infection (two-thirds of the staff) would have been a part of the employee population more inclined to take extra action to reduce their risks. Presumably handing out ivermectin wasn't the only thing the hospital was doing that was different; it was just the only thing the hospital was studying. – jeffronicus Feb 01 '22 at 23:11
  • 1
    @Laurel: The meta-analysis is completely broken, but the answer still is No! There is no possibility that ivermectin prevents infection. This is the astounding world we live in: over half of peer reviewed research papers are wrong. – Joshua Feb 02 '22 at 01:01
  • 7
    "this may be promising, more research is needed." This phrase is found in almost every publication. It does not necessarily mean what laymen might understand with it, but more like 'we keep working on this'. – pinegulf Feb 02 '22 at 07:07
  • Minor quibble - "the other taking a placebo" - this is not necessarily accurate; it could be they compare those taking the drug to those using some other drug - often it is compared to the "standard of care", for example in cancer drug trials they're not comparing to nothing/placebo, but they're comparing to people using the regular chemotherapy/etc. that is normally used. – Joe Feb 02 '22 at 17:57
  • It's also notable that 3,532 is a fairly small sample size. Normally a study will have tens of thousands of participants to make sure the results are as accurate as possible. Also, with the wide variety of exposure the employees saw, it could skew the results with an admin not being at risk as much as a surgeon or nurse. Not to mention that many could have had immunity from already having had covid previously, which prompted this study. It's also possible that participants under reported symptoms in order to help "prove" it worked, since it wasn't a blinded study. – computercarguy Feb 03 '22 at 19:33
  • This study doesn't even mention how many people were vaccinated or compare the results of vaccinated vs unvaccinated rates of infection. It only mentions vaccines in the conclusion, so it stating that Ivermectin works in conjunction with the vaccine is a pretty far stretch. India is still only 52% vaccinated. While vaccination rates might be higher specifically for HCWs, it's still not mandated for anyone, so with the study not mentioning vaccination rates or even using the word vaccine until the conclusion means there's significant data missing. – computercarguy Feb 03 '22 at 19:41
  • @computercarguy The study was conducted a few months before vaccines became available in India in January 2021. – jeffronicus Feb 03 '22 at 21:50
  • I appreciate the level of detail in this post, but indeed it is long. Can you give a 1-line summary at the top? – user1271772 Feb 03 '22 at 22:00
  • How does point 4 affect the study, when both employees who took and didnt take the drug could have been previously infected? – Pablo Feb 03 '22 at 22:22
  • @user1271772 Added. – jeffronicus Feb 03 '22 at 22:42
  • @Pablo That's not clear. The chart of cases among HCWs they includes shows about 250 cases among employees in the two weeks before ivermectin distribution began, and another 150 or so in the two weeks afterward. Whether those employees were quarantined, treated at the hospital or returned to work and/or received ivermectin isn't clear. – jeffronicus Feb 03 '22 at 22:53
  • @jeffronicus nice! So basically "the study did not isolate the role of ivermectin in the outcomes" which means that the 83% risk reduction could have been due to things other than ivermectin? – user1271772 Feb 03 '22 at 23:05
  • @user1271772 I believe so. The authors say the hospital did not use a randomized controlled trial -- which would have isolated those effects by giving only part of the subject group the real medication -- due to ethical concerns, presumably over potentially withholding helpful medication. But if you are sufficiently motivated -- by fear, by hospital guidance, whatever -- to take the medication that your institution's own doctors thinks is helpful, what else will you be willing to do differently? Was this the only policy change or practice the hospital implemented at the time to combat covid? – jeffronicus Feb 03 '22 at 23:36
  • @jeffronicus Good point, I wouldn't be surprised if a hospital implemented more than one policy change to help combat COVID. Probably people were in general more careful (wearing masks, keeping good airflow in the rooms, and minimizing close-contact?) in the hospital, which would reduce COVID transmission, and these are factors that are hard to isolate from any study on the effectiveness of something at reducing COVID risk? I haven't read the article yet, nor much more on the topic, as this week has been the busiest in a **long** time, hence why I asked you for the 1-line summary :) – user1271772 Feb 03 '22 at 23:50
  • @jeffronicus, considering when the vaccine was developed compared to this study, you'd think they would have at least mentioned that it hadn't been used in India yet. IMO that would clarify some of the unknowns in the study and makes me think this is another reason why the study wasn't well done, or at least well documented, due to all the missing data points. – computercarguy Feb 04 '22 at 00:00
21

This recent Cochrane review did consider the Behera study mentioned by the OP, but rejected it on the grounds that it wasn't an RCT (randomised controlled trial), along with eight other studies for the same reason.

Out of interest, there are other reviews of ivermectin for covid. This one is from last year, and this one is very recent. Both found significant issues in some of the studies they considered.

The first two reviews I linked to were mentioned by @jeffronicus in their excellent answer. However, the third wasn't, and maybe the most relevant finding in that review was how strongly the effect of ivermectin decreased as more problematic studies were excluded from the meta-analysis.

See Figure 1 here, which I've also posted below. The effect of ivermectin is strong when all studies are included, but the p-value is very low (0.01). However, when studies with any degree of concern are excluded, the effect of ivermectin is marginal (a 4% increase in terms of improvement in survival, i.e. 1.04, 95% CI, 0.56–1.66; p = 0.90).

Edit: there is also the PRINCIPLE trial, run by the University of Oxford. They are currently investigating ivermectin for covid treatment. For context, they're also testing favipiravir and the "usual standard of NHS care". They've already looked at inhaled budesonide, azithromycin, doxycycline and colchicine, of which only budesonide showed any benefit ("shortens recovery time by a median of three days in patients with COVID-19 who are at higher risk of more severe illness and are treated in the community").

Mentioning this as ivermectin results will presumably be released in the near future.

enter image description here

paddyr
  • 461
  • 2
  • 7
  • 3
    If you are going to refer to the higher quality reviews of ivermectin, it would be worth quoting their explicit conclusions on what the reliable evidence says. This would be a useful answer to the question which is far broader than the reliability of a single paper. – matt_black Feb 02 '22 at 12:21
  • 2
    @matt_black: Thanks, good point, I've made an edit to reflect your suggestion. – paddyr Feb 02 '22 at 12:47
  • 1
    Worth stating that the Cochrane review you link to considered ivermectin both for treatment and for prevention. On the latter it says "Ivermectin compared to no treatment for prevention of SARS‐CoV‐2 infection: We found one study. Mortality up to 28 days was the only outcome eligible for primary analysis. We are uncertain whether ivermectin reduces or increases mortality compared to no treatment" – Henry Feb 03 '22 at 11:42
  • 1
    This is a (common) misuse of the p-value. The p-value is the probability of observing *the data/observations* given your *hypothesis* is true. NOT the probability your *hypothesis is true* given the *data* (if you want this you need to convert using Bayes rule). – Cole Feb 03 '22 at 12:11
  • 2
    This tells us the low-bias studies do report anything that would be surprising if ivermectin had no effect, NOT the probability that ivermectin has no effect given the clinical data. (e.g. P(street is wet | raining) =/= P(raining | street is wet) (| = "given"), one is ~100% the other likely depends on how often your neighbor runs their sprinklers) This is somewhat a nitpick but I thought it was worth pointing out. – Cole Feb 03 '22 at 12:17
  • @Cole: Not nit-picking at all, thanks! I am not qualified to comment, please feel free to edit my answer. – paddyr Feb 03 '22 at 12:24
-10

I cannot fully explain why, but I am not sceptical of the massive amount of reproducible work that has gone into the collation of evidence for the efficacy of Ivermectin (and other treatments in linked pages) done by the analysts at the c19ivm.org live meta analysis. As I write this they have analysed 98 studies relating to Ivermectin. The good the bad and the ugly.

The 17 relevant studies included in the Prophylaxis results, with almost 20'000 patients, combined, offer an improvement of 85%. That seems to match the claim of the OP better than could be expected.

EDIT:
Response to popular criticism-
1: Wikipedia knows better, they choose to use a failed policy over the health of humanity. They have chosen to promote variable public policy instead of research results even though ephemeral data does not belong in an encyclopedia.
2: The lady and her dog have the following populist agendas that they promote without basis, Understand that using propaganda to support a narrative is not very scientific, much has transpired since the video was made to further weaken her case.
2A: A RCT is not the only way medicine is practised. The lady says "which is the only type of clinical trial that is able to determine if our treatment is effective" Yet the regulators know that is not true. Regulatory approval of pharmaceuticals without a randomised controlled study: analysis of EMA and FDA approvals 1999–2014. The contested site includes all trials but the results of the RCTs are available separately for those that are not able to see past the RCT myth. However he results are similar but statistical weight is achieved later, other trials show the same benefit 5 months earlier.
2B: The lady also says "need to remove any studies where the drug you are interested in is combined with another drug or therapy" which is also nonsensical, this would preclude many therapies, some that have been meta-analysed by her champion "The Lancet". Fixed-dose combination therapies with and without aspirin for primary prevention of cardiovascular disease: an individual participant data meta-analysis
2C: She states "need to remove studies where there is no placebo or standard of care arm". This is not an absolute and standard of care differs in different locations. Some researchers will accept some treatments as standard of care but not others while eliminating studies that use a treatment they say has no efficacy. Insisting on standard of care that does not allow other treatments prevents the development of multidrug remedies. This limitation is sidestepped by patent drugs that are combinations of single remedies. Many of the patent remedies were combination treatments and did not have isolated meta-analysees before approval.
2D: ROB fails to consider conflict of interest for some reason yet this is known to be a massive driver of bias in drug trials. Industry bias is real but not included in the ROB analysis. Industry sponsorship and research outcome, Why the Cochrane Risk of Bias Tool Should Include Funding Source as a Standard Item
2E: The Authors at IVMMeta are anonymous to limit cancel culture destroying their lives, if this was not a risk I expect they would be glad to unblind their identities. Note also that the editors at Wikipedia that black list the site are also anonymous.
2F: The Galan study she thought was important to add is included on the site but of limited value because there was no control arm due to ethical reasons. So no placebo that the lady was very concerned about earlier. It compared 3 working remedies that showed benefit compared to population statistics which are a proxy for Standard of care.
2G: The missing i2 value has been added to the forest plots for those that care.
2H: I do believe it would be impossible for the lady and her dog to use the same claims to dispute the evidence base as it now stands. Also if she applied her critical reasoning to other patent treatments we might find that they also show no evidence of efficacy, almost certainly at the time they were authorised for use. 2I: There are many results with exclusions of poor studies, RCTs only, Peer-reviewed only, etc., etc. Almost all show some or significant benefit.

For a sceptics stack there is a surprising amount of credulity for a random YouTube lady and her dog that she claims is her assistant. Why is it that the commentary and analysis at the offered site appear so much more credible than the casual criticism from the lady and her dog that look like something you would expect from a pharmaceutical industry advocate. Why do the results from the other sister sites show credible results using consistent methods for 51 treatments, some efficacious and some not. If I was a sceptic the Cost per life saved chart would trigger me big time even if there was an error or two included the differences are HUGE. It would be prudent to read the comments for each study and see if the conflicts of interest and failures are enough to change minds.

KalleMP
  • 89
  • 3
  • 7
    This seems to be just a rant about how much you dislike Wikipedia rather than an actual attempt to answer the question. – F1Krazy Jun 21 '23 at 19:17
  • I have removed a rant about Wikipedia. Not only was it irrelevant, but it was substantially inaccurate. Meantime, IVM Meta has been specifically called out as pseudo-science by a number of scientists: [2022 Meta-Analysis](https://bmcinfectdis.biomedcentral.com/articles/10.1186/s12879-022-07589-8), [David Gorski](https://www.vice.com/en/article/wx5z5y/why-is-the-intellectual-dark-web-suddenly-hyping-an-unproven-covid-treatment), [Ivermectin hype in Latin America](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8050401/) – Oddthinking Jun 22 '23 at 15:07
  • I just edited the post so the link text says c19ivm.org which is the domain it points to instead of imvmeta.com which it did not point to. – Joe W Jun 22 '23 at 16:47
  • 6
    [Back To Science](https://www.youtube.com/watch?v=v_1mmnRm7EU&ab_channel=BacktotheScience) produced a fairly accessible 15 minute video which politely rips this web-site's fake meta-analysis to pieces, showing where they make statistical errors and where they make fraudulent claims. – Oddthinking Jun 22 '23 at 17:39
  • 1
    For clarity "The lady and her dog" refers to [Dr Susan Oliver](https://scholar.google.com.au/citations?user=EwNWFOIAAAAJ&hl=en). It is a reference to the Back To Science YouTube video that I referenced above because it is easy to understand. I *also* put more substantial counter-evidence in the comments above, but they are harder to use flippant ad hominem attacks against. – Oddthinking Jul 03 '23 at 07:31
  • 2
    Several new claims have been added that are unreferenced. Please reference your claims. – Oddthinking Jul 03 '23 at 07:33
  • @Oddthinking I actually missed your links in the comment where you explained your edit about my rant. Will get to that later perhaps. How about you downgrade your lady and dog evidence, it is not much of a reference either. She brought the dog to the party, not me. You brought her and she says anyone can use the tools to perform analysis and this is what the running meta has done. – KalleMP Jul 03 '23 at 07:40
  • 3
    I don't know what "downgrade" means in this context. You can start giving Dr Oliver the respect of using her name, rather than othering her. I maintain it is an accessible reference to explain some of the issues at a popularist level. I can't see how any of your objections actually undermine what she says: You are defending non-RCTs in a meta-analysis?! You are confused about the role of standard of care. You seem to be arguing two wrongs make a right for untested medication. You are now claiming that two other treatments are equally good. My comments stand. – Oddthinking Jul 03 '23 at 07:48
  • 1
    "I cannot fully explain why" is the most insightful line in this rant. The "massive amount of reproducible work" on ivermectin is possibly the least. The problem is that much of the work is not reliably conducted and reproducible only in the sense that doing more badly controlled trials will often produce bad evidence. That's not science, just validation of prior opinion by using shonky protocols. – matt_black Jul 03 '23 at 10:12
  • 2
    Ivermectin has been shown to be ineffective as a treatment for COVID in multiple well-designed and executed RCTs, this isn't 2020 anymore. Meta-analysis of a bunch of terrible observational studies from bad scientists done during the pandemic is irrelevant. Frankly, even the medical grifters have moved on from this. – CJR Jul 03 '23 at 12:19