8

An October 2020 article in Germany's Finanztreff claims:

Almost two-thirds [of positive tests for COVID-19] are false positive - at least The Wadsworth Center, the laboratory of the US state of New York, analyzed the test numbers from last July.

However, an article in The Lancet from September 2020 reports:

RT-PCR assays in the UK have analytical sensitivity and specificity of greater than 95%

Do the PCR tests have a high rate of false positives?

David Hammen
  • 14,500
  • 9
  • 57
  • 51
user1721135
  • 2,706
  • 4
  • 17
  • 24
  • Related question: [Has SARS-COV-2 been purified in a lab](https://skeptics.stackexchange.com/questions/50205/has-sars-cov-2-been-purified-in-a-lab-if-not-does-this-imply-that-the-pcr-test) – Oddthinking Feb 26 '21 at 02:01
  • Related question: [Did a Portugese court find PCR is not fit for purpose?](https://skeptics.stackexchange.com/questions/50206/did-the-or-a-european-court-declare-that-covid-19-pcr-tests-are-not-fit-for) – Oddthinking Feb 26 '21 at 02:02

2 Answers2

10

The false-positive rate has been shown to be between 0.8% and 4.0% 1,2,3,4. This matches the 95% number you are stating that is likely from Sukova et al (2020). If this is a high rate is mostly subjective. To give a comparison, it is a far lower false-positive rate than the standard Flu test in the USA.

  1. False-positive COVID-19 results: hidden problems and costs Surkova, Elena et al. The Lancet Respiratory Medicine, Volume 8, Issue 12, 1167 - 1168 (2020).

  2. Watson, Jessica, Penny F. Whiting, and John E. Brush. "Interpreting a covid-19 test result." Bmj 369 (2020).

  3. Mayers, C., and K. Baker. "Impact of false-positives and false-negatives in the UK's COVID-19 RT-PCR testing programme." (2020).

  4. Cohen, Andrew N., and Bruce Kessel. "False positives in reverse transcription PCR testing for SARS-CoV-2." medRxiv (2020).

  • The FPR describes the rate of positivity among people who don't actually have the disease, but I think the more relevant statistic is the false discovery rate, which is the rate of positivity *among people who test negative*. Although people who test negative are unlikely to have the disease, since so many more people test negative than positive, the number of actual positives in each group is more similar. The FDR is the likelihood that a positive result is false, and is closer to 30% (depending on actual prevalence of the disease). – Nuclear Hoagie Mar 01 '21 at 15:45
  • 2
    The proportion of false positives depends on the prevalence, in addition to the performance characteristics of the test. This answer is a good example of how these statistics are often misunderstood. – De Novo Mar 01 '21 at 20:14
  • @DeNovo Post the answer! I actually agree with you about prevalence but that does not appear how the question is phrased. – If you do not know- just GIS Mar 02 '21 at 02:41
  • The first of your sources that I've checked (Surkova, Elena, et Al) does not support your claim, in fact your language is extremely misleading. The related quote in full is "The current rate of operational false-positive swab tests in the UK is unknown; preliminary estimates show it could be somewhere between 0·8% and 4·0%." If there were something wrong with the PCR tests, the assumptions that went into the estimate will not capture that issue, and the rate will be artificially low. – user2647513 Apr 04 '21 at 05:29
  • But if a country has avg. 1% positive per test, does that mean the case numbers for the country are completely irrelevant? This is a frequently cited argument and I can't figure out if it is correct. Basically, if nobody has covid, would the tests always produce abut 5% (or 0.8%) positives? Or does that mean, that among the 1% positve, 5% of that is wrong? @user2647513 – user1721135 May 08 '21 at 23:01
  • @user1721135 it would not be irrelevant, but my understanding is if nobody has the illness you would still expect the tests to come back positive about 0.8% percent of the time. Prevalence does matter though, see the last few sentences by Oddthinking to see why. – If you do not know- just GIS May 09 '21 at 12:59
  • @Ifyoudonotknow-justGIS what I mean is, if all tests are 0.8% positive not matter what, and current tests are 1% that means there are barely any cases. But I think thats wrong, because they also test kids around here and there are months on end with 0% positives, so 0.8% positive no matter what is probably wrong. – user1721135 May 09 '21 at 17:23
7

It appears that there is confusion between different measures here.

First, I am focussed on the PCR tests (i.e. trying to detect fragments of the virus) rather than the serology test (i.e. trying to detect the body's reaction to the virus.)

Now, how do we measure the accuracy?

Specificity

One measure is the "specificity".

Wikipedia's definition:

Specificity (True Negative rate) measures the proportion of negatives that are correctly identified (i.e. the proportion of those who do not have the condition (unaffected) who are correctly identified as not having the condition).

The specificity of the PCR test is going to depend on the lab and details of the test (e.g. the cycle-threshold, Ct), but it can generally be considered high.

RT-PCR assays in the UK have analytical sensitivity and specificity of greater than 95%

An unpeer-reviewed pre-print, Diagnosing SARS-CoV-2 infection: the danger of over-reliance on positive test results from September 2020 warns that the real-world figures may be lower than than 100% (that they claim others claim), and go on to express concern about how the authorities have interpreted these results. The figures from the studies they do cite include 97-99.7%, >93.7%, >99.6%, 99.3%.

So, even a paper concerned that the specificity is too low showed very high numbers.

So, to summarise: If a random uninfected person takes a PCR test to try to detect if there are parts of the virus in their body, the chance that they will told that the test wrongly said "Yes" is pretty low.


PPV

Another measure is the "Positive Predictive Value."

Paraphrasing Wikipedia:

The positive predictive values (PPV) is the proportion of positive results in statistics and diagnostic tests that are true positive results. The PPV describes the performance of a diagnostic test or other statistical measure. A high result can be interpreted as indicating the accuracy of such a statistic. The PPV is not intrinsic to the test (as true positive rate and true negative rate are); it depends also on the prevalence.

[This isn't a direct-quote. I removed an irrelevant definition pf NPV that was woven in.]

So this is the figure that shows how worried you should be if you receive a positive result.

This is an area that is hard for the public (and even doctors) to get their heads around. Gerd Gigerenzer's book "Reckoning with Risk" warns about widespread innumeracy in this area, and proposes techniques (e.g. frequency trees) to help resolve it.

If I can give an example using Gigerenzer's technique: I live in a state in Australia where we have been very fortunate recently to have a very low prevalence of COVID-19 and generally limited to travellers in quarantine and their immediate contacts. Let's say, for illustration, that the prevalence is around 3 per million. Let's assume the sensitivity was 100% and specificity was at the high end of the estimates given: 99.6%.

That means if 1 million people were tested, 3 would get a true positive test because they were infected, and 4000 would get a false positive test (0.4% of 1,000,000). Only 0.075% of positive tests would be true positives!

For as long as the virus does not escape its containment (again!) and we don't have another outbreak, as an Australian, who is not a traveller and not in close contact with any, both of these statements are true, and not in conflict with each other:

  • I can be confident that if I am uninfected and I get tested, I will not get a false positive - thank you, high specificity.
  • I can be confident that if I do get a positive, it is very likely a false positive - thank you, low prevalence leading to a low positive predictive value.

These figures would be very different if I was from an area where the prevalence was much higher. I don't offer these calculations as accurate and correct, but as an illustration of how even a low false-positive rate can overwhelm a test in an area of low-prevalence.


What is the claim?

The original claim in German sources the Wadsworth Center's report. Which definition was it using?

I was unable to find the original, but the clearest explanation I found of the contents was from a testimonial referencing a (now missing) newspaper report.

At Mandavilli’s request, New York state’s Wadsworth Center examined 872 positive PCR test results it had obtained in July, after amplification for 40 cycles. “With a cutoff of 35,” Mandavilli reported, “about 43 percent of those tests would no longer qualify as positive. About 63 percent would no longer be judged as positive if the cycles were limited to 30.”

It is clear that they are not referring to specificity, but to Positive Predictive Value.

In summary: There is no conflict between the two statements quoted in the question.

LangLаngС
  • 44,005
  • 14
  • 173
  • 172
Oddthinking
  • 140,378
  • 46
  • 548
  • 638
  • It appears the source is (made for) [this article](https://www.nytimes.com/2020/08/29/health/coronavirus-testing.html). // This appears to not answering the question as given in the title? – LangLаngС Mar 01 '21 at 15:21
  • This is a bit complicated, but the impact of cycle threshold on test results has a lot more to it than the difference between PPV/NPV and sensitivity/specificity. – De Novo Mar 01 '21 at 20:23
  • @LangLangC: I directly address the question in the title: No, the specificity for the PCR test is high. However, I also show that that isn't the (selected) claim in the article, which is related to PPV. – Oddthinking Mar 02 '21 at 16:27
  • @DeNovo: Sure, and the NYT article talks about using different results for different cycle thresholds as a proxy for viral load, but that's not the question here. – Oddthinking Mar 02 '21 at 16:31
  • "Do PCR COVID tests have high risk of false positives?" — "In summary: There is no conflict between the two statements quoted in the question." // I consider the bridge built for laymen to cross just not available at all on both sides of the river, yet the toll booth is working well in providing a living for its occupant. // (1st summary is *wrong*, imo. As not only is it plainly confusing to read but also axiomatic for 'the' PCR test. There are a few, none of them is globally standardised, targets, primers, cycles etc all up for grabs… Bayesian math may yield FPR up to 90%) – LangLаngС Mar 02 '21 at 17:41
  • @Oddthinking the Ct question is about what you're testing for, not about how well the test detects it. It's about whether you're looking for cases, or cases with active replication. – De Novo Mar 02 '21 at 20:45
  • @LangLangC: I would invite you to use English that is less flowery, because once again I am scratching my head trying to interpret your metaphors. What is axiomatic? What Bayesian math are you talking about? What is "plainly confusing"? Jumping from "no global standard" to high FPR is dubious. I explicitly say "The specificity of the PCR test [...] can generally be considered high." but I that is misleading, so I follow up with why we shouldn't rely on it. I am completely ignoring your political views, again, as off-topic. – Oddthinking Mar 03 '21 at 05:07
  • @DeNovo: I agree that looking at the results at different Ct levels can give information which is a proxy for transmissibility, and I think there may well be another valid Skeptics.SE question there, but I don't think this one is it. – Oddthinking Mar 03 '21 at 05:13
  • Usage of "the PCR (test)" needs to clarify that there are quite a few different designs out there, not just different implementations of one like lab/cycle threshold. *Meaning* of a 'result' is in all of them: "presence of 1–6 nucleotide targets detected (whether dead, 'alive', utter garbage FP". One isolated target; perhaps non-specific; low count; from long ago/successfully beaten by immune system, amplified into positivity ≠ '(currently) infected' (~active case). Colloquially: how well can results describe reality (ill now, soon)? Not well, just a puzzle piece. 4Qtitle: "Yes, but depends" – LangLаngС Mar 03 '21 at 11:06
  • In addition to my comments here & chat: https://doi.org/10.1101/2020.04.26.20080911 "The Patient's Fact Sheet for the US CDC's SARS-CoV-2 test states that there is only "a very small chance" that positive tests result could be wrong, but suggests that negative results could well be wrong.16 Public health officials have similarly suggested that positive test results are absolutely reliable but negative results are untrustworthy.17 In reality the opposite is true over a wide range of likely scenarios." Cf stats tool https://zenodo.org/record/4600597 – LangLаngС Mar 19 '21 at 13:59
  • In your example, the prevalence is taken as a given for the point of illustration, however, pragmatically, this question wants to know whether that prevalence based on PCR testing contains errors. From my understanding of your answer's definitions, it would seem to me that the PCR testing can *never* show 0 prevalence, ergo, there must be a minimum prevalence number based on testing that is meaningless, while higher prevalence figures would be reliable. What is that number? –  Sep 14 '21 at 21:50