Are current COVID-19 tests showing more false positives or false negatives?

Comment by InpharmD Researcher

The reported false-negative rate of RT-PCR COVID-19 tests is around 2-29%, but some data reports false-negatives as high as 63%. Rapid COVID-19 tests are more likely to show false-negative results (as high as 40% with some tests). There is a potential that antibody assays may show false-positives due to cross-reactivity with other coronaviruses, but the true incidence is unknown.

Background

A literature review and pooled analysis reviewed seven previously published studies providing data on RT-PCR performance by time since symptom onset or SARS-CoV-2 exposure using samples from the upper respiratory tract from 1330 patients. Over the 4 days of infection before the typical time of symptom onset (day 5), the probability of a false-negative result in an infected person decreases from 100% (95% CI, 100% to 100%) on day 1 to 67% (CI, 27% to 94%) on day 4. On the day of symptom onset, the median false-negative rate was 38% (CI, 18% to 65%). This review and pooled analysis showed that care must be taken when interpreting RT-PCR tests for SARS-CoV-2 infection. If clinical suspicion is high, infection should not be ruled out on the basis of RT-PCR alone, and the clinical and epidemiologic situation should be carefully considered. [1]

A review detailed the problems with false-negatives from RT-PCR tests. One study from Wuhan, China by Yang et al. (Feb 2020), consisting of 213 patients hospitalized for COVID-19, showed that 11% of sputum, 27% of nasal, and 40% of throat samples were deemed falsely negative. Another preprint systematic review (March 2020) of five other studies involving 957 patients had false negatives ranging from 2% to 29% (certainty of evidence was considered very low). Diagnostic testing can help in safely opening countries, but only if the tests are highly sensitive and validated under realistic conditions against a clinically meaningful reference standard. The FDA should ensure that manufacturers provide details of tests’ clinical sensitivity and specificity at the time of market authorization and assuring the test sensitivity in asymptomatic people is an urgent priority. Negative results, even on a highly sensitive test, cannot rule out infection if the pretest probability is high, so clinicians should not trust unexpected negative results. Thresholds for ruling out infection need to be developed for a variety of clinical situations. [2]

A systematic review of the accuracy of COVID-19 tests reported false-negative rates of between 2% and 29% based on negative RT-PCR tests which were positive on repeat testing. The lack of a gold-standard is a challenge for evaluating COVID-19 tests. Clinical adjudication may be the best available “gold standard,” based on repeat swabs, history, and contact with patients known to have COVID-19, chest radiographs, and computed tomography scans. Interpretation of a test result depends not only on the characteristics of the test itself such as sensitivity, specificity, positive likelihood ratios, and negative likelihood ratio, but also on the pre-test probability of disease based on their learned mental shortcut. A single negative test result may not be informative if the pre-test probability is high, especially when facing new and unfamiliar diseases such as COVID-19. Factors that increase pre-test probability include signs, symptoms, history of exposure, and local rates of COVID-19 infection. The likelihood of an alternative diagnosis decreases pre-test probability. [3], [4]

A case report and literature review compiled reports that display false-negative results of COVID-19 testing. Most studies compared RT-PCR to CT scans to verify the diagnosis. The incidence of false-negative results ranged from 17% to 63% for nasopharyngeal RT-PCR for SARS-CoV-2. [5]

A meta-analysis was performed to describe the accuracy of available tests to detect COVID-19 in Brazil. Sixteen tests were included and sensitivity (Se), specificity (Sp), true positive (TP), false positive (FP), true negative (TN), false negative (FN), positive predictive value (PPV), negative predictive value (NPV), and diagnostic odds ratio (DOR) were described for each test. The pooled diagnostic accuracy of tests in Brazil was satisfactory; however, the rate of false-negative results from tests which detect SARS-CoV-2 IgM antibodies ranged from 10 to 44%. [6]

An editorial warns about the possibility of false-positive results with SARS-COV-2 antibody tests. Cellex, the first antibody test approved by the U.S. Food and Drug Administration for the virus, has a reported sensitivity of 94% and specificity of 96%. When compared to a hypothetical SARS-COV-2 antibody test that is 90% sensitive and 99% specific, the false-positive incidences are much higher when the prevalence of SARS-COV-2 is low. Additionally, many of the other tests with provisional approval by the U.S. Food and Drug Administration have not been appropriately evaluated for accuracy. [7]

References:

[1] Kucirka LM, Lauer SA, Laeyendecker O, Boon D, Lessler J. Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure. Ann Intern Med. 2020;[E-pub].
[2] Woloshin S, Patel N, Kesselheim AS. False Negative Tests for SARS-CoV-2 Infection — Challenges and Implications. New England Journal of Medicine. 2020;383(6). doi:10.1056/nejmp2015897
[3] Watson J, Whiting PF, Brush JE. Interpreting a covid-19 test result. BMJ. 2020;369:m1808.
[4] West CP, Montori VM, Sampathkumar P. COVID-19 Testing: The Threat of False-Negative Results. Mayo Clin Proc. 2020;95(6):1127-1129.
[5] Kelly JC, Dombrowksi M, O’Neil-Callahan M, Kernberg AS, Frolova AI, Stout MJ. False-negative testing for severe acute respiratory syndrome coronavirus 2: consideration in obstetrical care. American Journal of Obstetrics & Gynecology MFM. 2020;100130. doi:10.1016/j.ajogmf.2020.100130
[6] Castro R, Luz PM, Wakimoto MD, Veloso VG, Grinsztejn B, Perazzo H. COVID-19: a meta-analysis of diagnostic test accuracy of commercial assays registered in Brazil. The Brazilian Journal of Infectious Diseases. 2020;24(2):180-187. doi:10.1016/j.bjid.2020.04.003
[7] Ebell MH, Barry HC. Beware of False-Positive Results with SARS-CoV-2 Antibody Tests. Am Fam Physician. 2020 Jul 1;102(1):5-6.

Literature Review

A search of the published medical literature revealed 5 studies investigating the researchable question:

Are current COVID-19 tests showing more false positives or false negatives?

Please see Tables 1-5 for your response.


 

Performance of Abbott ID Now COVID-19 Rapid Nucleic Acid Amplification Test Using Nasopharyngeal Swabs Transported in Viral Transport Media and Dry Nasal Swabs in a New York City Academic Institution

Design

Observational, in-vitro study

N=101

Objective

To evaluate the performance of the ID Now test by using the Cepheid-Xpert-Xpress SARS-CoV-2 (Xpert Xpress) on the GeneXpert Dx as the comparator reference method for detection of SARS-CoV-2

Methods

Samples were collected from 101 emergency department patients (aged 28 to 90 years), all suspected of COVID (onset of symptoms 1 day to 1 month) except 1 patient who was tested prior to surgery for ectopic pregnancy as per emergency department protocols.

Initial verification of the ID Now platform was accomplished by using two previously tested positive patients’ samples using the Xpert Xpress assay. All samples were tested on the ID Now and the results compared to those previously obtained using the Xpert Xpress.

The ID Now COVID-19 (Abbott Diagnostics Scarborough, Inc., Scarborough, ME) is a rapid test that qualitatively detects SARS-CoV-2 viral nucleic acids from nasal, nasopharyngeal, and throat swabs via nucleic amplification. The manufacturer claims the limit of detection is 125 genome equivalents/mL.

The Xpert Xpress SARS-CoV-2 test (Cepheid, Sunnyvale, CA) is a rapid, real-time RT-PCR that detects SARS-CoV-2 RNA in nasopharyngeal swab and/or nasal wash/aspirate specimens. The manufacturer claims the limit of detection for this assay is 250 copies/mL.

Results

ID Now

Xpert Xpress

Positive Negative Total

Positive

17 1 18

Negative

14 69 83

Total

31 70 101

The ID Now using dry nasal swabs agreed in 17 of 31 positive Xpert Xpress samples, with a positive percent agreement (PPA) of 54.8% (95% confidence interval [CI], 37.8 to 70.8%). The remaining 14 samples were not detected by ID Now. The ID Now results matched 69 of the 70 negative Xpert Xpress results, with 1 detected by ID Now but not by Xpert Xpress, with a negative percent agreement (NPA) of 98.6% (95% CI, 92.3 to 99.7%)

Ultimately, there was 85.1% agreement (95% CI, 76.9 to 90.8%) and a positive predictive value (PPV) and negative predictive value (NPV) of 94.4% (95% CI, 74.3 to 99.0%) and 83.1% (95% CI, 73.7 to 89.7%), respectively.

Study Author Conclusions

Regardless of method of collection and sample type, the Abbott ID Now COVID-19 test had negative results in a third of the samples that tested positive by Cepheid Xpert Xpress when using nasopharyngeal swabs in viral transport media and 45% when using dry nasal swabs. This revealed low positive percent agreement (PPA) of ID Now compared with Xpert Xpress, which raises concerns regarding its suitability as a diagnostic tool.

InpharmD Researcher Critique

The sample size is small and only was based in Manhattan. Positive and negative controls were not used.

 

References:

Basu A, Zinger T, Inglima K, et al. Performance of Abbott ID Now COVID-19 Rapid Nucleic Acid Amplification Test Using Nasopharyngeal Swabs Transported in Viral Transport Media and Dry Nasal Swabs in a New York City Academic Institution. J Clin Microbiol. 2020;58(8):e01136-20. Published 2020 Jul 23. doi:10.1128/JCM.01136-20

 

Evaluating the accuracy of different respiratory specimens in the laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections

Design

Uncontrolled analysis

N = 213

Objective

To evaluate and compare the efficacy of different respiratory specimens (including throat, nasal, sputum, and bronchoalveolar lavage fluid)

Study Groups

Severe case (n=37)

Mild case (n=176)

Methods

Respiratory samples (nasal swabs, throat swabs, sputum and bronchoalveolar lavage fluid) were taken from Guangdong CDC 7 confirmed novel coronavirus pneumonia (NCP) patients. Results were analyzed in combination with sample collection date and clinical information. Sample collection dates were divided into 0~7, 8~14 and ≥ 15 d.a.o groups, and patients were divided into severe and mild cases according to the guidelines of 2019-nCoV infection from the National Health Commission of the People’s Republic of China.

Duration

Patient samples taken from: 01/11/2020 to 2/3/2020

The samples were evaluated 1-24 days after being collected

Outcome Measures

Primary outcome: positive rate for COVID-19 of various tests, including nasal swabs, throat swabs, sputum, and bronchoalveolar lavage fluid (BALF)

Baseline Characteristics

  Severe (n=37) Mild (n=176)  

Median age, years (range)

65 (34-81) 47 (2-86)  

Male

23 (62.2%) 85 (48.3%)  

Sample Types 

Throat Swabs

Nasal Swabs

Sputum 

Bronchoalveolar lavage fluid (BALF)

n=260

93

96

45

26

n=606

112

394

97

3

 

Days after illness onset of first specimen collection (range)

7 (2-16) 4 (1-17)  

Results

 

Severe Case

Mild Case

P-value

Positive Rate 0-7 days after illness

Throat

Nasal

Sputum

Bronchoalveolar lavage fluid

 

12/20 (60%)

11/15 (73.3%)

8/9 (88.9%)

0/0

 

46/75 (61.3%)

147/204 (72.1%)

37/45 (82.2%)

0/0

 

1.000

1.000

0.26

N/A

Positive Rate 8-15 days after illness

Throat

Nasal

Sputum

Bronchoalveolar lavage fluid

 

18/36 (50%)

34/47 (72.3%)

15/18 (83.3%)

12/12 (100%)

 

8/27 (29.6%)

96/179 (53.6%)

32/43 (74.4%)

0/3 (0%)

 

0.127

0.03

525

0.002

Positive Rate ≥15 days after illness

Throat

Nasal

Sputum

Bronchoalveolar lavage fluid

 

14/38 (36.8%)

17/34 (50%)

11/18 (61.1%)

11/14 (78.6%)

 

1/9 (11.1%)

6/11 (54.5%)

3/7 (42.9%)

0/0

 

0.236

1.000

0.656

N/A

Adverse Events

Not studied

Study Author Conclusions

The results indicate that sputum may serve as the most sensitive samples for the virus detection, and followed by nasal swabs. Throat swabs were not recommended for virus detection, especially the samples collected 8-14 and ≥15 days after illness onset from mild cases, which may result in a large proportion of false-negative results.

InpharmD Researcher Critique

Results are specific to the specific tests used in this study (QIAamp RNA Viral Kit and qRT-PCR GeneoDX kit). The results may be skewed given that most of the samples were collected after antiviral treatment, possibly influencing viral shedding.



References:

Yang Y, Yang M, Shen C, et al. Evaluating the accuracy of different respiratory specimens in the laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections. 2020;medRxiv 2020.02.11.20021493. doi: https://doi.org/10.1101/2020.02.11.20021493

 

Diagnosis of the Coronavirus disease (COVID-19): rRT-PCR or CT?

Design

Retrospective study

N=87

Objective

To evaluate the diagnostic value of computed tomography (CT) and real-time reverse-transcriptase-polymerase chain reaction (rRT-PCR) for COVID-19 pneumonia

Study Groups

COVID-19 (n= 36)

Control group (n= 51)

Methods

Inclusion criteria: patients with a fever of > 38℃ and COVID-19 suspicion; underwent both CT of the chest and rRT-PCR

Exclusion criteria: transferred to another hospital or lost to follow-up

Patients with COVID-19  suspicion were examined by both CT and rRT-PCR at initial presentation. The sensitivities of both tests were then compared. 

Duration

From January 20 to February 8, 2020

Outcome Measures

Primary endpoint: comparison the sensitivity of CT imaging and rRT-PCR testing at presentation.

Baseline Characteristics

 

COVID-19 (n= 36)

Control (n=51)

 

Age, years

44.8 ± 18.2 47.1 ± 18.8  

Female

16 (44%) 25 (49%)  

Duration of fever, days

2.6 ± 1.7 3.2 ± 1.6  
Exposure History 33 (91.7%) 29 (56.8%)  
Results

Thirty-six patients (41%) were diagnosed with COVID-19:

  • 6/36 patients (17%) with negative rRT-PCR and positive CT at initial presentation (false-negative)
  • 1/36 patients (3%) with negative CT and positive rRT-PCR at initial presentation (false-positive)
  • 29/36 patients (80%) with positive CT and rRT-PCR at initial presentation

Study Author Conclusions

Computed tomography (CT) appears sensitive to virus detection, whereas rRT-PCR may produce false-negative results. The authors suggest that patients with typical CT findings but negative rRT-PCR results should be isolated, and rRT-PCR should be repeated to avoid misdiagnosis.

InpharmD Researcher Critique

Due to the outbreak of COVID-19 pneumonia in this area of China, the supply of nucleic acid detection kits was limited, and the rRT-PCR examinations were only performed in patients with fever and positive CT tests. Additionally, the sample size of this study was small, and the cases lacked follow-up due to time constraints. Larger sample sizes are therefore required for further verification.



References:

Long C, Xu H, Shen Q, et al. Diagnosis of the Coronavirus disease (COVID-19): rRT-PCR or CT?. Eur J Radiol. 2020;126:108961. doi:10.1016/j.ejrad.2020.10896

 

False negative RT-PCR or false positive serological testing in SARS-CoV-2 diagnostics? Navigating between Scylla and Charybdis to prevent misclassification bias in COVID-19 clinical investigations

Design

Observational study

N= 20

Objective

To develop a sick cohort of COVID-19 negative patients to serve as a control group, enabling identification of laboratory values unique to COVID-19 patients

Study Groups

COVID-19 negative control patients (n= 20)

Methods

Inclusion Criteria: currently sick (presenting with undifferentiated respiratory symptoms), admitted to the emergency department of the University of Cincinnati Medical Center, COVID-19 negative at the time of admission (via RT-PCR testing) 

Exclusion Criteria: COVID-19 positive at the time of admission (via RT-PCR testing)

Twenty patients who presented to the emergency department of the University of Cincinnati Medical Center with undifferentiated respiratory symptoms were prospectively enrolled and initially classified according to the results of their standard-of-care nasopharyngeal RT-PCR test for COVID-19. Samples were taken during collection of routine blood for clinical labs in the emergency department.

Serology was tested from these blood samples using the EURO-IMMUN Anti-SARS-CoV-2 ELISA IgG and IgA immunoassay.

Duration

March to May 2020

Outcome Measures

Primary Outcome: false-negative COVID-19 test result

Results

 

Sick COVID-19 Negative Control Group (n= 20)

Initial RT-PCR Test Result, negative

20 (100%) 
Secondary EUROIMMUN AG Test Result, negative

15 (75%)

Clinical suspicion of COVID-19 despite multiple RT-PCR testing

2 (10%)

Remaining sick COVID-19 negative control group patients

13 (65%)

Seven patients total (35%) from the initial COVID-19 negative control group were excluded from the control group. Five of the seven patients were found to be positive for anti-SARS-CoV-2 IgA, and the last two patients were excluded due to high clinical suspicion of COVID-19 despide multiple RT-PCR negative results.

Study Author Conclusions

Recruitment of "sick" control subjects who are truly SARS-CoV-2 negative remains a rather challenging enterprise, and otherwise well-designed studies are likely to produce biased results if RT-PCR tests alone are used to define COVID-19 negative cohorts.

InpharmD Researcher Critique

One major limitation of this study is its small sample size of only twenty participants in just one hospital. However, the author used these results to develop a potential algorithm to help reduce future false-negative COVID-19 results, which consists of: (1) standard-of-care RT-PCR tests should be used to exclude COVID-19 positivity, (2) patients with either IgA or IgG positive SARS-CoV-2 serology test results should be excluded from enrollment, and (3) screening serology negative controls with newly available prediction tools, such as the CoronaScore.



References:

Benoit J, Benoit SW, Lippi G, Henry BM. False negative RT-PCR or false positive serological testing in SARS-CoV-2 diagnostics? Navigating between Scylla and Charybdis to prevent misclassification bias in COVID-19 clinical investigations. Diagnosis (Berl). 2020;[E-pub].

 

Performance of VivaDiag COVID‐19 IgM/IgG Rapid Test is inadequate for diagnosis of COVID‐19 in acute patients referring to emergency room department

Design

Observational study

N= 110

Objective

To validate VivaDiag COVID‐19 IgM/IgG Rapid Test lateral flow immunoassay (LFIA) for the rapid diagnosis of COVID‐19

Study Groups

Volunteers with documented COVID negative result (n= 30)

Patient with documented COVID positive result (n= 30)

Patients with fever and respiratory symptoms (n= 50)

Methods

VivaDiag COVID‐19 IgM/IgG from VivaChek was performed according to the manufacturer's instruction. Respiratory samples were collected. Specific real‐time RT‐PCR targeting RNA‐dependent RNA polymerase and E genes were used to detect the presence of SARS‐CoV-2. Data from serological tests were compared to molecular results to define specificity, sensitivity, positive predictive value (PPV), and negative predictive value (NPV) of the rapid serological test.

Duration

Samples were taken  at a median 7 days after the first COVID-19 positive result

Outcome Measures

Primary outcome: results from VivaDiag COVID‐19 IgM/IgG Rapid Test and from specific real‐time RT‐PCR test

Baseline Characteristics

 

Patients with fever and respiratory symptoms (n= 50)

Patient with documented COVID positive result (n= 30)

Volunteers with documented COVID negative result (n= 30)

Age, years

61.5 73.5 38.5

Male

34 (68%) 25 (83.3%) 11 (36.7%)

Results

  Patients with fever and respiratory symptoms (n= 50)    

Real-time RT-PCR nasal swab

Positive

Negative

 

38 (76%)

12 (24%)

   

VivaDiag COVID‐19 IgM Rapid Test

Positive

Weakly positive

Negative

 

4 (8%)

3 (6%)

43 (86%)

   

VivaDiag COVID‐19 IgG Rapid Test

Positive

Weakly positive

Negative

 

0 (0%)

5 (10%)

45 (90%)

   

Twelve of 50 (24%) were negative for COVID‐19 by real‐time RT‐PCR. Of these, 1 (8.3%) showed a positive result for the VivaDiag COVID‐19 IgM/IgG Rapid Test.

Thirty-eight patients were positive for COVID‐19 by real‐time RT‐PCR. Of these, 7 (18.4%) showed a positive or weak positive serology for IgM and/or IgG, while the other 31 of 38 (81.6%) tested negative for the rapid serology assay.

The sensitivity of the VivaDiag COVID‐19 IgM/IgG Rapid Test was 18.4%, specificity was 91.7%, NPV was 26.2%, and PPV was 87.5% in patients enrolled from the emergency room department.

Adverse Events

Not studied

Study Author Conclusions

On the basis of the results, VivaDiag COVID‐19 IgM/IgG Rapid Test LFIA is not recommended for triage of patients with suspected COVID‐19.

InpharmD Researcher Critique

The VivaDiag COVID‐19 IgM/IgG Rapid Test LFIA showed has a low sensitivity and a low NPV in detecting SARS‐CoV-2. The majority of patients that tested positive for COVID‐19 by real‐time RT‐PCR would have been identified as negative using only the rapid serological assay, leading to a misdiagnosis of COVID‐19 disease in the vast majority of patients. Therefore, it is not recommended to use VivaDiag COVID‐19 IgM/IgG Rapid Test LFIA for triage of patients with suspected COVID‐19.



References:

Cassaniti I, Novazzi F, Giardina F, et al. Performance of VivaDiag COVID-19 IgM/IgG Rapid Test is inadequate for diagnosis of COVID-19 in acute patients referring to emergency room department. J Med Virol. 2020;10.1002/jmv.25800. doi:10.1002/jmv.25800