A Prospective Study to Evaluate the Clinical Performance of A Novel Multitarget-Based Antigen Test on the Detection of Sars-Cov-2 in Symptomatic Population-Comparison to Nuclear Acid-Based Tests

The study was designed to evaluate the clinical performance characteristics of the First Sign® SARS-CoV-2 Antigen Test, a visual read lateral flow immunoassay (W.H.P.M., Inc.). Individuals presenting at the study sites were evaluated by health care providers for COVID-19 symptoms and sequentially enrolled. The standard of care nasopharyngeal swab sample was first collected for RT-PCR testing. Participant self-collection nasal swab sample was then obtained for the Antigen Test. The clinical performance characteristics of the study device were calculated by comparing participant study device test result recorded at the point of care with the participant matched RT-PCR results provided by qualified CLIA high complexity laboratories. A total of 128 symptomatic adult participants were included in the study, of which 46.8% (60/128) were determined to be RT-PCR positive for SARS-CoV-2. The mean comparator cycle threshold value (Ct) for RT-PCR positive participants included in the study was Ct 25 (range Ct 18-Ct 39). The First Sign® SARS-CoV-2 Antigen Test demonstrated an overall clinical sensitivity of 96.7% (58/60) and a clinical specificity of 98.5% (67/68) when compared with EUA RT-PCR comparator results for the detection of SARS-CoV-2 in symptomatic individuals within 5 days of symptom onset. The clinical sensitivity for samples with RT-PCR Ct<30 was 100% (48/48), 90% (10/12) for samples with RT-PCR Ct ≥ 30, and 98.2% (55/56) for samples with RT-PCR Ct ≤ 34. Study results indicate that during the initial phase of infection, when upper respiratory viral loads and viral shedding levels are typically highest, an observed anterior nasal swab self-collected by the patient and applied to the First Sign® SARS-CoV-2 Antigen Test may be as effective in detecting the presence of SARS-CoV-2 in symptomatic individuals as a EUA RT-PCR assay. Additionally, the ability to obtain a low-cost qualitative result within 15 minutes, combined with the simplicity of the anterior nasal self-sampling procedure, may dramatically increase testing efficiency while significantly reducing risk to health care providers by minimizing or eliminating the need for close contact with the patient.


Introduction
The COVID-19 pandemic, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has to date infected approximately 100,000,000 individuals and resulted in over 2,000,000 deaths globally [1]. Current approach for the diagnosis of this RNA virus infection relies on nucleic acid amplification tests (NAATs). Reverse-transcription polymerase chain reaction (RT-PCR) molecular assay has been considered the gold standard for SARS-CoV-2 diagnosis. Indeed, in acute respiratory infection, RT-PCR is routinely used to detect causative viruses from respiratory secretions [2]. However, with the rapid progress of the COVID-19 pandemic, the test approach using RT-PCR alone has demonstrated a number of limitations that hindered the control of this devastating pandemic. RT-PCR is time-consuming and relatively expensive, requiring On the top of that, there is the need to perform the test in a well-controlled laboratory setting and to account for the long turnaround time of the test [3]. This proved difficult even for well-funded health care systems and led to delayed responses at the onset of the pandemic.
Recently, several SARS-CoV-2 antigen-based tests, which work via a lateral flow immunoassay mechanism, have been developed and received Emergency Use Authorization (EUA) from the United States Food and Drug Administration (FDA) [4][5][6]. Unlike serology assays, which detect the presence of the target antibodies that are developed in later stages of the body's immune response, antigen-based tests are capable of detecting the early onset of SARS-CoV-2 infection. The short turnaround time and one-step procedure of antigen tests may provide added value for rapid screening, patient triage, and case identification. The less expensive antigen-based tests may contribute to mass producibility when compared to RT-PCR, making rapid testing more readily available for lower-and middle-income countries. Antigen tests can also be used in high-risk congregate settings in which repeat testing could quickly identify persons with a SARS-CoV-2 infection to inform infection prevention and control measures, thus preventing transmission.
However, although rapid antigen tests generally have a satisfactory specificity (between 90% and 100%), they demonstrated a lower diagnostic sensitivity in comparison to molecular testing, especially with specimens containing low viral loads [7]. The implementation and widespread usage of an antigen test should be verified under well-defined circumstances. First Sign® SARS-CoV-2 Antigen Test was recently developed and manufactured by W.H.P.M., Inc. This unique test uses hybrid combination of monoclonal antibody pairs for multi-epitope targeting of the nucleocapsid protein in an effort to improve diagnostic sensitivity and retain specificity. We conducted a prospective study to investigate the clinical performance of this test on the detection of SARS-CoV-2 infection in a COVID-19 symptomatic population. Study device results were compared with two different standard of care FDA Emergency Use Authorized (EUA) RT-PCR tests to determine the negative percentage agreement (NPA), the positive percentage agreement (PPA), the positive percentage agreement relative to RT-PCR comparator cycle threshold values (Ct), and the positive percentage agreement relative to symptom onset timeline.

Study design and subject enrollment
This is a prospective study to evaluate the clinical performance characteristics of the First Sign® SARS-CoV-2 Antigen Test. The study was carried out at two different sites at Los Angeles: Genx Laboratories (Point of Care Pandemic Response Site, Site A) and Angel Wings Medical Associates Urgent Care (Site B). Eligible participants presenting at the study sites between December 2020 and January 2021 were continuously screened and enrolled in the study. Eligible participants were defined as individuals presenting with symptoms commonly associated with COVID-19, reporting symptom onset <6 days prior to testing, capable of conducting self-nasal swab specimen collection, willing and able to provide informed consent, and undergoing RT-PCR sample collection. We excluded asymptomatic individuals and participants with "invalid" or "inconclusive" RT-PCR results. Patient age, gender, symptom onset time and exposure history information were collected. No study procedures were performed without an informed consent process or signature of a consent form. This research was performed in accordance with Good Clinical Practice guidelines and the Declaration of Helsinki. This study was approved by Advarra Institute Review Board (Number: Pro00048222).

Specimen collection and antigen test
Two participant samples were consecutively collected from study participants. The standard of care sample was collected first by health care provider via nasopharyngeal swab eluted in Viral Transport Medium (VTM) in accordance with the Comparator Instructions for Use (IFU). Approximately 3 to 5 minutes post standard of care sample collection participants were provided with a copy of the study device specific Patient Quick Guide, an unopened nasal swab, and a study device sample collection tube containing 350 µL of viral extraction buffer. Participants were instructed to follow the Patient Quick Guide step-by-step instructions and self-collect a nasal swab sample under the observation of the health care provider. As part of the step-by-step procedure, the swab was eluted, mixed, and sealed by the participant in the patient sample tube immediately after completing the self-nasal swab. The sealed patient sample was then returned to the operator for testing. Antigen test device and results determination are shown in figure 1.

RT-PCR test
The two study sites utilized two different RT-PCR systems. Comparator device A (Site A): Applied Biosystems™ Quant Studio™ 7 Pro Real-Time PCR/ Quidel Lyra® SARS-CoV-2 assay. Nasopharyngeal patient swabs eluted in VTM (Viraport Direct VTM 1.5 mL) were tested in accordance with the IFU and laboratory SOP. Viral nucleic acid was extracted with the NucliSens® easy MAG™ extraction system, and transferred to a 96-well plate for testing. Patient samples testing positive were repeated in duplicate for confirmation. Comparator Device B (Site B): Applied Biosystems™ 7500 Real-Time PCR/ Quest SARS-CoV-2 RT-PCR assay. Nucleic acids were extracted from patient

Statistical analysis
Data was entered and organized by Excel. Clinical performance analysis including clinical specificity (negative percentage agreement, or NPA), clinical sensitivity (positive percentage agreement, PPA), PPA relative to RT-PCR comparator cycle threshold values (Ct), and PPA relative to symptom onset timeline, were calculated using RT-PCR tests as the gold standard.

Study population and RT-PCR test results
A total of 128 continuously enrolled symptomatic adults were included in the study. Overall study workflow is shown in figure 2. Samples from two participants were excluded due to inconclusive comparator results, and one sample was excluded due to procedural error. Participants testing RT-PCR positive for SARS-CoV-2 accounted for 46.8% (60/128) of the study cohort. Study participants were enrolled as they presented to the clinical study sites. Those enrolled represented a spectrum of socioeconomic backgrounds, ethnicities, and education levels. Females represented 68.2% of the study population (77/128), and 56.7% (34/60) of those testing RT-PCR positive for SARS-CoV-2. The median age of female participants was 48 years old with an age range of 19 to 77 years old. Male participants accounted for 39.8% (51/128) of the study population and 44.3% (26/60) of the group that tested RT-PCR positive for SARS-CoV-2. The median age of male participants was 50 years old with an age range of 19 to 81 years old. In total, 44.1% (34/77) of all female participants included in the study tested RT-PCR positive for SARS-CoV-2, while 50.9% (26/51) of male participants included in the study tested RT-PCR positive for SARS-CoV-2 (Table 1).
Of the 60 RT-PCR positive results, 80% (48/60) were reported with a Ct<30, and the remaining 20% (12/60) were reported as Ct ≥ 30, including 6.6% (4/60) with Ct ≥ 35. The mean Ct value for RT-PCR positive participants included in the study was Ct 25 (range Ct 18-Ct 39) ( Table 2). The distribution of RT-PCR Ct values relative to Ct 30 is consistent with observations made by the FDA regarding the natural distribution of SARS-CoV-2 viral loads in a sequentially enrolled study population. The Ct values did not show a correlation with patient symptom onset time (Table 3).

Clinical performance of antigen test
The study device demonstrated an overall clinical sensitivity of 96.7% (58/60) and a clinical specificity of 98.5% (67/68) when compared with EUA RT-PCR comparator results for the detection of SARS-CoV-2 in symptomatic individuals (

Discussion
This rapid antigen test clinical performance validation study was conducted at a time in which the Los Angeles region encompassing both study sites underwent a dramatic increase of approximately 427,800 new COVID-19 cases within a 4-week time frame. During the same period, disease prevalence with the region was estimated to be as high as 10%-20% within Los Angeles County. Increasing test efficacy and generating timely results are critical. Low sensitivity is the main hurdle for the widespread usage of antigen test. Previously, Cochrane Diagnostic Test Accuracy Group summarized the varied sensitivity across all reported studies of rapid antigen test in August 2020. The average sensitivity was 56.2% (95% CI 29.5 to 79.8%) and average specificity was 99.5% (95% CI 98.1% to 99.9%; based on 8 evaluations in 5 studies on 943 samples). Data from individual antigen tests were limited with no more than two studies for any test [8]. There is a growing demand for a rapid and accurate diagnostic test. Recently, more antigen test approaches have been developed and reported [9][10][11]. Roche SARS-CoV-2 rapid antigen test clinical verification study enrolled 321 consecutive patients with 149/321 (46.4%) positive for SARS-CoV-2. The overall accuracy compared to molecular testing was 86.9%, with 72.5% sensitivity and 99.4% specificity [12]. A recent rapid fluorescence immunoassays (FIAs) evaluation study tested 32 positive and 32 negative RT-PCR characterized clinical samples. Two assays detected 30 (93.8%) and 29 (90.6%) samples in RT-PCR positive group [13]. Another quantitative assay for detecting SARS-CoV-2 antigen was investigated in 100 nasopharyngeal samples collected from 47 SARS-CoV-2-infected patients. The sensitivity and specificity were 75.7% and 96.0%, respectively [14]. While there seems to be a slightly improved sensitivity in detecting SARS-CoV-2 with these new generation of antigen tests, the overall sensitivity is still less than 80%, which means over 20% infections may be missed by the test. One potential approach to overcome the low sensitivity of the antigen test is to utilize multiple antibody targets for the detection. This is the basis for WHPM's First Sign® SARS-CoV-2 Antigen Test. WHPM's test is a lateral flow immunochromatographic membrane assay that applies a hybrid formulation of monoclonal antibodies to detect SARS-CoV-2 nucleocapsid protein from anterior nasal swab specimens, which potentially improves diagnostic sensitivity and retain the specificity. In this study, an overall clinical sensitivity of 96.7% and a clinical specificity of 98.5% were reported for the detection of SARS-CoV-2 in symptomatic individuals. This is comparable to EUA RT-PCR comparator when testing individuals within the first five days of symptom onset.
It is worth noting that most of antigen test studies demonstrated the sensitivity was highly correlated with the PCR cycle threshold value of the molecular test. The diagnostic value of the Panbio COVID-19 Ag Rapid Test (Abbott) was investigated in community-dwelling mildly symptomatic subjects. In community-dwelling subjects with mild respiratory symptoms, the Panbio COVID-19 Antigen Rapid Test had a sensitivity above 95% for nasopharyngeal samples when using Ctvalues <32 cycles as cut-off for RT-qPCR test positivity [15]. Another Roche SARS-CoV-2 rapid antigen study retrospectively tested 75 swabs from patients previously tested positive by SARS-CoV-2 PCR and 75 swabs from patients previously tested negative. The assay's sensitivity with samples with a cycle threshold of <25 and 25-<30 was 100% and 95%, respectively. However, the sensitivity dramatically reduced to 44.8% and 22.2% for samples with a cycle threshold of 30-<35 and >35, respectively [16]. A similar trend was observed in our study. In our study population, 80% of the RT-PCR positive patients with COVID-19 symptoms had a <30 cycle threshold. Currently, the clinical value of cycle threshold and correlation between cycle threshold value and severity of disease or mortality of patients are still debatable [17,18]. High sensitivity is important for massive population screening. Further studies are needed to investigate the detection of patients with low viral load. Interestingly, when using viral culture in determining the risk for potential transmissibility, in comparison to culture-positive results, the antigen test demonstrated a higher positive predictive value (90%) than RT-PCR (70%) [19]. Another viral infectivity study investigated antigen test for detecting infectious patients. Antigen tests demonstrated a more significant correlation with cell culture infectivity (61.8-82.4%) compared to RT-PCR. These results indicated that rapid antigen test might be an alternative for screening potentially infective patients and reducing viral spread [20]. It should also be noted that both RT-PCR and antigen tests carry the risk of generating false-negative and false-positive results. Negative results may not exclude the possibility of SARS-CoV-2 infection. Patient clinical features need to be carefully reviewed for the management of COVID-19 pandemic.
Since the outbreak of the COVID-19 pandemic, several variants of SARS-CoV-2 have been identified in the UK, South Africa, and Brazil [21,22]. A latest study identified another novel variant of SARS-CoV-2, CAL.20C in California, United States. This strain is defined by three mutations in the spike protein within a known receptor binding domain that has been found to be resistant to certain spike protein monoclonal antibodies [23,24]. The impact of these mutations on the body's immune response and the effectiveness of vaccines are being investigated. A reasonable concern is whether rapid antigen test could cover these mutations. The study device, with multiple monoclonal antibody pairs each targeting different epitopes of the nucleocapsid protein, may be capable of detecting these potentially dangerous mutated strains. Previous genome-wide analysis of SARS-CoV reported that the nucleocapsid gene is more stable and has acquired fewer mutations over time [25]. Immunoinformatic analysis of SARS-CoV-2 also demonstrated immune genic areas in nucleocapsid protein in the context of vaccine design [26,27]. Multi-epitope targeting of a well-conserved viral protein may be an effective method to mitigate the risk of lost sensitivity or false-negative results caused by viral mutations. However, additional studies will be needed to test this hypothesis.

Conclusion
In conclusion, the clinical performance characteristics of the First Sign® SARS-CoV-2 Antigen Test was observed to be comparable to the EUA RT-PCR comparator when testing individuals within the first 5 days of symptom onset. Study results indicate that during the initial phase of infection, when upper respiratory viral loads and viral shedding levels are typically highest [28], an observed anterior nasal