Introduction
Healthcare systems worldwide were severely strained by the rapid spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). In general, detection and management of a growing surge of infections requires diagnostic tests that are easy to use and produce rapid results. Subpopulations, such as patients with asymptomatic infections and children, are poorly represented in studies that examine the accuracy of diagnostic tests [
1].
Since the beginning of the COVID-19 pandemic, laboratories have used nucleic acid amplification tests, such as real-time reverse transcription polymerase chain reaction (RT-PCR) assays, to detect SARS-CoV-2, but these tests are more laborious and time-consuming than the rapid antigen-detection tests (Ag-RDTs) [
2]. Ag-RDTs are designed to directly detect SARS-CoV-2 antigens produced by the replicating virus in the respiratory tract using point-of-care testing. After collecting a respiratory specimen and performing the test, the results are typically available within 30 min. The World Health Organization (WHO) recommended the use of Ag-RDTs that meet the minimum requirements of 80% sensitivity and 97% specificity [
2,
3].
Previous studies showed that the sensitivity of RT-PCR tests from the upper respiratory tract (nasal or nasopharyngeal swabs) was highly variable. In addition, a 2021 Cochrane systematic review of 48 studies using Ag-RDTs reported a notable difference in the sensitivity for symptomatic adults (72.0%, 95% CI: 63.7–79.0%) and asymptomatic adults (58.1%, 95% CI: 40.2–74.1%) [
1]. A July 2022 Cochrane systematic review analysed 152 studies and reported higher variations in sensitivity for symptomatic adults (73.0%, 95% CI: 69.3.2–76.4%) than asymptomatic adults (54.7, 95% CI: 47.7–61.6%), but consistently high specificity [
4].
More recent studies have reported improved accuracy of Ag-RDTs, specially for patients with high viral loads [
5]. A 2021 study of the Panbio™ Rapid Antigen Test with nasopharyngeal swabs in Mallorca (Spain) examined 1369 participants (≥ 18-years-old), reporting a sensitivity of 71.4%, being higher in symptomatic patients who were tested within 5 days since the onset symptoms (80.4%) [
6]. Dinnes et al
. (2022) compared the sensitivity in children and adults and found that the average sensitivity was 9.9% higher in adults, although this difference was not statistically significant [
4]. Although some studies using Ag-RDTs obtained high sensitivity and specificity, there are typically small numbers of cases in certain subpopulations, making it difficult to form definitive conclusions regarding test accuracy in these groups [
7]. For example, one study conducted in Spain examined 412 patients who visited Primary Care Health Centres (PHCs) and emergency wards, but only included 85 children (< 16 years-old). They found the sensitivity was higher in adults (82.6%, 95% CI: 63.3–90.9%) than children (62.5%, 95% CI: 30.6–86.3%) [
8]. Gonzalez-Donapetry et al
. (2021) examined 440 nasopharyngeal swabs taken from children at a hospital paediatric emergency department, and found the overall sensitivity was 77.7% and specificity was 100% [
9], whereas Villaverde et al
. [
10] examined 1620 symptomatic children and reported a sensitivity of 45.4% (95% CI: 34.1–57.2%).
Sample collection is a critical factor affecting the performance of any diagnostic test that examines respiratory fluids, especially in children. Alemany et al
. (2021) [
11] obtained adequate results from less invasive samples (nasal samples, saliva samples, or both) in symptomatic and asymptomatic subjects. Moreover, Spanish authorities provided a provisional authorization for the Panbio™ Test in November 2020, and the manufacturer reported a sensitivity of 98.1% and a specificity of 99.8% [
12]. However, there is a need for additional studies conducted in PHCs with large samples of children to establish the accuracy of Ag-RDT in paediatric populations. In particular, children are more likely to accept mid-turbinate sampling than nasopharyngeal sampling.
The main aim of this study was to evaluate the accuracy of the Panbio™ Test when used at a PHC in children who are symptomatic or had close contact with an infected person, with the RT-PCR test used as the gold standard.
Methods
Design and setting
This prospective diagnostic study was conducted from February to April 2021 in Mallorca (Balearic Islands, Spain) at different locations in the cities of Palma and Inca. There were 1023 participants recruited at a PHC testing location and 119 at the Emergency Department of the Son Espases University Hospital.
Study population
All participants were 18 years-old or younger, unvaccinated, and attended one of the above-named settings for RT-PCR testing. All of them had symptoms suggesting infection, who were referred by a general practitioner (GP), or were exposed to another patient who had an infection confirmed by RT-PCR (close contact), who were referred by a specific COVID-19 call centre.
SARS-CoV-2 testing
Trained health professionals collected all samples. First, a nasal mid-turbinate sample (for the Ag-RDT) was collected by inserting the swab about 2 cm into each nostril and rotated after insertion. Then, a nasopharyngeal sample (for the RT-PCR test) was collected by deep insertion of a swab into a single nostril and rotated.
RT-PCR
Within 24 h of sample collection, the nasopharyngeal swab was sent for processing at the Microbiology Department of Son Espases University Hospital. The lab technicians performing the RT-PCR tests were blinded to the results of the Ag-RDTs and all additional information about the participants. RNA extraction was performed using the MagMAX™ Viral/Pathogen Nucleic Acid Isolation Kit (ThermoFisher) and amplification was performed using the TaqPath™ COVID-19 CE-IVD RT-PCR Kit and QuantStudio™ (ThermoFisher). The relative viral load was expressed as the cycle threshold (Ct) for the open reading frame (ORF), nucleocapsid (N), and spike (S) genes. High viral load was considered as a Ct below 25, a moderate viral load as a Ct of 25 to 29.9, and a low viral load as a Ct above 30 [
2].
Rapid antigen test
The nasal mid-turbinate swab was analysed on-site using the Panbio™ Test COVID-19 Ag Rapid Test Device (nasal) (Abbott Diagnostics GmbH, Germany), and the results were interpreted within 15 min as described by the manufacturer. The sample on the swab was mixed in 300 µL of buffer, and then 5 drops were dispensed onto the device. This chromatographic test contains an immobilized anti-SARS-CoV-2 antibody that binds to the N protein on the test line and a mouse monoclonal anti-chicken IgY on the control line. The two conjugates (a human IgG specific to a SARS-CoV-2 Ag gold conjugate that binds the N protein [test] and a chicken IgY gold conjugate [control]) move upward on the membrane and react with their corresponding antibodies. Neither the control line nor the positive test line was visible in the result window before adding the specimen. After 15 min, a visible control line indicated the test was valid and a visible second line indicated a positive result.
Questionnaire
Participants answered a short questionnaire that asked about the reasons for the RT-PCR testing, socio-demographic information (gender and age), presence and type of symptoms, and number of days since symptom onset or epidemiological exposure to a positive patient.
Statistical analysis
To evaluate the accuracy of the Panbio™ Test, the initial pre-test probability was 12.7% for symptomatic patients and 7.3% for asymptomatic patients. The sample size was calculated according to a sensitivity of 85% and a precision of 10% for symptomatic patients, and a sensitivity of 70% and a precision of 15% for asymptomatic patients. Thus, it was necessary to test 880 participants.
The sensitivity, specificity, and their 95% CIs of the Ag-RDT were calculated using the RT-PCR results as the reference. Sensitivity analysis was also performed by stratification according to the declared reason for RT-PCR testing, the presence of symptoms, days since symptom onset or epidemiological exposure, and Ct-values. Predictive values and 95% CIs were estimated using the pre-test probability results for each analysed group. Means and standard deviations were used for descriptive analysis of the study populations. Interaction testing through a binary logistic regression model was used to determine the statistical significance differences between groups (symptomatic vs. asymptomatic; tested within 5 days or > 5 days; according to the viral load). All statistical calculations were performed using SPSS statistical software version 25.0 (SPSS Inc., Chicago, IL, USA).
Discussion
We assessed the accuracy of the Panbio™ Test for SARS-CoV-2 in a paediatric population. During the 3 months study period, most patients were referred to PHCs because they had close contact with individuals who had confirmed COVID-19 infections or had symptoms consistent with COVID-19. The major result of this study was that this test had the highest sensitivity when an individual was referred by a GP or paediatrician and when testing was performed within 5 days since the onset of symptoms or epidemiological exposure, although no statistically significant differences were found. It is also important that the specificity of this test was above 98.9% overall and in all subgroups. The WHO recommended that Ag-RDTs for SARS-CoV-2 have sensitivity of at least 80% and specificity of at least 97% [
2,
3]. Thus, in our paediatric population the Panbio™ test had acceptable specificity but unacceptable sensitivity.
A 2021 meta-analysis that assessed the diagnostic accuracy of Ag-RDTs for COVID-19 in paediatric populations under real-world conditions concluded that the performance of the current Ag-RDTs varied broadly among children [
13]. Other studies showed that the sensitivity of the Ag-RDTs was higher in adults than children [
4,
7,
9]. Gonzalez-Donapetry et al
. [
9] also tested nasopharyngeal swabs in a paediatric population using the Panbio™ Test and reported a sensitivity of 77.7%, with a lower value in children than adults. Eliseo et al
. [
8] reported a sensitivity of 62.5% (95% CI: 30.6–86.3%) in children (
n = 412) in PHC. Villaverde et al
. [
10] examined 1620 symptomatic children and reported a sensitivity of 45.4% (95% CI: 34.1–57.2%). Carbonell-Sahuquillo et al
. [
14] examined 357 patients at a paediatric emergency department and reported this test had a sensitivity of 70.5%. Möckel et al
. reported a sensitivity of 72.0% (95% CI: 53.3–86.7%) in paediatric cohort (
n = 202) in an emergency department [
15]. Pollock et al
. used a similar Ag-RDT (BinaxNOW) that uses anterior nasal swabs and reported a lower sensitivity in children than in adults at the hospital; the sensitivity was 84.6% (95% CI: 65.1–95.6%) in symptomatic children and 65.4% (95% CI: 55.6–74.4%) in asymptomatic children [
16]. Sood et al
. also used the BinaxNOW test (
n = 226) and reported a sensitivity of 56.2% (95% CI: 49.5–62.8%) [
17]. Thus, Ag-RDTs appear to be unsuitable for testing an entire asymptomatic population, a practice performed early during the pandemic in some countries [
18], in which testing was performed in locations such as airports, train stations, and schools [
19]. In contrast, the sensitivity was 70.0% (95% CI: 53.2–82.9%) in our symptomatic children, and was slightly higher when the child was referred by a GP for symptoms compatible with COVID-19 (73.0%, 95% CI: 52.0–87.5%). This result highlights the importance of these professionals in a real-world setting. Furthermore, a possible explanation for the lower sensitivity in children than adults in the present study and in previous studies may be the difficulty in collecting nasal samples from children. Sample collection is one of the most important factors affecting the performance of any diagnostic test [
3].
Notably, other SARS-CoV-2 tests also showed lower sensitivity for asymptomatic than symptomatic individuals [
20]. Drêvínek et al
. concluded that because Ag-RDTs have low sensitivity in asymptomatic individuals, they should be administered repeatedly at high frequency, and RT-PCR testing should be used for the most vulnerable populations [
21]. This aligns with the study by Torres et al
. [
5], who concluded that the Panbio™ Test had low sensitivity in asymptomatic close contacts of COVID-19 patients. In 2021, the WHO concluded that Ag-RDTs should be prioritized for symptomatic individuals who meet the case definition for COVID-19 and asymptomatic individuals who have high risk of infection. Moreover, it must be considered that the Ag-RDTs are less sensitive than the RT-PCR test, especially in asymptomatic patients [
3]. In contrast, Polechová et al. recommended use of Ag-RDTs at least 2 to 3 times per week as a large-scale Ag-RDT strategy for Austrian schools [
22].
Only 95 of our children were tested more than 5 days after the onset of symptoms or epidemiological exposure, and the overall sensitivity in these children was only 50.0% (95% CI: 9.1–90.8%). This is in accordance with the results of He et al
., who concluded that transmission could potentially occur 2 to 3 days before symptom onset, but only 1% of all transmissions occurred within 5 days before symptom onset [
23]. These results suggest that testing of paediatric populations should be prioritized for children within 5 days since symptom onset.
Notably, the overall pre-test probability in our study was only 7.3%. There is evidence that a lower disease prevalence leads to a lower PPV for Ag-RDTs [
24,
25]. Nonetheless, we had a high overall PPV (96.1%, 95% CI: 85.6–99.4%) and a high overall NPV (96.8%, 95% CI: 95.6–97.7%).
Some studies showed that despite the high specificity of the Ag-RDTs, false positive results can occur, especially when testing a population with a low prevalence [
3]. Although our pre-test probability was only 7.3%, so was our percentage of false positive results (0.2%). Our two false positive results were in children who were 5 years-old and 7 years-old. Similarly, L’Huillier et al. [
26] reported a trend for lower Ag-RDT sensitivity in symptomatic children who were under 12 years-old relative to those who were 12 years-old or more.
Use of the RT-PCR test with a nasopharyngeal swab is the reference method for detection of SARS-CoV-2 [
23,
27], and it is likely that nasal specimens have lower sensitivity than nasopharyngeal specimens [
28,
29]. However, we postulate that because of difficulties in obtaining nasopharyngeal samples from children, nasopharyngeal and nasal samples could have similar diagnostic value in children. There is also evidence that nasal samples could be an alternative to nasopharyngeal samples for RT-PCR testing [
30] for SARS-CoV-2. Thus, when children show symptoms compatible with COVID-19, one possible option is to use the Panbio™ Test with a nasal sample, because it is non-invasive and better tolerated. However, but a negative Ag-RDT result in a patient who has a high clinical suspicion of COVID-19 should be tested using RT-PCR [
9,
31‐
33].
It is important to highlight that this study mainly examined children in a PHC, and that most of the children who tested positive were asymptomatic or had mild disease. This is contrary to the situation at hospitals, in which most COVID-19 patients have high infectivity and the test sensitivity is therefore higher. For example, Eleftheriou et al
. (2021) [
34] examined 744 children in a tertiary children’s hospital and reported the Panbio™ Test had a sensitivity of 82.35%, and the sensitivity was more that 95% in symptomatic children, greater than in our study or any other study conducted in a PHC [
5].
Strengths and limitations
The main strength of this study was the availability of detailed characteristics of all paediatric participants in a PHC. Secondly, it is important to highlight the high specificity of the Panbio™ test in our population, which greater than 99.6% overall and in the different subgroups. Thirdly, this study has one of the largest samples (n = 1142) evaluating Ag-RDTs conducted in paediatric population in PHC. However, there were also some limitations. Firstly, the sensitivity in all 1142 patients was 59.5%, below the WHO recommendation of 80%. Secondly, the study was conducted in a non-vaccinated paediatric population. Thus, even though the vaccination rate is lower in children than adults, our results cannot be extrapolated to vaccinated children. Thirdly, this study was conducted using the Panbio™ Test (Abbott Diagnostics, GmbH, Germany), so caution should be used when applying our results to other Ag-RDTs.
Acknowledgements
We thank the nurses from primary health care centers and test sites that collected samples and performance the antigen test on- site without whom this work would not be possible, and all participants of the study that voluntarily participate.
Consortium name
Balearic COVID-19 Pediatric Primary Care Research Group members (alphabetical order):
Alfonso Leivaa,b, g, email: aleiva@ibsalut.es.
Anaida Obietac, email: anaida.obieta@ibsalut.es.
Victoria Pascualh, email: victoria.pascual@ssib.es.
Pau Pericasb, email: pau.pericas@ibsalut.es.
Carlos Radúanj, email: carlos.raduan@ibsalut.es.
Elsa Segurad, email: elsa.segura@hsll.es.
Verónica Vegai, email: veronica.vega@ibsalut.es.
aPrimary Health Care Research Unit, Balearic Public Health Service (Ib-Salut), Escola Graduada n9, 07002 Palma, Illes Balears, Spain.
bBalearic Islands Health Research Institute (IdISBa), Carretera de Valldemossa, 79, 07120 Palma, Illes Balears, Spain.
cSanta Ponça Primary Health Care Center, Balearic Public Health Service (Ib-Salut), Carrer del Riu Síl, 25, 07180 Santa Ponça, Illes Balears, Spain.
dSon Llatzer University Hospital, Ctra. de Manacor, 07198 Palma, Illes Balears, Spain.
eDirectorate of General Health Service, Balearic Public Health Service (Ib-Salut), Reina Esclaramunda n9, 07005 Palma, Illes Balears, Spain.
fMicrobiology Service, Son Espases University Hospital, Balearic Public Health Service (Ib-Salut), Carretera de Valldemossa, 79, 07120 Palma, Illes Balears, Spain.
gRICAPPS- Red de Investigación Cooperativa de Atención Primaria y Promoción de la Salud—Carlos III Health Institute (ISCIII), Madrid, Spain.
hSon Rutlan Health Care Center, Balearic Public Health Service (Ib-Salut), Passatge de Cala Figuera, 12, 07009 Palma, Illes Balears, Spain.
iEscorxador Health Care Center, Balearic Public Health Service (Ib-Salut), Carrer Emperadriu Eugènia 4 07010-Palma.
jMallorca Primary Health Care Directorate, Balearic Public Health Service (Ib-Salut), Escola Graduada n9, 07002 Palma, Illes Balears, Spain.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.