Evaluation of a new "all in one" SARS-CoV-2 antigen-detecting rapid diagnostic test and self-test: Diagnostic performance and usability in child and adult populations.

The control of the COVID-19 epidemics has been one global health priorities for the last 2 years. To that end, more reliable and easy-to-use, regardless of age, diagnostic tests are necessary. Considering that, we evaluated an innovative two-step self-test, the AAZ COVID-VIRO ALL IN®, switching from the classic nasal swab to a nasal sponge. We performed a multicenter study, on 124 adults and children, in a point-of-care setting. Sensitivity, specificity and overall acceptance of the COVID-VIRO ALL IN® self-test compared to reverse transcriptase polymerase chain reaction (RT-PCR) on nasopharyngeal samples were of 93.0%, 100%, and 97.5%, respectively. We then performed a multicenter, usability study to evaluate the ease of use of COVID-VIRO ALL IN® on 68 laypersons adults. A vast majority of participants correctly executed and interpreted the test. The usability was then specifically investigated on 40 children and teenagers, comparing COVID-VIRO® first generation to the new COVID-VIRO ALL IN®. They all found COVID-VIRO ALL IN® more comfortable and easier to use. For young children, the new self-test seems safer (less risk of trauma and no liquid exposure), and faster than saliva-based RT-PCR. Moreover, the COVID-VIRO ALL IN® can easily be adapted as a multiplex self-test for other respiratory viruses, opening new perspectives of simultaneous, rapid and massive detection of respiratory infections, especially among vulnerable populations like children and elderly people.

INTRODUCTION

The coronavirus disease 2021 (COVID‐19) is an infectious respiratory disease caused by the severe acute respiratory syndrome (SARS‐CoV‐2) virus. This disease has been acknowledged as a pandemic disease on March 11, 2020 by the World Health Organization (WHO). Since the beginning of the pandemic, variants of concern with mutation on the spike (S) protein have emerged. As of December 21, 2021, there are 274 million confirmed cases including 5 million deaths worldwide. To date, the COVID‐19 pandemic has continued to evolve as a global public health crisis.

Reverse transcriptase polymerase chain reaction (RT‐PCR) based on the molecular detection of the virus genetic material from a nasopharyngeal sample has routinely been used to detect SARS‐CoV‐2 infection. Although RT‐PCR is the gold standard for COVID‐19 testing, this method still requires very specific and expensive material/equipment, as well as trained staff to perform the test in a safe and reliable manner. Furthermore, results can take several days before acquisition, due to logistical issues (i.e., samples transport and results communication). This is problematic in a pandemic situation and that is why rapid tests are needed to quickly identify individuals who are likely to transmit the SARS‐CoV‐2 thus contributing to the spread of infection. Finally, although nasopharyngeal sampling is generally safe, this procedure is not exempt of any physical nor psychological risk, especially if performed in a repetitive and intensive manner. Indeed, although rare, the occurrence of complications such as epistaxis, retained foreign body, nasal septal infection, cerebrospinal leak, and meningitis can be found in the literature.

SARS‐CoV‐2 specific antigen lateral‐flow immunoassays, have been developed to detect the presence of the virus antigens from nasal samples. Less invasive than nasopharyngeal swabs, antigenic tests on nasal swabs have the advantage of combining rapid results with the possibility of repeated use in a wider audience. These Rapid Antigen Diagnostic Test (RADT) proved to be reliable and were adapted for at‐home testing, allowing large scale screening. Their practicality of use and the speed of obtaining results, in few minutes, enables early detection and isolation of COVID‐19 cases. The WHO recommend antigen‐detecting rapid diagnostic tests for symptomatic individuals in the first 5−7 days after symptoms onset.

The FDA has already approved several antigen home tests and the French health authority (Haute Autorité de Santé, HAS) has defined the minimal performance requirements for these tests with an emphasis on the necessity of conducting real life studies. , , To break the chains of contamination, especially in schools, the HAS had lifted the age limit, initially reserved for individual over 15 years of age, for the use of antigenic tests on nasal samples in April 2021.

In a previous study, we demonstrated that the COVID‐VIRO® (AAZ‐LMB, Boulogne‐Billancourt) antigen‐based rapid detection self‐test was appropriate RADT thanks to high performance and good usability assessments results, on an adult population. To further improve the test usability and thus user's satisfaction while keeping the best performance possible, AAZ developed a new, simpler COVID‐VIRO ALL IN® nasal samples test where all components of a classic cassette test are gathered in an “all‐in‐one” device.

COVID VIRO ALL IN® uses a nasal sponge sampling method integrated at the end of the device. The first objective of this study was to evaluate the diagnostic performance of COVID‐VIRO ALL IN® on children and adults compare to the reference method (nasopharyngeal RT‐PCR). The second objective was to evaluate usability of COVID‐VIRO ALL IN® for both children and adults. For children the usability of the COVID‐VIRO ALL IN® was additionally compared with the previous COVID‐VIRO® test.

MATERIALS AND METHODS

The study was approved in October 2020 by the French ethics committee (Comité de Protection des Personnes Nord‐Ouest IV) and was notified to the French data protection authority. In accordance with the Declaration of Helsinki, all adult participants or legal representative for children participants provided written informed consent before undergoing any study‐specific procedure.

The study was divided into two subparts according to the evaluation objective: performance evaluation conducted in the entire study population, and usability evaluation performed separately in adults and children. Usability for children was assessed by comparing COVID‐VIRO ALL IN® and COVID‐VIRO®.

While no specific inclusion/noninclusion criteria were applied for the usability part of the study, the performance part selected patients with the following characteristics: minors (ages from 5 to 17 years old) with the agreement of the legal representative or adults volunteers (>18 years old), harboring mild to moderate symptoms (headache, fatigue, fever, sore throat, aches and pains, loss of smell and taste, etc.), for less than 7 days and not requiring immediate hospitalization and naive for self‐testing. The noninclusion criteria were: hospitalized patients, symptomatic patients with symptoms duration >7 days, asymptomatic patients, or asymptomatic contact with a known case.

Patients from the two COVID units of the Centre Hospitalier Régional d'Orléans (La Madeleine Hospital and La Source Hospital) were considered for the performance study. Adult volunteers who took part into the usability study were patients of a medical analysis laboratory (Drouot laboratory), whereas children participants were recruited from the infectious diseases department of the Orleans Regional Hospital.

In vitro diagnostic device under investigation

COVID‐VIRO ALL IN® (AAZ‐LMB) is a vertical flow test using highly sensitive monoclonal antibodies to detect SARS‐CoV‐2 core antigen in a nasal sample. Monoclonal antibodies to the SARS‐CoV‐2 core protein are fixed to the test area (T) on a nitrocellulose strip. A monoclonal antibody to the SARS‐CoV‐2 core protein labeled with colloidal gold is used as a freeze‐dried conjugate.

In the test, a colored antibody−antigen complex is formed upon interaction of SARS‐CoV‐2 antigens with monoclonal anti‐SARS‐CoV‐2 antibodies. This complex is captured by the membrane bound monoclonal anti‐SARS‐CoV‐2 antibodies after capillarity‐induced migration to the test line (Figure 1). A colored test line appears in the results window only if SARS‐CoV‐2 antigens are present in the sample. The intensity of the colored test line will vary depending on the amount of SARS‐CoV‐2 antigens present in the sample. In the event of no SARS‐CoV‐2 antigen being present, the test line will not display any colouration. The control line is used as a procedural control and should always appear in the control area if the test procedure is performed correctly. Results can be visually interpreted after 15 min.

Figure 1

Visual appearance of the COVID‐VIRO ALL IN® test device and representation of the potential results.

Comparator

The RT‐PCR test for SARS‐CoV‐2 was performed in the virology unit of the CHR Orléans, France. Nucleic acid extraction was performed with an automated sample preparation system MGISP‐960 (MGI). Real‐time PCR detection of SARS‐CoV‐2 RNA targeting the ORF1ab, S, N genes was performed with the TaqPath V2 COVID‐19 Multiplex RT‐PCR kit (Thermofisher). Amplification was performed on QuantStudio5 (Applied Biosystems). The results of the assay were performed according to the manufacturer's instructions. The assay includes an internal RNA extraction control and an amplification control. The samples were analyzed taking into account the new positivity criteria of the French Microbiology Society's expert committee (version 4 of January 14, 2021), in particular taking into account the specific characteristics of the Thermofisher kit used for the RT‐PCR measurement.

Methodology

Performance study

Patients showing up to one of the study centers for RT‐PCR testing were informed about the study. If the patient met the inclusion criteria, he/she or his/her legal representative was asked to sign the consent to participate in the study. In a first step, the COVID‐VIRO ALL IN® nasal test was performed immediately on a fresh nasal sample by a trained nurse and test result entered in the form provided for this purpose. At no time was the result communicated to the patient. In a second step, the nurse took a nasopharyngeal swab for the RT‐PCR test. The Thermofisher TaqPath V2 COVID‐19 Multiplex RT‐PCR (Thermofisher) and the Thermofisher TaqMan SARS‐CoV‐2 Mutation Panel (Thermofisher) were performed by the hospital laboratory. The result was communicated to the patient within 24 h and recorded in the patient's file.

Usability study on adults (>18 years old)

Comprehension of instructions and test execution

Adult participants were asked to read the entire instructions for use before carrying out the self‐test. Then, they were asked to proceed with the test procedure by collecting a nasal sample in both nostrils with the nasal sponge included in the device, dipping the device back into the diluent pad, piercing the buffer capsule by pushing the device in the support, and waiting for the valid result. A questionnaire was to be completed by the participant at each step of the self‐test process to comment the experience (Table 1). A member of the study team (laboratory staff and nurse or doctor) observed the test without intervening and gave an a posteriori assessment of the various steps by filling in an evaluation form for each participant (Table 1).

Table 1

COVID‐VIRO ALL IN® usability and supervisor's questionnaires for Adults.

Interpretation of test results

A test result interpretation exercise was also planned as part of the usability study. The participant was asked to randomly select one of four contrived self‐tests (one negative, two positive, and one invalid), read it and give his/her interpretation of the result. The participant interpretation as well as his/her opinion on the ease of interpretation was collected by the study staff on a questionnaire (Table 2).

Table 2

Interpretation questionnaire for Adults.

Usability study on children and teenagers (<15 years old)

The two self‐tests, COVID‐VIRO ALL IN® and COVID‐VIRO®, were both performed by the child population. This population was divided into two age groups: the 3−11‐years‐old participants for whom the tests were performed by the parent/legal representative, and 12−15‐years‐old participants who independently performed the tests under investigation. As described for adults, it was asked to participants to comment on the different steps of tests on a questionnaire (Table 3). An a posteriori assessment of the performance of the various steps was given by an observer only for the 12−15 years old participants (Table 3). Finally, the exercise of result interpretation of tests already done and ready to be read was also performed by participants (Table 4). At the end, his/her preferences concerning the ease of reading of COVID‐VIRO ALL IN® or COVID‐VIRO® self‐test was requested (Table 5).

Table 3

COVID‐VIRO ALL IN® and COVID‐VIRO® usability and supervisor's questionnaires for Children/Teenagers.

Table 4

COVID‐VIRO ALL IN® and COVID‐VIRO® interpretation questionnaire for Children and Teenagers.

Table 5

COVID‐VIRO ALL IN® and COVID‐VIRO® comparison questionnaire for Children and Teenagers.

Data analysis

For the performance part of the study, populations were described in terms of percentage, mean, standard deviation, range, and median values. The concordance or discordance between RT‐PCR, the diagnostic reference method, and COVID‐VIRO ALL IN® test self‐test results was analyzed in the Department of Infectious Diseases. Results were classified in four categories:

‐ TP (true positive), corresponding to individuals both positive for RT‐PCR and COVID‐VIRO ALL IN® test,

‐ FP (false positive) for individuals COVID‐negative according to the reference method but considered positive by COVID‐VIRO ALL IN® test,

‐ FN (false negative) when RT‐PCR positive but negative the self‐test, and

‐ TN (true negative) for individuals with negative results for both tests, RT‐PCR and self‐test under study.

The specificity (Sp), sensitivity (Se), positive predictive value (PPV), negative predictive value (NPV) and overall percent agreement (percentage of correctly classified instances) of the COVID‐VIRO ALL IN® test compared to the reference test (RT‐PCR) were calculated according to the formulas indicated below. Confidence intervals (CI) for sensitivity and specificity were obtained with the Wilson score method.

Sp (%) = 100 x [TN/(TN + FP)]

Se (%) = 100 x [TP/(TP + FN)]

PPV (%) = 100 x [TP/(TP + FP)]

NPV (%) = 100 x [TN/(TN + FN)]

OPA (%) = 100 x [(TN + TP)/(TN + FN + TP + FP)]

For the usability part of the study, populations were described in terms of absolute number and percentage.

RESULTS

COVID‐VIRO ALL IN® performance study

A total of 124 participants was recruited. Five participants were excluded from the analysis: four participants identified as asymptomatic patients and one participant excluded because of a RT‐PCR considered to be unconclusive according to the classification criteria of the French Microbiology Society. Specifically, this patient was RT‐PCR positive for the N gene and ORF gene but with a C t value of 33 and 37, respectively. The patient was only suffering from muscles soreness and could not give a precise date of the beginning of pain. According to the French guideline, this sample, positive from a laboratory stand point, but excreting very low level of SARS‐CoV‐2 virus (C t > 32) can either be considered as weak positive or negative. On the basis of this ambiguous result, the participant was removed from the analysis.

The final study population consisted of 119 patients. The sex ratio was 0.78 (52 men and 67 women). The median age was 37 years (mean: 38 years, range: 80 years). Among this population, the median duration of symptoms before the sampling date was 2 days (mean: 2.55, range: 7). The results of the study performance are presented in Table 6. The sensitivity of the COVID‐VIRO ALL IN® test was evaluated at 93.0% (95% CI: 81.4%−97.6%) and the specificity equals to 100% (95% CI: 95.2%−100%). The positive and NPVs were 100.0% and 96.2% respectively. Overall agreement of results between RT‐PCR and COVID‐VIRO ALL IN® were observed for 116 patients (97.5% of concordance).

Table 6

Contingency table of the performance study in the overall children and adult population (N = 119).

	RT‐PCR positive	RT‐PCR negative
COVID‐VIRO ALL IN® positive	40 (33.6%)	0 (0%)
COVID‐VIRO ALL IN® negative	3 (2.5%)	75 (63.9%)
Sensitivity	93.02%	IC: 81.4−97.6
Spécificity	100%	IC: 95.2−100

Abbreviation: RT‐PCR, reverse transcriptase polymerase chain reaction.

The performance of the COVID‐VIRO® and COVID‐VIRO ALL IN® tests for the Omicron variant was evaluated on samples from patients confirmed positive by RT‐PCR for this variant. No decrease in sensitivity was observed for these Omicron positive samples.

COVID‐VIRO ALL IN® usability study

Substudy 1—COVID‐VIRO ALL IN®: Adults comprehension of instructions, test execution, and interpretation of results

A total of 68 patients (41 women and 27 men), from 18 to 67 years old (mean: 34 years) and from different occupational categories, participated in this study. None of them was excluded. Regarding the quality of the COVID‐VIRO ALL IN® self‐test instructions, all participants but one (98.5%) found the quality of the written instructions to be either good or very good. The last participant chose to respond “bad” to that question. Regarding the ease of execution of the COVID‐VIRO® sample collection (nasal sampling), 100% of the participants found it easy or very easy (16.2% and 83.8%, respectively). Likewise, COVID‐VIRO ALL IN® self‐test procedures were considered easy or very easy to perform by nearly all the participants (25.0% easy and 73.5% very easy). Although every participant had the opportunity of requesting assistance from the trained professional supervising the test, one participant found them hard to execute. Interestingly, it appeared that it was the same patient that was not satisfied with the written instructions. Then, this supervisor had to rate the execution quality of the test procedures by the participant. Only 2/68 (2.9%) of the subjects were considered as having poorly executed the test procedures whereas 66/68 (97.1%) were rated as good or very good: 21 (30.9%) and 45 (66.2%), respectively. The most frequent observation was that some test users were repositioning the protective cover on top of the test after sample collection, despite instructions asking to throw it away, making it impossible to carry out the next steps properly. Among all participants who performed the interpretation exercise, 21 sorted a negative test, 4 an invalid test, and 43 a positive test. Overall, none of the 68 patients misinterpreted the test. Only one patient found the reading and interpreting steps difficult, the majority considering them to be either easy (14.7%) or very easy (83.8%).

Substudy 2—COVID‐VIRO ALL IN® and COVID‐VIRO® comparison on child and teenager populations

A total of 24 children (from 3 to 11 years old) and 16 teenagers (from 12 to 15 years old), with parents from different occupational categories, participated in this comparison study between COVID‐VIRO ALL IN® and COVID‐VIRO® usage and interpretation. None of them was excluded. All were followed by a supervisor who had to rate the execution quality of the test procedures, and none of them had any problem with the realisation of the test except one 10‐years‐old participant.

Regarding the quality of both self‐test instructions, nearly all participants considered the quality of the written instructions to be either good or very good. In the 3−11‐years‐old population, most children found the COVID‐VIRO ALL IN® instructions to be very good (20/24, 83.3%), a slightly lower result was observed for COVID‐VIRO® instructions (17/24, 70.8%). In the teenager population COVID‐VIRO ALL IN® instructions seem as easier to understand (13/16 very good, 81.3%) compare to COVID‐VIRO® instructions (11/16 very good, 68.8%). One 14‐years‐old participant considered the COVID‐VIRO® instructions poorly written, but not COVID‐VIRO ALL IN® instructions.

Similarly, all participants but one found both the COVID‐VIRO® and COVID‐VIRO ALL IN® sample collection rather easy or very easy. Following this, 7 out of 30 total participants (23.3%) found that the COVID‐VIRO® nasal swab sampling was uncomfortable, whereas only 2/30 (6.7%) felt the same way for COVID‐VIRO ALL IN® nasal sample collection.

Altogether, both populations found the procedures for both COVID‐VIRO® and COVID‐VIRO ALL IN® easy or very easy to perform. Only two teenagers found the COVID‐VIRO® self‐test difficult to execute, and another one felt the same way about the COVID‐VIRO ALL IN® self‐test.

Interpretation of results

Just like the adult population, child and teenager populations were randomly given a contrived test to interpret the results (both a COVID‐VIRO® and a COVID‐VIRO ALL IN® self‐test). Overall, there was only one child participant who misunderstood the results on one test. Otherwise, no interpretation error was observed for all other participants

Lastly, children and teenagers were asked whether they preferred the COVID‐VIRO® or COVID‐VIRO ALL IN® self‐test. In the child group, 19 participants out of 24 preferred the COVID‐VIRO ALL IN® test (79.2%). Similarly, 14 out of 16 teenagers preferred the COVID‐VIRO ALL IN® test (87.5%). Overall, 33 participants out of the 40 preferred the COVID‐VIRO ALL IN® (82.5%)

Participants expressed that the COVID‐VIRO ALL IN® system was easier to read, simpler to use and more comfortable/less painful.

DISCUSSION

In our previous prospective study on COVID‐VIRO® rapid antigenic test diagnostic performance in real life conditions on nasopharyngeal samples, the performance of the test was very similar to the reference method since the specificity and sensitivity were 100% and 96.88%, respectively, which placed it above the requirements of the French National Authority for Health (HAS) (sensitivity ≥ 80%, specificity ≥ 99%) and the WHO. Our current study showed COVID‐VIRO ALL IN® present almost same performances with 93.0% sensitivity, 100% specificity, and an overall agreement of 97.5% with nasopharyngeal RT‐PCR.

In addition to the performance assessment, we conducted a usability study following the FDA recommendations. The participant was asked to read the test instructions and perform all the procedures while being supervised, to assess whether the participant was able to correctly perform the test on his/her own and interpret it accurately. This time, 68 adults of different age, education level, and socioeconomic background, were included in the study constituting a representative sampling of the French general population. According to them, the quality of written instructions is good enough to be easily understood, proving that the documents provided with the COVID‐VIRO ALL IN® test are accessible for all laypersons.

Moreover, this study showed that the COVID‐VIRO ALL IN® test is very practicable since all participants were able to obtain a valid and interpretable result without requesting the supervisor's assistance.

In terms of satisfaction about COVID‐VIRO ALL IN® usage, all the participants declared that the sample collection or the subsequent testing procedures were easy or very easy to perform, except one. Similarly, only one participant found the reading and the interpretation steps somewhat difficult. All together, these results demonstrate that the COVID‐VIRO ALL IN® test is highly adapted for use by an adult layperson.

Although SARS‐CoV‐2 vaccines are now available, public health measures remain critical to control the pandemic. In the United States, a large‐scale, public health intervention implementing at‐home SARS‐CoV‐2 self‐testing has been launched: it aims at providing insights into association of methods to mitigate viral transmission. The program consists of a combination of complementary approaches: an at‐home testing program associated with a broad communication and community engagement strategy. This kind of approach of frequent at‐home testing could ease the burden of large‐scale facility‐based testing programs and provide communities with an additional course of action to reduce the impact of COVID‐19.

The usability study conducted in children and that compared COVID‐VIRO ALL IN® to COVID‐VIRO® first generation showed that more than 80% of participants preferred the new generation of test because of its greater ease and simplicity of use associated with more comfort. As such, the new COVID‐VIRO ALL IN® self‐test allows a high accurate diagnostic as good as first generation COVID‐VIRO®, but with better usability and participant satisfaction.

Following our previous study, it seems that the usability of a COVID self‐test is improved with COVID‐VIRO ALL IN® for all types of population and especially for children, by combining all components into one easy‐to‐use self‐device, while maintaining high performance of the test. As a RADT, it is a great alternative since it lowers the risks and adverse effects of nasal classic swabs actually used with COVID self‐tests. The sampling device seems indeed well adapted for use by young children under adult supervision. The current French school screening strategy for young children is based on the RT‐PCR test done on saliva. This matrix is known to be less sensitive than the nasopharyngeal sample that remains the current gold standard. In our study, the performance of COVID‐VIRO ALL IN® self‐test were as good as RT‐PCR done on saliva samples. The ALL IN ONE self‐test seems to be a potential additional tool for large testing operations and especially for young children in schools or at home under adult supervision.

A recent study from Colosi et al. showed that weekly screening would reduce the number of cases on average by 24% in elementary and 53% in middle school compared to symptom‐based testing alone. This result confirms the great interest of a massive repeated screening campaigns in school based on self‐testing. COVID‐VIRO ALL IN® could replace or help to complement actual testing strategies (saliva RT‐PCR tests) and give both a reliable and quick answer to a key population (children aged 5−11 years) for which vaccination has just been authorised in France. Vaccination coverage of this population will take several months and it is therefore essential to continue large‐scale screening campaigns in schools and other establishments catering for this young public.

Apart from the current COVID‐19 pandemic, seasonal epidemics of influenza and respiratory syncytial virus (RSV) affect millions of people each year and are associated with important morbidity and mortality in young children population , and elderly population. In 2015, around the world, acute low respiratory infection associated with RSV, led to the hospitalization of 30 million children under 5 years old and caused almost 60 000 deaths at hospitals, mainly in the developing countries. In France, about one third of the paediatric population is affected either by flu and/or RSV during the seasonal outbreak of these diseases. Although COVID‐19 pandemic changed the dynamic of influenza and RSV epidemiology, this remains a public health concern. , The technology of the COVID‐VIRO ALL IN® test could be easily extended to the detection of other respiratory viruses like RSV and seasonal influenza, in a multiplex device allowing the simultaneous detection of several diseases that present similar symptoms. It would allow massive screening especially in vulnerable populations and therefore greatly improve patients care.

AUTHORS CONTRIBUTIONS

Experimental strategy design: Thierry Prazuck, Raphael Serreau, and Nino Guy Cassuto. Experiments: Anne Gravier, Mathilda Colin, Aurelie Theillay, Daniela Pires‐Roteira, and Sandra Pallay. Data Curation: Thierry Prazuck. Manuscript writing: Thierry Prazuck, Laurent Hocqueloux, and Raphael Serreau. Manuscript editing: Thierry Prazuck and Nino Guy Cassuto. The authors alone are responsible for the content and the writing of the paper.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.