Assessment of potential factors associated with the sensitivity and specificity of Sofia Influenza A+B Fluorescent Immunoassay in an ambulatory care setting.

BACKGROUND: Seasonal influenza leads to an increase in outpatient clinic visits. Timely, accurate, and affordable testing could facilitate improved treatment outcomes. Rapid influenza diagnostic tests (RIDTs) provide results in as little as 15 minutes and are relatively inexpensive, but have reduced sensitivity when compared to RT-PCR. The contributions of multiple factors related to test performance are not well defined for ambulatory care settings. We assessed clinical and laboratory factors that may affect the sensitivity and specificity of Sofia Influenza A+B Fluorescence Immunoassay. STUDY
DESIGN: We performed a post-hoc assessment of surveillance data amassed over seven years from five primary care clinics. We analyzed 4,475 paired RIDT and RT-PCR results from specimens collected from patients presenting with respiratory symptoms and examined eleven potential factors with additional sub-categories that could affect RIDT sensitivity.
RESULTS: In an unadjusted analysis, greater sensitivity was associated with the presence of an influenza-like illness (ILI), no other virus detected, no seasonal influenza vaccination, younger age, lower cycle threshold value, fewer days since illness onset, nasal discharge, stuffy nose, and fever. After adjustment, presence of an ILI, younger age, fewer days from onset, no co-detection, and presence of a nasal discharge maintained significance.
CONCLUSION: Clinical and laboratory factors may affect RIDT sensitivity. Identifying potential factors during point-of-care testing could aid clinicians in appropriately interpreting negative influenza RIDT results.

Introduction

Seasonal influenza poses a significant annual disease burden [1-3] and is common in outpatient clinical settings. Although clinicians often diagnose possible influenza based on patient history and symptoms, studies indicate symptoms alone perform inadequately for influenza [4,5]. Rapid influenza diagnostic tests (RIDTs) have been shown to significantly improve the accuracy of physicians’ estimates of influenza during peak influenza season [6]. RIDTs allow laboratory confirmation of clinically-suspected influenza cases within a timeframe that is clinically meaningful in primary care and urgent care settings and therapeutically meaningful for prompt initiation of antiviral therapy and avoidance of inappropriate antibiotic prescribing [7].

Point-of-care RIDTs are easy to use, relatively inexpensive, and provide results in as little as 15 minutes. Most RIDTs are highly specific (>95%), but they exhibit varying and often low sensitivity [8-10] when compared to reverse transcription–polymerase chain reaction (RT-PCR). Thus, the Center for Disease Control and Prevention (CDC) advises caution when interpreting negative test results [11].

Clinicians are rarely provided adequate training on the performance characteristics of RIDTs and approaches for an informed interpretation of results. Accordingly, falsely negative results can lead to missed opportunities or errors in treatment and patient education; falsely positive results may lead to inappropriate treatment.

Understanding which factors influence sensitivity may enable clinicians to better select appropriate patients for testing and improve their ability to interpret RIDT results. Pragmatic RIDT operating characteristics, however, have not been well defined in primary care settings. Studies that attempt to identify potential factors often limit the number of assessed variables. Several studies examined only two potential factors [6,12-14]. Age and virus strain were assessed most frequently [6,12-19]. Other factors included timing of illness onset relative to testing and illness severity [16-18], specimen collection method [19], and viral load or influenza RT-PCR cycle threshold (Ct) value [16,20-22]. To our knowledge, no study has examined the effect of within-season influenza vaccination status on sensitivity, or combined multiple clinically-relevant factors simultaneously.

We performed a post-hoc assessment of a large surveillance dataset to evaluate multiple clinical and laboratory factors that may affect RIDT sensitivity using seven years of data amassed from five primary care clinics that sequentially employed Sofia® Influenza A+B Immunoassay (Sofia-FIA; Quidel Corporation) for point-of-care influenza testing. We focused primarily on factors that clinicians could take into consideration during a clinical encounter. Our a priori hypotheses were that sensitivity declined with increasing age and with increasing time from illness onset. We were also interested in the combined roles of sex, influenza vaccination status, influenza type, meeting influenza-like illness (ILI) criteria, and influenza RT-PCR Ct value.

Methods

The University of Wisconsin Health Sciences Minimal Risk Institutional Review Board deemed protocols exempt and classified the project as clinical care and public health surveillance. Thus, patients were not required to sign a consent form to be enrolled in the surveillance program.

Setting

Clinicians—including physicians, resident physicians, nurse practitioners, and physician assistants—at five primary care clinics in southcentral Wisconsin collected respiratory specimens on patients presenting with acute respiratory infections (ARIs) between October 26, 2012 and June 30, 2019. Four of the sites are University of Wisconsin Department of Family Medicine and Community Health residency training clinics. The fifth non-residency community clinic was incorporated into the surveillance program prior to the 2014–2015 influenza season. The clinics are located in two urban, one suburban, and two rural communities and serve diverse populations. All clinics were enrolled in the Influenza Incidence Surveillance Project (IISP), later renamed the Optional Influenza Surveillance Enhancement (OISE) program. IISP/OISE monitors medically attended influenza-like illness (ILI) and estimates the incidence of influenza [1,2]. The Centers for Disease Control and Prevention initiated IISP in 2009 and the program continues to operate year around. This platform—as implemented by the Wisconsin study team—provided an opportunity for the pragmatic evaluation of one RIDT within the context of real-life clinical practice where numerous clinicians, at various stages of their training and careers, were engaged to identify suitable ARI patients and collect surveillance data and respiratory specimens.

Population

Patients of all ages were eligible for inclusion if the clinician identified the presence of an ARI and the patient had at least two acute respiratory tract symptoms (nasal discharge, nasal congestion, sore throat, cough, fever) that began within seven days of their clinic visit. For patients aged ≥2 years, the IISP definition of ILI was fever with cough and/or sore throat [23]. For patients aged <2 years, ILI was defined as fever with ≥1 respiratory symptom(s).

Procedures

Surveillance program staff provided a brief initial training to all clinicians and trained all incoming family medicine residents. Clinicians collected extensive demographic, epidemiologic, and symptom data (Table 1) on each patient along with paired respiratory specimens: (a) anterior nasal specimen using a nasal swab (Pur-Wraps®) and (b) either a nasopharyngeal (NP) or a high oropharyngeal (OP) specimen using a flocked swab (Copan®). The anterior nasal swab was returned to its paper sheath and immediately transferred to the on-site clinical laboratory. Sofia-FIA was performed on the anterior nasal swab specimen at the time of the patient visit, thus allowing for clinical decision-making. Clinic laboratory technicians followed laboratory-approved procedures for Sofia-FIA, as detailed in the package insert [24]. The NP/OP swab was immediately placed into a labeled 3 ml viral transport medium (Remel MicroTest™ M4RT®) tube, kept at 4–8° Celsius, and shipped with a requisition form to the Wisconsin State Lab of Hygiene (WSLH) via courier, usually within 24 hours of collection. The WSLH requisition form contained the Sofia-FIA results, patient demographic information, and clinical information, including number of days from illness onset, symptoms, and whether patients received an influenza vaccine prior to their illness. Surveillance staff confirmed vaccine status through the Wisconsin Immunization Registry [25].

Table 1

Routinely collected data for patients with acute respiratory infection selected for influenza surveillance at 5 primary care clinics.

Data Element	Description
Epidemiological
Time from symptom onset	Number of days from illness onset to home visit
Exposure to similar illness	Exposure to similar illness 1–3 days prior to illness onset
Demographic
Age	years
Sex	male / female / other
Race	Standard categories
Ethnicity	Standard categories
Clinical
Illness Severity	Mild, moderate, severe as recorded by clinician
Measured temperature	Temperature taken by clinic staff
Presence of symptoms	Presence/absence of 17 symptoms; other recorded
Use of antipyretics in last 6 hours	Reported by patient
Recent influenza antiviral use	Reported by patient
Receipt of current seasonal influenza vaccine	Reported by patient (verified through registry)

Diagnostics

Clinics transitioned from the Quidel QuickVue® Influenza A+B to Sofia-FIA between October 2012 and April 2013. Both RIDTs are CLIA-waived rapid antigen tests [24]. This change was prompted by a CDC request that Wisconsin clinics serve as a testing site for automated transfer of daily results, as made possible by the wireless feature of Sofia-FIA [26]. The Sofia-FIA and QuickVue platforms have similar specificities, but Sofia-FIA demonstrates superior sensitivity due to immunofluorescence-based lateral-flow technology [27]. The Sofia package insert cites nasal swab sensitivity and specificity as 90% and 95% for influenza A and 89% and 96% for influenza B, respectively [24].

We used the In-vitro Diagnostic (IVD) CDC Human Influenza Virus RT-PCR Panel as our comparison standard [28]. This panel allowed for identification of influenza type and subtype. Cycle threshold values were reported for all specimens. All confirmatory testing was performed at the WSLH. In addition, a respiratory pathogen panel (RPP)—identifying 17 viral targets—was performed on all specimens [29].

Data analysis

Categorical characteristics were described by counts (%) and continuous characteristics were described by mean (sd). The overall sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated along with Agresti-Coull confidence intervals. Sensitivity and specificity were described when stratified by each categorical variable.

Unadjusted associations with sensitivity and specificity were analyzed using chi-square tests for categorical variables and Mann-Whitney-Wilcoxon tests for numerical variables. Adjusted models were fit for predicting improvements in sensitivity and specificity based on participant gender, presence of an ILI, days from symptom onset, severity (mild, moderate, or severe), vaccination status, co-detection of other pathogens (RPP), season (early, peak, or late), age, presence/absence of each of a number of symptoms, and an interaction term of age and days from onset. Quadratic terms for age were considered but only included if they significantly improved their model based on likelihood ratio tests. Subjects with unknown gender or vaccination were removed from adjusted models.

The adjusted odds ratios, 95% confidence intervals, and their corresponding p-values were reported. Significance was assessed at the alpha = 0.05 level. No corrections were made to unadjusted or adjusted p-values to control for inflated Type 1 error rate. Binomial logistic regression models were used to predict true positives (sensitivity) and true negatives (specificity) from corresponding RT-PCR samples. Data from 1,126 RT-PCR-positive subjects with no missing data were used in the adjusted sensitivity model, and 3,190 RT-PCR-negative subjects in the adjusted specificity model. Statistical analysis was performed with R version 4.02.

Results

A total of 5,989 respiratory specimens were collected from October 26, 2012 through June 30, 2019. We excluded 1,514 specimens due to one or more of the following criteria: (1) Sofia RIDT was not performed (n = 1,259 RIDT results were obtained with QuickVue); (2) specimen collected >7 days after illness onset (n = 222); and (3) RT-PCR was not performed (n = 33). We analyzed 4,475 paired specimens, of which one pair was missing symptom information so negatives were imputed for the symptoms. A detailed description of how samples were selected for data analysis is provided in Fig 1.

Fig 1

The demographics of the surveillance population were reflective of general primary care populations with a broad range in patient ages (0.0–98.8 years) and a majority of female patients (Table 2). The majority of patients evaluated for ARI met the ILI criteria (57.4%) and presented for care an average of 3.47 days after symptom onset. Patients presented most commonly with cough (83.7%), nasal discharge (75.6%), sore throat (63.4%), and fever (60.6%). The influenza vaccination rate of 46.2% was slightly above state and national averages [30].

Table 2

Demographics and distribution of sample characteristics of 4,475 primary care patients presenting with acute respiratory infections and selected for influenza surveillance.

Characteristic	Total, n (%)
Total specimens	4,475
Female	2,687 (60.0)
Influenza-like Illness (ILI)	2,567 (57.4)
Vaccinated against influenza	2,068 (46.2)
Days from Onset (mean ± SD)	3.47 ± 1.79
Age
Mean ± SD	34.9 ± 21.5
Median [range]	35.08 [.03–98.8]
Clinic
Belleville (rural)Northeast (urban)Oregon (rural)Verona (suburban)Wingra (urban)	837 (18.7)1,025 (22.9)360 (8.0)1,034 (23.1)1,219 (27.2)
Severity (as recorded by clinician)
MildModerateSevere	1,314 (29.4)2,843 (63.5)236 (5.3)
Season
Early (July-Nov)Peak (Dec-Feb)Late (March-June)	877 (19.6)2,224 (49.7)1,374 (30.7)
Symptoms
ChillsCough	2,301 (51.4)3,746 (83.7)
Fever	2,712 (60.6)
HeadacheMalaiseMyalgiaNasal Congestion	2,353 (52.6)2,456 (54.9)1,918 (42.9)2,750 (61.5)
Runny Nose	3,381 (75.6)
Sore Throat	2,837 (63.4)
PCR Results a
Influenza A H1Influenza A H3Influenza A (other)Influenza B	318 (7.1)553 (12.4)7 (0.2)293 (6.5)

Both influenza A(H1) and A(H3) were detected in two samples.

PCR results

Of the 4,475 specimens collected and analyzed, 1,169 (26.1%) were positive by RT-PCR for influenza (Fig 1). Influenza A(H3) was identified in 553 specimens (47.3% of the positives, 12.4% overall prevalence), influenza A(H1) was identified in 318 specimens (27.2% of the positives, 7.1% overall prevalence), and influenza B was identified in 293 specimens (25% of the positives, 6.5% overall prevalence). Unknown or unsubtypeable influenza A strains were detected in 7 specimens. Cycle threshold values were not available for 141 influenza results, and ranged from 14.25 to 37.79.

RIDT performance characteristics

Sofia-FIA detected the presence of influenza in 874 specimens, 774 of which were confirmed influenza-positive by RT-PCR (PPV 88.6%). Overall sensitivity of Sofia-FIA for influenza A was 66.2% (95% confidence interval: 63.0–69.3), with a specificity of 97.9% (97.4–98.3). For influenza B, sensitivity was also 66.2% (60.6–71.4) while the specificity was 97.4% (96.8–97.8). A summary of Sofia-FIA performance statistics can be found in Table 3.

Table 3

Summary performance statistics for Sofia® Influenza A + B fluorescent immunoassay.

	Overall	Influenza A	Influenza B
Sensitivity (95% CI)	66.2 (63.4–68.9)	66.2 (63.0–69.3)	66.2 (60.6–71.4)
Specificity (95% CI)	96.2 (95.5–96.8)	97.9 (97.4–98.3)	97.4 (96.8–97.8)
PPV (95% CI)	86.1 (83.7–88.2)	88.5 (85.8–90.7)	63.7 (58.1–68.9)
NPV (95% CI)	89.0 (87.9–89.9	92.2 (91.3–93.0)	97.6 (97.1–98.0)

Specificity by type was calculated by including all individuals who did not test positive for a given virus as ‘True Negative’.

Clinical and laboratory predictors

In an unadjusted analysis, greater sensitivity was associated with the following factors: illness meeting ILI criteria, no non-influenza virus co-detection (RPP), no seasonal influenza vaccination, younger age, lower Ct value, fewer days since illness onset, and presence of nasal discharge, nasal congestion, and fever (p < .05, Table 4A and 4B). Factors not significantly associated with sensitivity were sex, influenza type, illness severity, and seasonality (early, peak, and late influenza season) and all other recorded symptoms. None of the factors were significantly associated with specificity.

Table 4

a: Unadjusted analyses of the effects of clinical and laboratory factors on Sofia® Influenza A + B fluorescent immunoassay sensitivity as compared to RT-PCR.

Subjects were excluded from analysis if necessary data was missing (1 from sex analysis, 24 from severity analysis). Significant results indicated with an asterisk. b: Unadjusted analysis of the effect of age and Ct value and days from illness onset to specimen collection on sensitivity of Sofia® Influenza A + B fluorescent immunoassay. Significant results indicated with an asterisk.

Characteristic	True Positive	Positive	Sensitivity, n (95% CI)	p-value
MaleFemale	350424	518650	67.6 (63.4–71.5)65.2 (61.5–68.8)	0.401
ILINo ILI	601173	863306	69.6 (66.5–72.6)56.5 (50.9–62.0)	< 0.001*
Influenza Vaccine				0.022*
VaccinatedUnvaccinated	307456	490660	62.7 (58.3–66.8)69.1 (65.5–72.5)
Severity (as recorded by clinician)				0.973
MildModerateSevere	19751053	29976779	65.9 (60.3–71.0)66.5 (63.1–69.7)67.1 (56.1–76.5)
Season				0.403
Early (July-Nov)	14	26	53.8 (35.5–71.3)
Peak (Dec-Feb)Late (March-June)	486274	731412	66.5 (63.0–69.8)66.5 (61.8–70.9)
Symptom
ChillsCough	482723	7291092	66.1 (62.6–69.5)66.2 (63.4–69.0)	0.9310.996
Fever	615	885	69.5 (66.4–72.4)	< 0.001*
HeadacheMalaiseMyalgiaNasal Congestion	449463378506	668690579737	67.2 (63.6–70.7)67.1 (63.5–70.5)65.3 (61.3–69.1)68.7 (65.2–71.9)	0.4010.4400.5080.021*
Runny Nose	636	923	68.9 (65.8–71.8)	< 0.001*
Sore Throat	494	733	67.4 (63.9–70.7)	0.267
Co-detected virusesSingle virus detection	32741	601108	53.3 (40.9–65.4)66.9 (64.1–69.6)	0.031*

After adjustment, illness meeting ILI criteria, younger age, fewer days from onset, no co-detection, and presence of a nasal discharge maintained significance. Sensitivity was significantly improved by including a quadratic term for age (LRT p = 0.010). Adjusted odds ratios, confidence intervals, and p-values can be found in Fig 2.

Fig 2

Odds ratios with 95% confidence intervals for clinical and laboratory factors used in a sensitivity model based on a referent sample from a female patient with an ILI, moderate severity, no flu vaccination, no non-influenza virus co-detection, during peak flu season, and with no symptoms.

Age estimates are per 5 years and centered on the median age for this set (36.1 years). Estimates for days from onset are per 1 additional day. Red triangles depict factors that lower sensitivity. Green triangles depict factors that increase sensitivity.

Again, no factors were significantly associated with specificity in the adjusted analysis.

Discussion

We found that increased sensitivity of Sofia-RIDT was associated with four clinical factors that are readily identifiable at the time of a medical evaluation (presence of an ILI, younger age, fewer days from illness onset, and presence of nasal discharge), and one additional factor that would not be discernable by a clinician (no co-detection of additional viruses in a respiratory pathogen panel). All of these factors are likely to increase influenza antigenic load in the anterior nares. No factors significantly affected specificity in either unadjusted or adjusted analyses.

Compared with other studies, we had a considerably larger sample size over several consecutive years that included seven sequential influenza seasons. In a 2012 meta-analysis of 159 studies assessing RIDT accuracy, the average sample size was 131 confirmed cases of influenza [31]. Only two of the studies referenced had over 1,000 influenza positive samples, and all samples were collected during a single influenza season, possibly limiting generalizability [32,33]. In addition to incorporating multiple influenza seasons that differed in timing, intensity, and predominant types/subtypes, this study introduced additional variability by including dozens of clinicians who had received the level of training for respiratory specimen collection typical in primary care settings. Accordingly, the results from this pragmatic assessment are more likely to represent performance of Sofia-FIA in real-life settings.

Many studies have noted the effect of age on RIDT sensitivity [6,12-19], but few have considered other clinical factors that could contribute to RIDT performance. More commonly, studies examine laboratory factors such as Ct value and virus type and subtype, which are not generally available during a clinical session. Our study assessed simultaneously eight potential clinical factors (ILI status, sex, age, illness severity, common respiratory symptoms, seasonality, influenza vaccination status, and days from illness onset) and three laboratory factors (Ct value, influenza type, and detection of another virus).

The effect of Ct value, sex, age, and time from illness onset on RIDT performance are consistent with previous findings within the literature [17,34]. Seasonality is not well defined, but studies indicate that RIDTs are most useful when community prevalence of influenza is high because positive predictive value is greatest at that time [10]. We are unaware of any studies that assess the effect of vaccination status and individual symptoms.

Few studies have examined the relationship between sensitivity and the presence/absence of an ILI. During a performance assessment of QuickVue, Koul et al. reported no difference in sensitivity for patients with an ILI or a severe acute respiratory illness (SARI) [16]. ILI is well defined in the literature, but how severity is measured for sensitivity analysis varies greatly. SARI was defined as those who have ILI (fever accompanied by cough and/or sore throat) and are hospitalized. Another study that used hospitalization as a marker for severity found that sensitivity was especially poor among hospitalized adults (45%) compared with outpatient adults (75%), with a similar mean time from illness onset (2.7 days and 2.1 days, respectively) [35]. Hospitalized children had a higher sensitivity (84%), which was likely due to the higher viral loads commonly found in younger populations. The higher sensitivity among children may be due to a higher viral load in younger populations. In our study, we did not implement an age cutoff, and sensitivity was greater for those who had an ILI, but it was not influenced by severity. Our definition of severity was based on a clinician-reported three-point scale and may be subject to bias. A more uniform definition of severity may be needed.

As is commonly found in the scientific literature, Sofia-FIA did not perform as well as expected in real-world clinical settings. The Sofia package insert cites nasal swab sensitivity and specificity as 90% and 95% for influenza A and 89% and 96% for influenza B, respectively [24]. In contrast, we found an overall sensitivity for influenza A of 66.2% (95% confidence interval: 63.0–69.3), with a specificity of 97.9% (97.4–98.3). For influenza B, sensitivity was also 66.2% (60.6–71.4) while the specificity was 97.4% (96.8–97.8). Most studies have indicated lower sensitivity for influenza B, but our study showed 66.2% sensitivity for both influenza A and B [19,22,36]. Furthermore, our study found that detecting another virus could decrease sensitivity. Although research in this area is limited, a study comparing four RIDTs saw no cross reactivity in samples with co-infections [37].

This analysis had several limitations. First, we only analyzed the performance of one RIDT, but many of our findings are compatible to those of studies of other RIDTs. Second, our surveillance program encompasses a limited geographical area in Southcentral Wisconsin, and we defined seasonality based on the temperate climate. The distribution of influenza types and subtypes, however, are similar to distributions reported nationally in any given year. For the purpose of generalizability, we defined seasonality as early (July-November), peak (December-February), and late (March-June) for each year analyzed, but influenza outbreaks occur at varying times of the year, and this may have had an effect on the analysis. Third, by utilizing multiple clinicians to identify potential patients, collect clinical data, and obtain respiratory specimens, much more variability is introduced than would occur in other research settings. This study provides results obtained in real-world primary care settings in which patient selection, specimen collection, and testing occurred within routine clinical activities. Although this is an asset in determining whether RIDTs are appropriate for point-of-care diagnosis and treatment, we cannot guarantee rigor in all data elements to the same degree as a more traditionally controlled research study or randomized clinical trial. Moreover, variability in patient selection and specimen collection may contribute to lower estimates of sensitivity. Finally, despite the large sample size, there may not have been sufficient power to identify factors that may affect specificity. Their effects, however, would be trivial given the overall high specificity.

Several clinical and laboratory factors in this study appear to affect RIDT sensitivity. Awareness of these factors and their identification and consideration by clinicians at the point-of-care may aid in patient selection and the appropriate interpretation of negative influenza RIDT results. If a clinician suspects that a patient has influenza, but an RIDT test is negative, the clinician may be able to take age, symptoms, and days from illness onset into consideration when assessing whether or not the results are accurate.

Due to the ongoing SARS-CoV-2 pandemic, clinicians are relying more on diagnostic tests than symptom assessment, and nucleic acid amplification testing (NAAT) has become more widely available in multiple areas of the world. NAAT tests are sensitive and specific, but they can be costly and it can take hours to days to receive results. RIDTs, in contrast, are relatively inexpensive and produce results in the time that it takes a clinician to assess and treat patients. Thus, an RIDT can be used as an initial tool during point-of-care with the caveat that some factors my influence sensitivity and NAAT confirmation may be necessary.

Additional performance characteristic studies analyzing potential factors affecting sensitivity are necessary to explain broad ranges sensitivity across RIDT platforms and to establish standard guidelines for clinical interpretation of influenza RIDT results.

9 Nov 2021

Assessment of potential factors associated with the sensitivity and specificity of Sofia Influenza A+B Fluorescent Immunoassay in ambulatory care

PLOS ONE

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Marianne Clemence

Associate Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide the full names of the five primary care clinics in southcentral Wisconsin.

3. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

4. Thank you for stating the following in the Competing Interests section:

"JLT has received past research funding from Quidel Corporation: https://www.quidel.com/. Quidel has provided Sofia analyzers and Influenza A+B FIA tests to the Wisconsin surveillance team. Quidel did not play a role in study design, data collection and analysis, decision to publish, or preparation of the manuscript"

We note that you received funding from a commercial source: Quidel Corporation

Please provide an amended Competing Interests Statement that explicitly states this commercial funder, along with any other relevant declarations relating to employment, consultancy, patents, products in development, marketed products, etc.

Within this Competing Interests Statement, please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your amended Competing Interests Statement within your cover letter. We will change the online submission form on your behalf.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The submitted manuscript of Bell et al. assesses potential clinical and laboratory factors which are associated with the sensitivity and specificity of Sofia Influenza A+B fluorescent immunoassay. Those immune assays are easy to use and provide results within a short time frame. As pointed out and statistically tested by Bell et. al. caution is required since the sensitivity is extremely low. They found 4 out of 11 tested factors that can influence the results and should be monitored in future. All of these factors are likely to increase antigenic load. Although the results reach statistical significance for those 4 factors, the sensitivity does not increase higher than 70%. In conclusion, those four factors are helpful for future diagnosis nevertheless those factors are no game changer nor is it known how these results could be implemented to increase the accuracy in near future. The following minor concerns might improve the quality of the manuscript and strengthen their findings

- Flow diagram of sample selection would be very helpful

- What does “a quadratic term of age” mean?

- One would expect increased sensitivity in severe cases!? (higher AG load!?!)

- Since ct and “days from illness onset” are comprehensible to reach statistical significance, why does the little mean age difference (32.4 / 39.3) reached statistical difference in your analysis?

Reviewer #2: The strength of this paper is the testing over multiple years and the number of tests performed. The identified risk factors for a positive RAT are not surprising, but it is important to publish these data.

The paper is well-written, but a number of questions arises, which I hope the authors are able to clarify.

In general, children and adults are normale separated as younger people are more likely to have symptoms such as fever and to be positive for multiple respiratory pathogens at a time, whereas elderly people often doesn't get fever and only are positive for one pathogen at a time. Please consider that you may be comparing two different patient populations in your statistical analysis, and should perform the analysis separately for children and adults.

On page 9 top. The IFU sensitivity and specificity is reported and the authors calculate a clinical sensitivity and specificity (RIDT performance on page 12), but does not include these data in the discussion. Please reflect on your findings compared to the reported values in the IFU in the discussion, as these data are interesting.

On page 10 second paragraph. Data from 1126 RT-PCR positive is included together with 3190 RT-PCR negative or in total 4316 individuals, but in the next paragraph (results), 4475 paired specimens are included. Is this difference due to multiple testing and if so, then please report the number of repeated testing to allow the reader to understand the math of calculations.

On page 12 PCR results: 1169 were positive by RT-PCR (how is this possible when 1126 were RT-PCR positive) and 1171 are reported positive in table 2, so are two individuals dual-positives?

On page 13 Table 4: Please define severity in the manuscript, what is mild/moderate/severe? and please define the season, what is early/peak and late, does this change from year to year or do you use fixed periods of time?

In the discussion. Please comment on the role of RAT, when bearing in mind that NAAT based platforms e.g. Liat, ID now, biofire or QIAstat are now commonly available in multiple areas of the world.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Uffe Vest Schneider

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

24 Dec 2021

JOURNAL REQUIREMENTS

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

I have formatted the manuscript per the guidelines listed in the links you provided above.

2. Please provide the full names of the five primary care clinics in southcentral Wisconsin.

• UW Health Belleville Family Medicine

• UW Health Verona Clinic

• UW Health Oregon Clinic

• UW Health Northeast Family Medical Center

• UW Health Wingra Family Medical Center

3. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

We did not receive additional funding for this analysis, but I have updated the “Funding Information” with the grant award number: MSN256387. I have also deleted the “Financial Disclosure” in the manuscript, per your online instructions: https://journals.plos.org/plosone/s/submission-guidelines#loc-references

4. Thank you for stating the following in the Competing Interests section:

"JLT has received past research funding from Quidel Corporation: https://www.quidel.com/. Quidel has provided Sofia analyzers and Influenza A+B FIA tests to the Wisconsin surveillance team. Quidel did not play a role in study design, data collection and analysis, decision to publish, or preparation of the manuscript"

We note that you received funding from a commercial source: Quidel Corporation

Please provide an amended Competing Interests Statement that explicitly states this commercial funder, along with any other relevant declarations relating to employment, consultancy, patents, products in development, marketed products, etc.

Within this Competing Interests Statement, please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your amended Competing Interests Statement within your cover letter. We will change the online submission form on your behalf.

I amended the Competing Interest Statement in the manuscript to explicitly state that Quidel Corporation provided “in-kind” materials. I also stated that this does not alter our ability to adhere to all PLOS ONE policies on sharing data and materials. The statement now reads as follows:

JLT has received past research funding from Quidel Corporation. Quidel provided in-kind Sofia analyzers and Influenza A+B FIA tests to the Wisconsin surveillance team. This does not alter our adherence to PLOS ONE policies on sharing data and materials. Quidel did not direct or exert any influence over study design, data collection and analysis, decision to publish, or preparation of the manuscript.

REVIEWER COMMENTS

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Thank you for reviewing our manuscript. We are pleased to know that you feel our manuscript is technically sound and that our conclusions are supported by the data.

________________________________________

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

The statistical analysis was performed with the oversight of a PhD-level biostatistician.

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Dataset can be accessed at the following data repository: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/JNC1XJ

________________________________________

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

We are pleased that the quality of writing in our manuscript meets your standards.

________________________________________

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1

The submitted manuscript of Bell et al. assesses potential clinical and laboratory factors which are associated with the sensitivity and specificity of Sofia Influenza A+B fluorescent immunoassay. Those immune assays are easy to use and provide results within a short time frame. As pointed out and statistically tested by Bell et. al. caution is required since the sensitivity is extremely low. They found 4 out of 11 tested factors that can influence the results and should be monitored in future. All of these factors are likely to increase antigenic load. Although the results reach statistical significance for those 4 factors, the sensitivity does not increase higher than 70%. In conclusion, those four factors are helpful for future diagnosis nevertheless those factors are no game changer nor is it known how these results could be implemented to increase the accuracy in near future. The following minor concerns might improve the quality of the manuscript and strengthen their findings

- Flow diagram of sample selection would be very helpful

We have provided a flow chart to demonstrates how the samples were selected for analysis.

- What does “a quadratic term of age” mean?

Quadratic term of age is age-squared and implies that the fit is better with a non-linear relationship. This is fairly established in the literature.

- One would expect increased sensitivity in severe cases!? (higher AG load!?!)

This is not what we found in our analysis. How severity is defined may be a factor. In other studies, severe cases are equated with hospitalization. In our case, clinicians report severity based on a 3-point scale. Both definitions may be subject to bias, ours is perhaps even more so. We did, however, see a relationship between sensitivity and the presence of an influenza-like illness (ILI). The definition of ILI is based on whether a patient has specific symptoms. A yes or no to the presence of a symptom may be a better measure than severity in this case. We had added the following information and an additional citation to clarify that the definition of severity varies within the literature.

“ILI is well defined in the literature, but how severity is measured for sensitivity analysis varies greatly. SARI was defined as those who have ILI (fever accompanied by cough and/or sore throat) and are hospitalized. Another study that used hospitalization as a marker for severity found that sensitivity was especially poor among hospitalized adults (45%) compared with outpatient adults (75%), with a similar mean time from illness onset (2.7 days and 2.1 days, respectively).[35] Hospitalized children had a higher sensitivity (84%), which was likely due to the higher viral loads commonly found in younger populations. In our study, we did not implement an age cutoff, and sensitivity was greater for those who had an ILI, but it was not influenced by severity. Our definition of severity was based on a clinician-reported three-point scale and may be subject to bias. A more uniform definition of severity may be needed.

- Since ct and “days from illness onset” are comprehensible to reach statistical significance, why does the little mean age difference (32.4 / 39.3) reached statistical difference in your analysis?

The mean age is consistent with what we would expect to see in primary care clinics. Cycle threshold, days from illness onset, and age were all significant when unadjusted. Age was still significant in the adjusted analysis.

Reviewer #2

The strength of this paper is the testing over multiple years and the number of tests performed. The identified risk factors for a positive RAT are not surprising, but it is important to publish these data.

The paper is well-written, but a number of questions arises, which I hope the authors are able to clarify.

In general, children and adults are normale separated as younger people are more likely to have symptoms such as fever and to be positive for multiple respiratory pathogens at a time, whereas elderly people often doesn't get fever and only are positive for one pathogen at a time. Please consider that you may be comparing two different patient populations in your statistical analysis, and should perform the analysis separately for children and adults.

It is common in the literature to divide patient population into pediatric and adult cases, but we decided against implementing an age cutoff and instead accounted for age in the multivariant analysis.

On page 9 top. The IFU sensitivity and specificity is reported and the authors calculate a clinical sensitivity and specificity (RIDT performance on page 12), but does not include these data in the discussion. Please reflect on your findings compared to the reported values in the IFU in the discussion, as these data are interesting.

We have added the following paragraph to the Discussion section:

As is commonly found in the scientific literature, Sofia-FIA did not perform as well as expected in real-world clinical settings. The Sofia package insert cites nasal swab sensitivity and specificity as 90% and 95% for influenza A and 89% and 96% for influenza B, respectively.(24) In contrast, we found an overall sensitivity for influenza A of 66.2% (95% confidence interval: 63.0—69.3), with a specificity of 97.9% (97.4—98.3). For influenza B, sensitivity was also 66.2% (60.6—71.4) while the specificity was 97.4% (96.8—97.8).

On page 10 second paragraph. Data from 1126 RT-PCR positive is included together with 3190 RT-PCR negative or in total 4316 individuals, but in the next paragraph (results), 4475 paired specimens are included. Is this difference due to multiple testing and if so, then please report the number of repeated testing to allow the reader to understand the math of calculations.

On page 12 PCR results: 1169 were positive by RT-PCR (how is this possible when 1126 were RT-PCR positive) and 1171 are reported positive in table 2, so are two individuals dual-positives?

We apologize for the confusion. To help clarify how we selected samples for analysis, we have created a flow chart (Fig 1) and added a note to table two about co-detections.

On page 13 Table 4: Please define severity in the manuscript, what is mild/moderate/severe? and please define the season, what is early/peak and late, does this change from year to year or do you use fixed periods of time?

We defined season in Table 4 as Early (July-November), Peak (December-February), Late (March-June) so that it matches Table 2 and added “as reported by clinician” to Table 2 and Table 4 to match Table 1. As stated in the discussion, seasonality is not well defined in the literature. We added the additional sentence below to the limitations:

For the purpose of generalizability, we defined seasonality as early (July-November), peak (December-February), and late (March-June) for each year analyzed, but influenza outbreaks occur at varying times of the year and this may have had an effect on the analysis.

In the discussion. Please comment on the role of RAT, when bearing in mind that NAAT based platforms e.g. Liat, ID now, biofire or QIAstat are now commonly available in multiple areas of the world.

The following paragraph has been added to the discussion section:

Due to the ongoing SARS-CoV-2 pandemic, clinicians are relying more on diagnostic tests than symptom assessment, and nucleic acid amplification testing (NAAT) has become more widely available in multiple areas of the world. NAAT tests are sensitive and specific, but can be costly and it can take hours to days to receive results. RIDTS, in contrast, are relatively inexpensive and produce results in the time that it takes a clinician to assess and treat patients. Thus, an RIDT can be used as an initial tool during point-of-care with the caveat that some factors my influence sensitivity and NAAT confirmation may be necessary.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

27 Apr 2022

Assessment of potential factors associated with the sensitivity and specificity of Sofia Influenza A+B Fluorescent Immunoassay in an ambulatory care setting

PONE-D-21-09174R1

Dear Dr. Bell,

Thank you for submitting your manuscript to PLOS ONE; I sincerely apologise for the unusually delayed review timeframe. We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Emily Chenette

Editor in Chief

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In their recently revised manuscript, Bell et al. have managed to largely address the concerns and suggestions that I raised during the initial round of review.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

1 May 2022

PONE-D-21-09174R1

Assessment of potential factors associated with the sensitivity and specificity of Sofia Influenza A+B Fluorescent Immunoassay in an ambulatory care setting

Dear Dr. Bell:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr Emily Chenette

Staff Editor

PLOS ONE