Influenza vaccine effectiveness estimates in the Dutch population from 2003 to 2014: The test-negative design case-control study with different control groups.

Information about influenza vaccine effectiveness (IVE) is important for vaccine strain selection and immunization policy decisions. The test-negative design (TND) case-control study is commonly used to obtain IVE estimates. However, the definition of the control patients may influence IVE estimates. We have conducted a TND study using the Dutch Sentinel Practices of NIVEL Primary Care Database which includes data from patients who consulted the General Practitioner (GP) for an episode of acute influenza-like illness (ILI) or acute respiratory infection (ARI) with known influenza vaccination status. Cases were patients tested positive for influenza virus. Controls were grouped into those who tested (1) negative for influenza virus (all influenza negative), (2) negative for influenza virus, but positive for respiratory syncytial virus, rhinovirus or enterovirus (non-influenza virus positive), and (3) negative for these four viruses (pan-negative). We estimated the IVE over all epidemic seasons from 2003/2004 through 2013/2014, pooled IVE for influenza vaccine partial/full matched and mismatched seasons and the individual seasons using generalized linear mixed-effect and multiple logistic regression models. The overall IVE adjusted for age, GP ILI/ARI diagnosis, chronic disease and respiratory allergy was 35% (95% CI: 15-48), 64% (95% CI: 49-75) and 21% (95% CI: -1 to 39) for all influenza negative, non-influenza virus positive and pan-negative controls, respectively. In both the main and subgroup analyses IVE estimates were the highest using non-influenza virus positive controls, likely due to limiting inclusion of controls without laboratory-confirmation of a virus causing the respiratory disease.

Introduction

The most effective way to prevent influenza virus infection and (severe) illness is by vaccination [1]. However, the composition of the influenza vaccine should be reconsidered annually, and eventually updated, due to amino acid substitutions causing antigenic drifts of the hemagglutinin and neuraminidase virus surface proteins which occurs continually over time to escape neutralization by the immune response [2], [3]. Despite the yearly update, the ability of the vaccine to prevent influenza virus infection in the general population during an influenza season (vaccine effectiveness [VE]) varies each year [4]. Hence, VE information is important for immunization policy decision makers, e.g. to decide which type of vaccine should be used (i.e. inactivated or live attenuated virus, with or without adjuvant) and who should be immunized (e.g. health care workers, children, elderly) [5]. However, it is not possible to determine the VE before an influenza season. Therefore, retrospective studies using observational data are performed to estimate the VE annually [4], [6].

The test-negative design (TND) case-control study is a commonly used study design to estimate influenza VE (IVE). In this study design, patients seeking medical care for influenza-like illness (ILI) are tested for influenza virus infection [7]. The IVE is determined by comparing the prevalence of influenza vaccination between ILI patients who tested positive for influenza [cases] and those who tested negative for influenza [controls] [7], [8]. As both cases and controls are selected from patients seeking medical care for ILI, the study design is assumed to minimize confounding by health care-seeking behavior or functional status compared to other types of observational studies [9], [10], [11]. Moreover, laboratory tests are used to define the influenza outcome which, compared to other study designs using non-specific influenza outcomes (e.g. ILI symptoms), reduces misclassification bias [9], [10], [11].

Several studies have shown that the definition of the control group in TND studies may influence the estimates of the IVE [12], [13], [14], [15], [16]. Three types of control groups have been used in TND studies: (1) all ILI patients tested negative for influenza virus infection (all influenza negative), (2) ILI patients tested negative for influenza virus but positive for another respiratory virus (non-influenza virus positive), and (3) ILI patients tested negative for both influenza virus and other respiratory viruses (pan-negative) [11], [12], [13], [14], [15], [16], [17], [18]. Although all influenza negative controls are commonly used, in several studies non-influenza virus positive controls have been used arguing that if another respiratory virus than influenza virus could be detected in the control group, the presence of misclassification is highly unlikely, as there is a confirmed infectious cause of ILI in both cases and controls. This is based on the fact that the same laboratory tests for influenza virus are used for both cases and controls [13], [15], [16]. On the other hand, other investigators argued that the presence of a non-influenza respiratory virus infection could be partly explained by the association between influenza vaccination and the increased risk of another respiratory virus infection due to a temporary nonspecific immune response [10], [11], [12], [18], [19]. Consequently, the definition of the second control group could lead to selection bias and thereby an overestimation of IVE since the risk of ILI symptoms caused by another pathogen would be higher in the vaccinated patients than in unvaccinated patients, resulting in a higher proportion of vaccinated individuals in the control group [11], [12], [14], [17], [18]. As a consequence, several studies have used pan-negative controls.

The aim of the present study is to estimate the IVE over ten influenza epidemic seasons in The Netherlands (from 2003/2004 to 2013/2014) using the three most commonly applied definitions of TND control groups and evaluate the differences among the IVE estimates.

Methods

Study database

We used data from the Sentinel Practices of NIVEL Primary Care Database [20], [21]. Sampling of patients with ILI or another acute respiratory infection (ARI) for laboratory diagnostics started in 1992. Since 2003 participating general practitioners (GPs) are asked to take nose and throat swabs from two ILI patients each week. Since 2005/2006 with the additional instruction to sample preferably one patient less than 10 years of age. If no ILI patients are encountered, the GP is asked to swab patients with another ARI instead [22]. The official standard definition of ILI was used in the GP offices to diagnose a patient with ILI, namely an acute onset of symptoms (full development of typical symptoms in ≤4 days) including a rectal temperature of at least 38 °C and at least one respiratory or systemic symptom (i.e. cough, nasal catarrh, sore throat, frontal headache, retrosternal pain, myalgia) [21]. ARI is defined as an acute respiratory illness other than ILI, such as acute sinusitis or pneumonia, and with at least one of the following symptoms; coughing, rhinorrhea or sore throat [23]. Both ILI and ARI patients were included in this study to maximize the power. Patient information is registered on the sample form, e.g. personal information (gender, age), date of symptoms onset and swabbing, use of antiviral medication and underlying medical conditions. The surveillance study has been registered in the Personal Data Protection Act Register of the Dutch Personal Data Protection Commission [No. RIVM/EPI-043]. No further ethical approval was needed since only anonymized data was used for the current study.

Laboratory testing

Collected samples from all swabbed subjects were sent to the National Institute for Public Health and the Environment (RIVM) for laboratory tests for a number of pathogens. These pathogens were identified using virus isolation and/or reverse transcription polymerase chain reaction (RT-PCR). RT-PCR changed over time from conventional block-based to real-time format with necessary adjustments in primer and probe design. Laboratory tests for the respiratory viruses influenza virus, respiratory syncytial viruses (RSV), rhinovirus (RV) and enterovirus (EV) were performed throughout the study period from 2003 to 2014. Laboratory tests for other pathogens differed per season: the identification of parainfluenza virus (PIV) type 1–4, coronavirus (CoV) (229E, OC43 and NL63) and metapneumovirus (hMPV) stopped after the 2007/2008 influenza season and adenovirus (ADV) was tested only from 2005 until the 2007/2008 season. We used information on these other pathogens for sensitivity analyses only.

Selection of cases and controls

For each influenza season from 2003/2004 through 2013/2014 patients were selected when they were swabbed between week 48 and week 14 of the following year. Patients were excluded if (1) the vaccination status was unknown, (2) time between symptoms onset and swabbing was more than seven days, (3) a patient had received antiviral medication within the two weeks prior to the GP visit, (4) the date of swabbing was before the first of December of each season to make sure vaccination was given 14 days before symptoms onset, or (5) data was missing on other variables (i.e. gender, age, ILI/ARI diagnosis, underlying chronic disease and respiratory allergy) [7], [24]. Patients swabbed in the season 2009/2010 were excluded since this was an atypical (pandemic) influenza season. Eligible swabbed patients who tested positive for influenza virus A(H1N1), A(H1N1)pdm09, A(H3N2) or B were regarded as cases. Controls were defined as those patients tested (1) negative for influenza virus (all influenza negative) (2) negative for influenza virus, but positive for RSV, RV or EV (non-influenza virus positive), and (3) negative for these four respiratory viruses (pan-negative). We included RSV, RV and EV since only these viruses were tested throughout the whole study period.

Statistical analysis

Chi-square tests were used to test for significant differences in proportions of categorical covariates, and T-tests for differences in mean age between cases and control groups. A P-value <0.05 was considered statistically significant.

IVE was calculated by IVE = (1 − OR) × 100% with influenza vaccine status as the exposure [7]. The unadjusted and adjusted IVE for potential confounders were estimated, i.e. age, ILI/ARI diagnosis, respiratory allergy, underlying chronic disease (e.g. asthma, chronic obstructive pulmonary diseases, diabetes mellitus and cardiovascular diseases), influenza season and level of vaccine match. Variables that were associated with the outcome (changed the OR > 5%) were retained in the final generalized linear mixed-effect model (GLMM) or multiple logistic regression model. When the 95% confidence interval (95% CI) did not contain zero or negative values, the IVE was considered significant [25].

The GLMM in which influenza seasons are modelled as a random effect was used to estimate the IVE over all seasons and by seasons categorized by level of vaccine match [26]. Vaccine match status was categorized as (partial) match or mismatch based on the circulating influenza viruses and seasonal influenza vaccines used which information was extracted from data published by the Dutch National Influenza Center [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37]. The multiple logistic regression model was used to assess the IVEs for the individual seasons. Statistical analyses were conducted using SAS software (version 9.4) and R (version 3.2.0) [38].

Sensitivity analysis

We performed several sensitivity analysis. We estimated (1) the overall IVE excluding seasons 2003/2004 and 2004/2005 since those two seasons had a low number of subjects in the control group, especially the non-influenza virus positive controls, (2) the overall IVE including the pandemic season of 2009/2010 were the vaccination status was based on receiving the seasonal influenza vaccine only, (3) the overall IVE using the patients which were swabbed within 4 days after symptom onset and (4) we calculated the IVE for the influenza seasons 2005/2006, 2006/2007 and 2007/2008 using laboratory test results of PIV virus 1–4, CoV, RSV, hMPV, RV, EV and ADV. Influenza virus negative patients testing positive for at least one, or negative for all the viruses were considered as non-influenza virus positive and pan-negative controls, respectively. This sensitivity analysis was conducted since the definition of the control groups is similar to previously conducted studies [12], [13], [14], [15], [16].

Results

Subject Characteristics

GPs of the Dutch Sentinel Practices network swabbed a total of 11,199 patients from 2003 through 2014. From these, 4051 (36%) fulfilled the in- and exclusion criteria for this study (Table 1 ). The majority of subjects were excluded because they were swabbed outside the influenza season (4153, 37%). Other subjects were excluded since information on e.g. age, clinical diagnosis or influenza vaccination was missing (928, 8%), the time between symptom onset and swabbing was more than seven days (1444, 13%) or they had received antiviral medication within two weeks prior to the GP visit (62, 1%). From the included subjects, a total of 1297 (32%) patients tested positive for influenza virus (cases) and 2754 (68%) tested negative for influenza virus. Among those patients testing negative for influenza virus, 676 (25%) tested positive for RSV, RV or EV (non-influenza virus positive) and 2078 (75%) tested negative for these viruses (pan-negative). Statistical significant differences in age, GP ILI/ARI diagnosis, presence of any chronic disease and influenza vaccination status were found between cases and the different control groups. Compared to the control groups, cases were younger, more likely to be diagnosed with ILI, and had a lower proportion of any chronic disease and influenza vaccination. In addition, there were statistically significant differences in age and diagnosis when comparing the non-influenza virus positive and pan-negative controls; pan-negative controls were older and were more likely to be diagnosed with ILI compared to non-influenza virus positive controls (see Table 1).

Table 1

Characteristics of cases and the three different control groups: all influenza negative (Control group 1), non-influenza virus positive (Control group 2) and pan-negative (Control group 3).

	Cases (n = 1297)	Controls (n = 2754)
		Control group 1 (n = 2754)	Control group 2 (n = 676)	Control group 3 (n = 2078)
Gender
Female	668 (51.5%)	1470 (53.4%)	339 (50.1%)	1131 (54.4%)
Male	629 (48.5%)	1284 (46.6%)	337 (49.9%)	947 (45.6%)
		P1 = 0.280	P1 = 0.600	P1 = 0.105
				P2 = 0.058
Age	0–83 (Mean: 31.8)	0–93 (Mean: 34.4)	0–93 (Mean: 27.0)	0–91 (Mean: 36.8)
0–4 years	115 (8.9%)	397 (14.4%)	213 (31.5%)	184 (8.9%)
5–14 years	242 (18.7%)	282 (10.2%)	78 (11.5%)	204 (9.8%)
15–59 years	816 (62.9%)	1637 (59.4%)	291 (43.0%)	1346 (64.8%)
≥60 years	124 (9.5%)	438 (15.9%)	94 (13.9%)	344 (16.6%)
		P1 < 0.001	P1 < 0.001	P1 < 0.001
				P2 < 0.001
Diagnosis
ARI	261 (20.0%)	1237 (44.9%)	335 (49.6%)	902 (43.4%)
ILI	1036 (80.0%)	1517 (55.1%)	341 (50.4%)	1176 (56.6%)
		P1 < 0.001	P1 < 0.001	P1 < 0.001
				P2 = 0.006
Time between symptom onset and swab date
<3 days	479 (36.9%)	950 (34.5%)	263 (38.9%)	687 (33.1%)
3–5 days	711 (54.8%)	1430 (51.9%)	335 (49.6%)	1095 (52.7%)
6–7 days	107 (8.3%)	374 (13.6%)	78 (11.5%)	296 (14,2%)
		P1 < 0.001	P1 = 0.011	P1 < 0.001
				P2 = 0.044
Any chronic disease	77 (5.9%)	286 (10.4%)	65 (9.6%)	221 (10.6%)
		P1 < 0.001	P1 = 0.004	P1 < 0.001
				P2 = 0.495
Respiratory allergy	107 (8.2%)	221 (8.0%)	62 (9.2%)	159 (7.7%)
		P1 = 0.855	P1 = 0.542	P1 = 0.574
				P2 = 0.237
Influenza vaccination	171 (13.2%)	579 (21.0%)	142 (21.0%)	437 (21.0%)
		P1 < 0.001	P1 < 0.001	P1 < 0.001
				P2 = 1.000
Seasons
Matcha	442 (34.1%)	775 (28.1%)	164 (24.3%)	611 (29.4%)
Partially matchb	191 (14.7%)	491 (17.8%)	134 (19.8%)	357 (17.2%)
Mismatchc	664 (51.2%)	1488 (54.0%)	378 (55.9%)	1110 (53.4%)

P1: Comparison cases versus controls. P2: Comparison Control group 2 versus Control group 3.

Seasons: 2008–2009; 2010–2011.

Seasons: 2005–2006; 2006–2007.

Seasons: 2003–2004; 2004–2005; 2007–2008; 2011–2012; 2012–2013; 2013–2014.

Determination of confounding factors

When comparing the unadjusted IVE estimates with IVE estimates adjusted for possible confounding factors, age, ILI/ARI diagnosis, chronic disease, respiratory allergy and the influenza season changed the OR by more than 5%. Therefore, in the following paragraphs only IVE estimates adjusted for these confounding factors are shown (influenza season parameter not included in estimating season-specific IVE).

Overall IVE estimate

The adjusted IVE estimate over all seasons was 35% (95% CI 15–48) when using all influenza negative controls (Fig. 1 ). When using non-influenza virus positive controls the IVE increased to 64% (95% 49–75) whereas the IVE decreased to 21% (95% CI −1 to 39) when using the pan-negative controls.

Fig. 1

Adjusted1 influenza vaccine effectiveness (IVE) against laboratory-confirmed influenza in The Netherlands for all seasons combined and for the (mis)matched seasons2. Significant IVE indicated in bold. Control group 1 = all influenza negative controls; Control group 2 = non-influenza virus positive controls; Control group 3 = pan-negative controls. 1Adjusted for age, ILI/ARI diagnosis, chronic disease, respiratory allergy and influenza season. 2(Partially) matched seasons include seasons 2005–2006, 2006–2007, 2008–2009 and 2010–2011. Mismatched seasons include all other seasons from the study period.

(Mis)matched seasons

From 2003 to 2014, the vaccine strains and circulating viruses (partially) matched in the seasons 2005/2006, 2006/2007, 2008/2009 and 2010/2011 (Table 2 ) [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37]. The pooled adjusted IVE estimates for these (partially) matched seasons were 39% (95% CI 16–56), 62% (95% CI 40–76) and 30% (95% CI 2–50) for all influenza negative, non-influenza virus positive and pan-negative controls, respectively (Fig. 1). The pooled adjusted IVE estimates for the mismatched seasons were 31% (95% CI 7–49), 66% (95% CI 47–78) and 14% (95% CI −18 to 37) for Control group 1–3, respectively.

Table 2

Proportion of virus (sub)types and lineages (%) and vaccine mismatch per subtype/lineage based on virus isolates and specimens submitted to the National Influenza Center with a specimen collection date in week 40 of one year through week 39 of the following year.

Influenza season
Virus (sub)type	2003/04	2004/05	2005/06	2006/07	2007/08	2008/09	2009/2010	2010/11	2011/12	2012/13	2013/2014
Proportion of virus (sub) types (%)
A	99	80	64	99	49	92	100	60	90	69	94
A(H1N1)a	0	18	4	13	86	1	100	97	1	39	40
A(H3N2)	100	82	96	87	14	99	0	3	99	61	60
B	1	20	36	1	51	8	0	40	10	31	6
B	Y	Y	Y/V	Y	Y	V	NA	Y/V	Y/V	Y/V	Y/V
B/Vic			91					95	12	6	24
B/Yam			9					5	88	94	76
Mismatch per subtype
A(H1N1)	NA	No	No	Yes	Yes	No	Yesb	No	No	No	No
A(H3N2)	Yes	Yes	No	No	No	No	NA	No	Yes	Yes	Yes
B/Vic	NA	NA	Yes (Yam in vaccine)	NA	NA	Yes (Yam in vaccine)	NA	No	No	NA	NA
B/Yam	Yes (Vic in vaccine)	Yes (Antigenic mismatch)	Yes (Antigenic mismatch)	Yes (Vic in vaccine)	Yes (Vic in vaccine)	NA	NA	Yes (Antigenic mismatch)	Yes (Vic in vaccine)	Yes (Antigenic mismatch)	Yes (Antigenic mismatch)
Vaccine (mis)matchc	Mismatch	Mismatch	Partially match	Partially match	Mismatch	Match	Mismatch	Match	Mismatch	Mismatch	Mismatch

V: B/Victoria/2/87-lineage; Y: B/Yamagata/16/88-lineage; NA: Not applicable.

Bold percentages indicate the predominant viruses per season. An influenza virus was considered as predominant when detected in a proportion of ≥60% among the total influenza virus detections in week 40 through week 39 of the following year.

2003/2004 through 2008/2009 season former seasonal A(H1N1); 2009/2010 through 2013/2014 season A(H1N1)pdm09.

Former seasonal A(H1N1) in the vaccine; A(H1N1)pdm09 monovalent vaccine was available to too late in season.

The influenza vaccine was considered to be a match if at least one of the two following criteria was fulfilled: (1) all the vaccine components were antigenically similar to the circulating A subtypes (H1N1 or H1N1pdm09 and H3N2) and B lineages (Victoria or Yamagata); (2) vaccine stain antigenically matched the predominant and one of the non-predominant circulating virus subtypes. The influenza vaccine was considered to partially matched if the vaccine strain matched the predominant virus subtype but mismatched the non-predominant subtypes. In all other situations the vaccine was considered mismatched with circulating viruses.

IVE estimates individual influenza seasons

Adjusted IVE estimates for the different individual influenza seasons varied from negative IVE estimates to 95% (Fig. 2 ). Significant IVE estimates were identified for the 2007/2008, 2010/2011 and 2012/2013 seasons irrespective of the control group used. Except for season 2013/2014 when using all influenza negative, no significant IVE estimates were identified for any of the other seasons. In all seasons, with the exception of season 2005/2006, the estimated adjusted IVE was the highest for non-influenza virus positive controls, followed by all influenza negative and pan-negative controls.

Fig. 2

Adjusted1 influenza vaccine effectiveness (IVE) against laboratory-confirmed influenza in The Netherlands for the individual seasons2. Significant IVE indicated in bold. Control group 1 = all influenza negative controls; Control group 2 = non-influenza virus positive controls; Control group 3 = pan-negative controls. 1Adjusted for age, ILI/ARI diagnosis, chronic disease and respiratory allergy. 2VE could not be estimated for the seasons 2003/2004 and 2004/2005 due to the small sample size.

Sensitivity analysis

The results of the sensitivity analysis are summarized in Table 3 . The adjusted overall IVE estimates increased when excluding seasons 2003/2004 and 2004/2005. The pooled overall IVE estimate was 42% (95% CI 25–55), 66% (95% CI 51–76) and 30% (95% CI 8–46) for all influenza negative, non-influenza virus positive and pan-negative controls, respectively. When including the pandemic season of 2009/2010 the adjusted IVE estimates are comparable with the main analysis. On the other hand the adjusted IVE estimates decreased when restricting to subjects which were swabbed within 4 days after disease onset. For the sensitivity analysis using the laboratory test results of PIV virus 1–4, CoV, RSV, hMPV, RV, EV and ADV to define the cases and controls, 343 eligible patients tested positive for influenza virus (cases) and 794 tested negative (all influenza negative). Among those tested negative for influenza virus 288 (36%) and 506 (64%) patients were included as non-influenza virus positive and pan-negative controls respectively (Table 4 ). The adjusted IVE estimates for the 2005/2006, 2006/2007 and 2007/2008 seasons with the different control groups varied from negative estimates to 92% (Table 3). The adjusted IVE was only statistically significant for the 2007/2008 season when using the different control groups. For all sensitivity analysis the IVE estimate was the highest when using non-influenza virus positive controls, followed by all influenza negative controls and pan-negative controls.

Table 3

Adjusted vaccine effectiveness estimates against laboratory-confirmed influenza sensitivity analysis.

Sensitivity analysis	Adjusted IVE (%) (95% CI)a
	Control group 1	Control group 2	Control group 3
Overall IVE excluding seasons 2003/2004 and 2004/2005	42 (25–55)	66 (51–76)	30 (8–46)
Overall IVE time between disease onset and swab ≤ 4 days	34 (13–49)	59 (40–72)	22 (−2 to 41)
Overall IVE including pandemic season 2009/2010 b	33 (15–47)	63 (48–74)	20 (−3 to 37)
IVE using several laboratory test resultsc to define control group
Season 2005/2006	24 (−69 to 67)	4 (−155 to 63)	44 (−38 to 78)
Season 2006/2007	−8 (−158 to 56)	39 (−81 to 80)	−30 (−234 to 51)
Season 2007/2008	82 (55–94)	92 (78–98)	73 (28–91)

Control group 1 = all influenza negative controls; Control group 2 = non-influenza virus positive controls; Control group 3 = pan-negative controls.

Significant IVE indicated in bold.

Adjusted for age, ILI/ARI diagnosis, chronic disease and respiratory allergy, and for the overall estimates for influenza season.

Vaccination status based on seasonal vaccine only.

PIV virus 1 to 4, CoV, RSV, hMPV, RV, EV and ADV.

Table 4

Characteristics of cases and three different control groups sensitivity analysis: all influenza negative (Control group 1), non-influenza virus positive (Control group 2) and pan-negative (Control group 3).

	Cases (n = 343)	Controls (n = 794)
		Control group 1 (n = 794)	Control group 2 (n = 288)	Control group 3 (n = 506)
Gender
Female	180 (52.5%)	410 (51.6%)	145 (50.3%)	265 (52.4%)
Male	163 (47.5%)	384 (48.4%)	143 (49.7%)	241 (47.6%)
		P1 = 0.845	P1 = 0.650	P1 = 1.00
				P2 = 0.635
Age	0–83 (Mean: 31.4)	0–93 (Mean: 34.3)	0–93 (Mean: 30.8)	0–89 (Mean: 36.3)
0–4 years	30 (8.7%)	101 (12.7%)	64 (22.2%)	37 (7.3%)
5–14 years	65 (19.0%)	95 (12.0%)	40 (13.9%)	55 (10.9%)
15–59 years	218 (63.6%)	480 (60.5%)	138 (47.9%)	342 (67.6%)
≥60 years	30 (8.7%)	118 (14.9%)	46 (16.0%)	72 (14.2%)
		P1 = 0.028	P1 = 0.765	P1 < 0.001
				P2 = 0.001
Diagnosis
ARI	73 (21.3%)	320 (40.3%)	126 (43.8%)	194 (38.3%)
ILI	270 (78.7%)	474 (59.7%)	162 (56.2%)	312 (61.7%)
		P1 < 0.001	P1 < 0.001	P1 < 0.001
				P2 < 0.001
Time between symptom onset and swab date
<3 days	147 (42.9%)	304 (38.3%)	112 (38.9%)	192 (37.9%)
3–5 days	177 (51.6%)	412 (51.9%)	158 (54.9%)	254 (50.2%)
6–7 days	19 (5.5%)	78 (9.8%)	18 (6.2%)	60 (11.9%)
		P1 = 0.040	P1 = 0.592	P1 = 0.006
				P2 = 0.035
Any chronic disease	21 (6.1%)	94 (11.8%)	42 (14.6%)	52 (10.3%)
		P1 = 0.005	P1 < 0.001	P1 = 0.046
				P2 < 0.001
Respiratory allergy	16 (4.7%)	69 (8.7%)	31 (10.8%)	38 (7.5%)
		P1 = 0.025	P1 = 0.006	P1 = 0.128
				P2 = 0.152
Influenza vaccination	31 (9.0%)	151 (19.0%)	62 (21.5%)	89 (17.6%)
		P1 < 0.001	P1 < 0.001	P1 < 0.001
				P2 = 0.206
Influenza season
2005–2006	103 (30.0%)	221 (27.8%)	87 (30.2%)	134 (26.5%)
2006–2007	88 (25.7%)	265 (33.4%)	101 (35.1%)	164 (32.4%)
2007–2008	152 (44.3%)	308 (38.8%)	100 (34.7%)	208 (41.1%)

P1: Comparison cases versus controls. P2: Comparison Control group 2 versus Control group 3.

Discussion

In general, the point IVE estimates varied for the different definitions of control patients and were generally, but not in each season, the highest when using non-influenza virus positive controls, followed by respectively all influenza negative controls and pan-negative controls which is consistent with other studies [12], [13], [15], [18].

Similar differences in IVE estimates were found in a study performed in Portugal, in which the IVE was estimated for the mismatched 2012/2013 influenza season among ILI patients which were tested on RSV, PIV, hMPV, RV and ADV to define the control group patients [12], [39]. On the other hand, an American study which estimated the IVE over six influenza seasons (2004/2005 through 2009/2010) among children and adults aged 50 years and older found only minor differences in the IVE estimates when using the different control groups based on the detection of the same viruses as the Portugal study and enterovirus [14]. However, other studies performed in Australia which estimated the IVE in children for the matched influenza season of 2008/2009 and 2008–2012 seasons using all influenza negative and non-influenza positive controls, found the highest IVE estimates when using non-influenza positive controls [13], [15]. The IVE estimate for (partial) matched seasons using non-influenza positive controls in our study is higher than the IVE estimate in a Dutch randomized controlled trial (RCT) conducted with patients aged 60 years and older (50% (95% CI: 35–61)) in the matched season of 1991–1992 [40], [41]. Moreover, the IVE estimate for mismatched seasons in our study is also higher than the IVE estimate found in a Cochrane review of RCTs with healthy subjects aged 18–65 years when using non-influenza positive controls (55% (95% CI: 41–66)) [42].

The variation in the IVE estimates using different control groups can be explained by selection bias or misclassification bias [11], [12], [17]. In order to reduce selection bias, several studies suggested including non-influenza ILI patients tested negative for other respiratory viruses as controls (pan-negative controls). This suggestion is based on the assumption that influenza vaccination may increase the risk of a non-influenza respiratory virus infection by a temporary nonspecific immunity (viral interference), resulting in a higher proportion of vaccinated individuals in the control group [11], [12], [13], [14], [17], [18]. In our study we found no indication that influenza vaccination might have increased the risk of non-influenza respiratory virus infection in vaccinated patients. We observed no difference in the vaccination coverage between the non-influenza positive controls and pan-negative controls (coverage in both groups 21.0% (P = 1.00)) [13]. Others have indicated that if a temporary nonspecific immunity is present, this is a short lasting response (around two weeks). Therefore, it is unlikely that this response influences the risk of other non-influenza respiratory viruses throughout the full study period since we have only included swabbed patients who received the vaccine at least 14 days before symptom onset [14], [19], [43]. Although, this explanation is only true when we assume individuals are susceptible to influenza virus infection only at the beginning of the study period and timing of infections is not taken into account [44].

It should especially be questioned which population patients is represented by pan-negative controls since these are ILI/ARI patients tested negative for the evaluated respiratory viruses, i.e. ILI/ARI patients without an infection of one of the tested pathogens. Pan-negative patients might be more susceptible for health care-seeking behavior, and therefore include a higher proportion of ILI/ARI not caused by infection, since patients in this control group were older and the prevalence of any chronic disease was higher compared to the cases and other control groups. Also the delay in swabbing might play a role since the time between symptom onset and swabbing influences the sensitivity of laboratory tests [45], [46]. If the time between swabbing and symptoms onset is greater, the sensitivity of influenza A and B viruses RT-PCR detection decreases, e.g. from 88% for 4 days till 70% for 7 days delay [46]. In our study, 14.2% of the pan-negative patients were swabbed 6–7 days after symptom onset compared to 13.6% and 11.5% of all influenza negative and non-influenza virus positive controls.

The sensitivity of the laboratory tests is also important for the assumption of including patients tested negative for influenza virus but positive for another respiratory virus as control group (non-influenza positive controls). The inclusion of this control group is based on the assumption that laboratory tests are adequate for both cases and controls, thereby eliminating false-negative controls and misclassification bias. However, in the current study different laboratory techniques have been used to detect an infection with one of the tested pathogens. RT-PCR was used throughout the whole study period to detect influenza virus, RSV, RV and EV infection. However, due to innovations in these techniques over the study period, the sensitivity of the assays has increased a little. This might have reduced misclassification in later seasons. Furthermore, in season 2004/2005 only virus isolation was used for the detection of RV and EV, which also was the method used to detect PIV virus infections. Virus isolation has a lower sensitivity than RT-PCR. Due to this sensitivity gap misclassification of patients can still not be ruled out absolutely [46], [47], [48].

Our sensitivity analysis excluding the seasons 2003/2004 and 2004/2005 for the overall estimate showed higher IVE estimates compared to the primary analysis, especially when using all influenza negative and pan-negative controls. No significant difference between the IVE estimates could be observed when using the different control groups, the confidence intervals overlapped more than in the primary analysis. When including the pandemic season the overall estimates were almost identical to the estimates from the primary analysis. However, when restricting to patients who were swabbed within 4 days after disease onset the IVE estimates decreased, especially when using non-influenza positive controls. Also the sensitivity analysis which used several extra respiratory pathogens to define the control groups, showed slightly different IVE estimates compared to the primary analysis which included three respiratory viruses for the definition of control groups. However, no general trend (increase or decrease of the IVE) could be observed when comparing the IVE estimates of the primary and sensitivity analyses which may be explained by the relative infrequent occurrence of these infections in primary care.

We were able to analyze the IVE estimates over several seasons among the Dutch general practice population using the TND study with different control groups, however our study has several limitations. Firstly, a certain level of misclassification might be present in the non-influenza virus positive controls and pan-negative controls due to the differences in sensitivity of the laboratory tests used for the detection of other pathogens, especially for those detected by RT-PCR and those detected by virus isolation [49], [50]. Secondly, due to the limited number of eligible patients both patients diagnosed by the GP with ILI and ARI were included in this study to maximize the power. However, there may be differences in health care-seeking behavior of these two patient groups, which could have resulted in misclassification bias. Moreover, patients in the Sentinel Practices of NIVEL Primary Care Database network are not a systematic sample of ILI/ARI patients in the population since most persons with ILI/ARI will not visit their GP and from those who visit the GP, the GP did not have a strict instruction about which patients to sample (e.g. first patients in the week). In addition, subjects who have been hospitalized with an influenza virus infection through the first aid are not covered by the NIVEL Primary Care database, since those subjects are swabbed in the hospital. Despite including both ILI and ARI patients, the number of eligible patients was limited which resulted in wide confidence intervals and only few statistically significant IVE estimates. Finally, we did do not have information on how the vaccination status per individual patient was exactly collected and this might have resulted in misclassification bias. However, since all inhabitants in the Netherlands are registered at one GP office, we can assume that if the GP has vaccinated the patient that the GP takes this information from the registry data of the patient.

Conclusion

Our study shows differences in the IVE estimates when using three different control groups which are in line with other studies. Pan-negative controls seems less valid because a high proportion of patients in this group likely consult the GP because of non-infectious causes of ILI or ARI symptoms. When using non-influenza positive controls, the IVE estimates are more consistent with previous effectiveness studies and clinical trial data, likely due to limiting controls without an infectious cause of respiratory disease; all controls have a laboratory confirmed infection explaining ILI and ARI symptoms. Nevertheless, when using non-influenza positive controls a higher IVE estimate was found in the primary analysis for mismatched seasons compared to matched seasons which is not as expected, but based on the sensitivity analysis seemed to be caused by the mismatched seasons. Future studies using the TND design should take into account the potential impact of defined control group as well as vaccine match status. Further studies on the study design are needed to confirm our findings and to give a more clear statement about the differences observed between the control groups. Future research on the study design should include more patients to be able to detect statistical significant IVE estimates, especially since adjusting for confounding is needed. This possibly can be done by expanding the GP network, increasing the number of patients a GP has to swab or by pooling data from several countries [51], [52], [53]. In addition, it is recommended for future studies to use if feasible, standardized equal-sensitive tests for pathogen detection throughout the whole study period to minimize misclassification bias, although innovations in diagnostic tests increasing the sensitivity should not be ignored.

Conflict of interest

This study was partly funded by the Dutch Ministry of Health, Welfare and Sport, that had no influence on data collection, study design or writing of this paper. All authors declare no conflict of interest.

Authors contribution

ED, MD, FD, EH and AM contributed to the conception and design of the work. AM, FD, GD and PO collected the data and performed laboratory tests. ED and MD analyzed the data. All authors interpreted information for the work. ED drafted the manuscript. All authors critically reviewed the manuscript for contributing to important intellectual content. All authors approved the final version of the manuscript.