Vaccine eagerness: A new framework to analyse preferences in single profile discrete choice experiments. Application to HPV vaccination decisions among French adolescents.

Background: We previously conducted a single-profile discrete choice experiment to elicit preferences of adolescents around HPV vaccine communication, finding that only half of participants made variable choices (non-uniform respondents) from which preferences were elicited. In this paper we provide a framework to evaluate post-choice certainty information to elicit preferences even among respondents who uniformly accepted (serial demanders) or refused (serial non-demanders) hypothetical vaccination scenarios.
Methods: During an in-class online questionnaire among 1458 French adolescents aged 13-15 years old, we collected certainty levels (0-10) after decisions on nine hypothetical scenarios, including four vaccination attributes: information on vaccine-preventable disease type, on vaccine safety, on potential for indirect protection and on vaccine coverage. We developed a vaccine eagerness scale (ranging from -10 to 10), by combining information on the binary decision (accept vs. refuse the hypothetical vaccine) and the decision certainty level. We used random effects linear regressions to evaluate attributes' impact on vaccine eagerness. Sensitivity analyses were performed taking into account low response quality, assessed as invariant certainty and low response time.
Results: Attributes' impact on decision certainty were similar between serial demanders (N = 659) and non-uniform respondents (N = 711): mentioning a positive benefit-risk balance significantly decreased certainty to accept (coefficient -0.93), while information on 80% coverage in other countries (+0.33) and potential for disease elimination (+0.09) increased it. Among serial non-demanders, significant attribute impacts were observed only after exclusion of low-quality responses (N = 31): a potential for disease elimination (coefficient: +0.24) and 80% coverage in other countries (+0.42) significantly increased certainty of refusing vaccination. Combining decision and certainty into a vaccine eagerness indicator allowed analysing preferences in the full sample, including "hesitant" respondents, who were sensitive to the content of the vaccination profile.
Conclusion: Choice certainty informs on respondents' preferences in single-profile discrete-choice experiments, in particular among those with uniform responses.

Background

The World Health Organisation (WHO) recommends using the principles of social marketing to improve communication on vaccination promotion (Nowak et al., 2015). In this perspective, current research analyses how to measure preferences around vaccination in specific groups, notably using tailored communication (Dempsey & Zimet, 2015). HPV vaccination uptake among adolescents remains low in many countries, including in France where complete vaccine coverage (VC) at 16 years reached 32.7% among girls in 2020 (Santé Publique France, 2021) and is one of the lowest in Europe (Nguyen-Huu et al., 2020). As a comparison, the US VC was 68.5% among girls (same age, same year) (Pingali et al., 2021).

HPV vaccines are mainly accessible through general practitioners in France for adolescents aged 11–14 years while other European countries that have introduced school-based vaccination programmes usually have higher VC (Nguyen-Huu et al., 2020). Factors associated with HPV vaccine uptake have mostly been investigated among parents and healthcare providers and can be classified into environmental (policies, access, lack of recommendation) and individual factors (lack of knowledge, fear of serious side effects, fear of new vaccines, mistrust etc.) (Karafillakis et al., 2019). Few articles have focused on the adolescent’ perspective although adolescents’ involvement in the vaccine decision-making process could reinforce their vaccine confidence (Cadeddu et al., 2021). Communication contents need then to be tailored for adolescents.

To better understand which arguments could best motivate HPV vaccine acceptance among adolescents, we designed a single-profile discrete choice experiment with opt-out and a certainty scale. In a previous paper, we analysed the impact of four choice attributes (type of vaccine-preventable disease, vaccine safety, potential for indirect protection and vaccine coverage) on the theoretical vaccine acceptance of French adolescents (Chyderiotis et al., 2021). Attributes and attribute levels were chosen in line with the current literature on factors influencing vaccine intentions. During the experiment, adolescents had to choose whether, for a series of 9 scenarios, they would accept or refuse a proposed vaccination. These 9 scenarios differed by the combinations of presented attribute levels. Among the half of participants that made variable decisions – i.e., who at times accepted or refused the vaccination profile - we estimated that social conformism statements (i.e. high vaccine coverage in neighbouring countries) strongly increased vaccine acceptance, while some statements on safety – albeit meant to be reassuring - were demotivating. However, about half of adolescents made uniform decisions, always accepting or refusing vaccination, which did not allow inferring preferences (i.e., marginal sensitivities) for each attribute’s level.

Discrete choice experiments (DCEs) are grounded in standard economic theory (i.e., random utility maximization (McFadden, 1973)), and allow analysing trade-offs as well as forecasting demand for hypothetical services (Hanemann, 1984). In standard DCE, respondents are asked to repeatedly choose their preferred option among several alternatives (including opt-out options) described by different characteristics or attributes. Econometric modelling of responses based on multinomial (conditional) logit models allows eliciting preferences and assigning weights of importance to the various characteristics of the alternatives being valued. In single-profile DCEs, participants are asked whether they would accept or refuse a presented hypothetical scenario. This procedure may be more realistic in the case of preventative decisions. Indeed, for vaccination decisions, people are used to choosing whether to get vaccinated, rather than choosing the “best” vaccine among several alternatives. Besides, this type of design allows incorporating inter-individual or contextual factors, such as peer behaviour, thus allowing to analyse social preferences (Godinot et al., 2021). The answers to single profile DCE tasks are analysed using binary (conditional) logit models.

A disadvantage of single profile DCEs compared to other choice formats or conjoint analyses is that there is often less information gathered from choices, which are binary by nature. In single-profile DCEs, some respondents always make the same choice (accept or refuse) across scenarios, which leads to unidentified preferences for those respondents (so-called “uniform respondents”).

Respondents who are certain of their choices are assumed to be more consistent in their choices. Post-choice certainty scales have often been used in the literature to reduce the hypothetical bias, i.e. the gap between the stated (hypothetical) choices and (unobserved) real-life behaviour (Brouwer et al., 2010; Loomis, 2011; Lundhede et al., 2009). Recent research also showed that choice certainty variability may contain important information to identify more engaged and more thoughtful respondents. In particular, researchers have found that higher choice certainty variability was associated with higher response times, choice consistency, and internal validity of responses (Regier et al., 2019).

In this paper, we argue that, in addition to these quality considerations, analyses of choice certainty can reveal preferences of “uniform respondents” in single profile DCEs. We also propose a new concept for DCEs on vaccination, “vaccine eagerness”, which combines information of decision and choice certainty to provide additional (and more subtle) information on preferences for the entire sample.

The contribution of this paper to the literature is threefold. First, we analyse the determinants of serial demanding or serial non-demanding behaviours, whose choices (and preferences) are not used to inform optimal vaccination strategies in standard DCE models. Second, we show how to evaluate preferences of uniform respondents based on choice certainty responses, and how to interpret the results. Third, we develop a vaccine eagerness scale combining information on decision and choice certainty to reveal preferences among the full sample, including uniform and non-uniform respondents.

Methods

Discrete choice experiment

Data source

Data were collected from January 31st to March 13th 2020 in five middle schools located in three French regions (Grand Est, Pays de la Loire, and Auvergne-Rhône-Alpes, schools’ characteristics available in Appendix Table A5). The study population comprise adolescents (girls and boys) enrolled in their last two years of middle school (4e and 3e, typically aged 13–15 years, corresponding to grades 8 and 9 in the US educational system). Participants from five private or public schools completed a self-administered and internet-based questionnaire during class and could refuse or stop their participation at any time. Middle schools were randomly selected and contacted for voluntary participation in regions without previous or ongoing school vaccine campaigns. Data collection was stopped with school closure in March 2020 (following first wave of COVID-19 pandemic) but enough data had been collected at that time in those five schools. The survey was completely anonymous at all stages, and no information on the health of respondents was collected. A good level of information on HPV or vaccines was not necessary to answer the DCE section. Some basic knowledge on viruses were included before the choice tasks, so that every respondent had the same level of information. The 15-min questionnaire included questions on participants’ characteristics and opinions on vaccination as well as the DCE tool.

Table 5

Impact of attributes on vaccine eagerness (range −10 to +10) in the full study sample. Non-representative sample of 1458 adolescents in France, January–March 2020.

Vaccine eagerness	Without individual characteristics		Parsimonious model
	Coefficient	95% CI	Coefficient	95% CI
Attributes
Disease
Respiratory disease	ref		ref
Cancer	0.36*	[0.03,0.69]	0.35*	[0.04,0.67]
Genital warts	−0.12	[-0.42,0.18]	−0.14	[-0.45,0.17]
Safety
No side effect	ref		ref
Scientific surveillance	−0.25*	[-0.50,-0.00]	−0.22	[-0.48,0.04]
Safety other countries	−2.08***	[-2.29,-1.86]	−2.07***	[-2.32,-1.83]
Benefit/risk	−2.18***	[-2.61,-1.75]	−2.20***	[-2.64,-1.76]
Indirect protection
Protects only you	ref		ref
Protects other people	0.32***	[0.15,0.50]	0.30**	[0.10,0.49]
Elimination	0.47***	[0.23,0.71]	0.46***	[0.22,0.70]
Coverage
Not enough	ref		ref
Already a third	0.66**	[0.24,1.08]	0.67**	[0.24,1.09]
Most adolescents	1.16***	[0.85,1.47]	1.17***	[0.86,1.47]
Other countries 80%	1.04***	[0.65,1.42]	1.03***	[0.61,1.46]
Individual characteristics
Middle school			ref
School 1			1.62***	[1.25,1.99]
School 2			−0.26*	[-0.50,-0.02]
School 3			1.64***	[1.35,1.93]
School 4			1.30***	[1.05,1.54]
School 5
In favour of vaccination (binary)
Disagree or I do not know			ref
Agree or strongly agree			3.54***	[2.79,4.29]
Education level of the parents
Inferior or equal to French baccalaureat			ref
Superior to French baccalaureat for at least one parent			−0.11	[-0.49,0.28]
I do not know, non-applicable			−0.56**	[-0.95,-0.16]
Language spoken with parents
Only French			ref
Also another language			−0.77**	[-1.35,-0.20]
‘‘It is useful to get vaccinated”
Disagree			ref
Agree			3.15***	[2.29,4.01]
I do not know			−0.23	[-1.87,1.41]
‘‘Do you find it easy to get trustworthy information on vaccination?”
Difficult			ref
Easy			0.81*	[0.08,1.54]
I do not know			0.61	[-0.26,1.49]

Parsimonious model obtained after adding significant individual characteristic variables at p < 0.02 then using a backward stepwise approach at p < 0.05.

*: p-value<0.05; **: p-value<0.01; ***: p-value<0.001.

The following individual characteristics were collected: age, gender, school, presence of a foreign language spoken at home, education level of the parents in three categories, being in favour of vaccination in general (recoded in “agree or strongly agree” vs “disagree or do not know”), vaccination attitudes (perceived utility of vaccination, protection of others through vaccination, fear of needles, fear of substances in vaccines, perceived ease of finding trustworthy information on vaccination, recoded in “disagree/agree/I don’t know”) and self-reported vaccination status with five vaccines: DTaP-IPV (Tetanus, Diphtheria, Pertussis, Polio), MMR (Measles, Mumps and Rubella), Meningococcal C, HPV, and Hepatitis B. The latter was coded as a score, using the sum of answers (Yes = 3/No = 1/I do not know = 2). The score ranged from 5 (minimum) to 15 (maximum). The score was recoded in three categories: mostly “no” from 5 to 8, mostly “I don’t know” from 9 to 12, and mostly “yes” from 13 to 15.

Attributes and levels

Four attributes were selected after a literature review and interviews of experts (Table 1): 1) the disease against which the vaccine protects (three levels: respiratory disease, cancer in 20 years, genital warts), 2) vaccine safety statements (four levels: no serious side effects (SSE), absence of scientifically confirmed SSE, no increase in risk of SSE in countries with high vaccine coverage (VC), presence of a rare SSE but greater benefit), 3) potential for indirect protection (three levels: the vaccine protects only you, avoids transmission to other people, or helps making the disease disappear), and 4) mention of VC among peers (four levels: not enough adolescents registered for vaccination, already one third, most adolescents, or 80% of teens vaccinated in neighbouring countries). Attributes and levels are further described in Table 1.

Table 1

Attributes and levels of the discrete choice experiment.

Attributes	Levels (labels)	Levels (short definition)
Disease	Respiratory disease	The vaccine can protect against a disease with high fever and breathlessness.
	Cancer	The vaccine can protect against a cancer, which could occur 20 years from now.
	Genital warts	The vaccine can protect against genital warts.
Safety	No side effect	The vaccine does not cause serious side effects.
	Scientific surveillance	The vaccine’s safety has been monitored for more than 10 years worldwide. No serious side effect has been scientifically confirmed.
	Safety other countries	In countries where most adolescents are vaccinated, the risk of a serious side effect that could be due to vaccination has not increased.
	Benefit/risk	The vaccine can only on rare occasions cause a serious side effect, but the benefits from vaccination are much greater than its risk.
Indirect Protection	Protects only you	The vaccine protects only you.
	Protects others	By getting vaccinated, you can avoid transmitting the infection to other persons.
	Elimination	By vaccinating most young people of your age, one can make the disease disappear from the population.
Coverage	Not enough	Not enough adolescents of your school have registered to get vaccinated.
	Already one third	Already one-third of adolescents of your school has registered to get vaccinated.
	Most adolescents	Most adolescents of your school have registered to get vaccinated (80%).
	Other countries 80%	In some countries like England and Portugal, more than 80% of adolescents are vaccinated.

Choice tasks

Participants were asked to imagine the following hypothetical situation: a vaccination campaign, during which free vaccination would be offered by a physician during an individual consultation, will be organised at school in two weeks. Adolescents’ parents have already been informed and asked for consent. Based on a variable set of information provided by the school nurse, participants need to decide whether they would register to get vaccinated during the campaign. They were then asked, after each choice task, to indicate their level of certainty about the choice, on a scale from zero (not at all certain) to ten (absolutely certain).

The choice task was designed as a single-profile DCE format with an opt-out. In total, 36 scenarios were used and split in four randomly assigned blocks, leaving nine scenarios per participant.

More details on the study design and DCE tool are available in (Chyderiotis et al., 2021). This survey was conducted as part of a large interventional research project called PrevHPV, which will evaluate the impact of several interventions, including school-based interventions on HPV vaccine coverage.

Study sample

Among the full sample of 1458 participating adolescents, 711 (49%) made non-uniform decisions, i.e., at least one acceptance and one refusal across scenarios (non-uniform sample), with a mean acceptance of 72%. Participants making uniform decisions for all nine scenarios (e.g., always accepting or always refusing vaccination, uniform sample) represented 51% of the full sample (n = 747, Fig. 1). Participants who always accepted hypothetical vaccination are subsequently labelled “serial demanders” (n = 659) and those who always refused it, “serial non-demanders” (n = 88). Because all questions were mandatory, missing data were extremely rare, due to respondents who stopped the questionnaire before the end.

Fig. 1

Distribution of decisions made in the DCE. Non-representative sample of 1458 adolescents in France, January–March 2020.

Statistical analyses

Objective 1: exploring the determinants of serial demanding and serial non-demanding behaviours

We hypothesised that uniform respondents (serial demanders on one hand and serial non-demanders on the other) and non-uniform respondents could differ in terms of sociodemographic and vaccine attitude characteristics, which we tested using Pearson’s Chi-squared tests. Then, we estimated a multinomial logistic regression (serial demanders and serial non-demanders vs. non-uniform respondents) to estimate adjusted effect sizes of the individual characteristics, including an interaction term between education level of parents and language spoken at home (being a proxy for foreign nationality, as ethnicity information can not be assessed according to French law). We estimated clustered standard errors to account for potential unobserved contextual factors in the five middle schools - creating correlations between errors for each school - (“school effect”).

Objective 2: Using choice certainty to analyse the preferences of uniform respondents

In single-profile DCEs, uniform respondents do not reveal information about their underlying preferences, and do not contribute to the likelihood of the model. We hypothesised that choice certainty does provide some information on preferences (i.e., attributes’ weights in the indirect utility function) and to some extent, allow respondents more amplitude for expressing their preferences than binary choices. Choice certainty scales take into account the level of uncertainty around a hypothetical decision, and may better represent real-life decisions. Using random effects linear regression models with clustered standard errors accounting for school level effects, we evaluated the effect of attributes on the choice certainty (dependent variable) among uniform respondents, distinguishing serial demanders and non-demanders. In line with standard assumptions in the literature (Loomis, 2011; Lundhede et al., 2009; Ready et al., 2010), we assumed that higher choice certainty was associated with higher likelihood to make the same decision in a real-life situation, independently of the choice made (“accept” / “not accept” vaccination).

Objective 3: Development of a vaccine eagerness scale to estimate preferences in the full sample

In order to analyse the attributes’ impact on choice certainty in the full sample (including certainty to refuse), we created a variable applicable to each respondent independently of the decision pattern. The certainty level was transformed into a “vaccine eagerness” scale by reverse coding certainty of refusal (eagerness = −10 to 0) and positive values for certainty of acceptation (eagerness = 0 to +10). Thus, −10 represented a strong and certain refusal of vaccination, +10 represented a strong and certain acceptance, and 0 a high level of hesitancy or uncertainty.

We evaluated the effect of attributes on vaccine eagerness using random effects linear regressions and clustered standard errors to take into account the school effect. Random effect allowed accounting for between individual unobserved heterogeneity in vaccine eagerness.

Sensitivity analyses

We hypothesised that the response quality of participants in the uniform sample could have been lower compared to the non-uniform sample (e.g., due to higher prevalence of no-brainers). We thus evaluated response quality based on two indicators: invariant certainty level across all scenarios and survey completion time. The invariant certainty level across all scenarios was obtained by recoding the mean absolute deviation from the mean certainty into three categories to classify the intra-individual variability in certainty for each participant: invariant certainty (always provide the same number for all ten scenarios), variability inferior or equal to the median, variability superior to the median (Regier et al., 2019).

The survey completion time was available for 1420 of 1458 respondents and was on average 13 min (min: 2; max: 31 min). Short response time was defined as below the 20th percentile (9.38 min in our survey) (Zhang et al., 2014).

We conducted sensitivity analyses for the attributes’ impacts on 1) certainty among serial non-demanders 2) certainty among serial demanders (Appendix Table A.3) vaccine eagerness in the full sample (Appendix Table A.4), by excluding suspected low-quality responses from the sample. In analyses A, we excluded participants with invariant certainty (n = 118 excluded), in analyses B, those with short response time (n = 284), in analyses C, those with both characteristics (n = 55) and in analyses D, those with one or both characteristics (n = 347) (Appendix Table A.1).

Table A.3

Sensitivity analyses for serial demanders

Certainty	Analysis A (n = 599)		Analysis B (n = 543)		Analysis C (n = 634)		Analysis D (n = 508)
	Coef.	95% CI	Coef.	95% CI	Coef.	95% CI	Coef.	95% CI
Disease
Respiratory disease	ref		ref		ref		ref
Cancer	0.00	[-0.11,0.10]	−0.05	[-0.19,0.09]	−0.01	[-0.1,0.09]	−0.05	[-0.21,0.11]
Genital warts	−0.04	[-0.22,0.14]	−0.06	[-0.24,0.11]	−0.04	[-0.2,0.12]	−0.06	[-0.26,0.13]
Safety
No side effect	ref		ref		ref		ref
Scientific surveillance	−0.17	[-0.37,0.04]	−0.13	[-0.34,0.09]	−0.16	[-0.36,0.05]	−0.14	[-0.35,0.08]
Other countries	−0.89	[-1.1,-0.69]	−0.89	[-1.14,-0.64]	−0.85	[-1.07,-0.63]	−0.95	[-1.18,-0.72]
Benefit/risk	−1.03	[-1.31,-0.74]	−1.00	[-1.29,-0.7]	−0.97	[-1.27,-0.67]	−1.07	[-1.35,-0.78]
Indirect Protection
Protects only you	ref		ref		ref		ref
Protects other people	−0.01	[-0.09,0.06]	−0.05	[-0.16,0.07]	−0.01	[-0.09,0.06]	−0.05	[-0.17,0.07]
Elimination	0.10	[-0.02,0.21]	0.09	[-0.06,0.25]	0.09	[-0.03,0.21]	0.10	[-0.06,0.26]
Coverage
Not enough	ref		ref		ref		ref
Already a third	0.31	[0.15,0.47]	0.31	[0.2,0.43]	0.29	[0.14,0.44]	0.34	[0.22,0.46]
Most adolescents	0.46	[0.35,0.58]	0.46	[0.39,0.54]	0.44	[0.33,0.55]	0.49	[0.41,0.58]
Other countries 80%	0.37	[0.23,0.51]	0.36	[0.24,0.48]	0.35	[0.21,0.49]	0.38	[0.26,0.51]

A: excluding participants with invariant certainty; B: excluding participants with low response time; C: excluding participants with both invariant certainty and low response time; D: excluding participants with at least one characteristic. *: p-value<0.05; **: p-value<0.01; ***: p-value<0.001.

Table A.4

Sensitivity analyses for vaccine eagerness in the full sample

Vaccine Eagerness	Analysis A (n = 1340)		Analysis B (n = 1174)		Analysis C (n = 1403)		Analysis D (n = 1111)
	Coef.	95% CI	Coef.	95% CI	Coef.	95% CI	Coef.	95% CI
Disease
Respiratory disease	ref		ref		ref		ref
Cancer	0.34	[-0.05,0.73]	0.33	[-0.04,0.70]	0.34	[-0.04,0.72]	0.33	[-0.04,0.69]
Genital warts	−0.14	[-0.44,0.15]	−0.11	[-0.37,0.14]	−0.14	[-0.44,0.15]	−0.11	[-0.35,0.12]
Safety
No side effect	ref		ref		ref		ref
Scientific surveillance	−0.20	[-0.50,0.09]	−0.30	[-0.63,0.03]	−0.23	[-0.52,0.06]	−0.28	[-0.62,0.07]
Safety other countries	−2.18***	[-2.41,-1.94]	−2.30***	[-2.59,-2.01]	−2.11***	[-2.35,-1.87]	−2.39***	[-2.68,-2.11]
Benefit/risk	−2.29***	[-2.77,-1.81]	−2.30***	[-2.71,-1.89]	−2.22***	[-2.69,-1.76]	−2.39***	[-2.82,-1.95]
Indirect Protection
Protects only you	ref		ref		ref		ref
Protects other people	0.38***	[0.23,0.54]	0.32***	[0.17,0.46]	0.37***	[0.22,0.53]	0.33***	[0.19,0.47]
Elimination	0.50***	[0.24,0.75]	0.52***	[0.35,0.69]	0.49***	[0.25,0.73]	0.53***	[0.34,0.71]
Coverage
Not enough	ref		ref		ref		ref
Already a third	0.72***	[0.34,1.11]	0.74***	[0.33,1.15]	0.71***	[0.32,1.10]	0.76***	[0.36,1.15]
Most adolescents	1.25***	[0.91,1.59]	1.33***	[0.94,1.72]	1.21***	[0.89,1.54]	1.38***	[0.97,1.79]
Other countries 80%	1.16***	[0.84,1.49]	1.17***	[0.81,1.53]	1.14***	[0.8,1.47]	1.20***	[0.85,1.55]

A: excluding participants with invariant certainty; B: excluding participants with low response time; C: excluding participants with both invariant certainty and low response time; D: excluding participants with at least one characteristic. *: p-value<0.05; **: p-value<0.01; ***: p-value<0.001.

Table A.1

Distribution of invariant certainty and short completion time in the sample.

	Acceptable completion time	Short completion time	Completion time missing	Total
Variable certainty	1,082	229	29	1,340
Invariable certainty	54	55	9	118
Total	1,136	284	38	1,458

Ethics

This study is part of the PrevHPV study, conducted under the responsibility of Inserm. It was granted approval by Evaluation Committee of Inserm, the Institutional Review Board (IRB00003888, IORG0003254, FWA00005831) on 10 December 2019. All study participants gave their oral, informed, non opposition to participation, in line with French legal guidelines.

Results

Objective 1: Exploring the determinants of serial demanding and serial non-demanding behaviours

Our sample was comprised of 1,458 respondents, including 53.5% of girls, with a mean age of 13.8 years old. Half of the respondents declared at least one of their parents had a high school diploma, while 30.1% declared not knowing. A quarter declared speaking (also) another language than French with their parents. They were 76.5% in favour of vaccination in general, with 13.9% had no opinion and 7.8% were disagreeing.

Comparison of serial non-demanders and serial demanders to the non-uniform sample

Serial non-demanders showed more frequently the following characteristics: having an unfavourable opinion (or no opinion) about vaccination in general (72.7% vs 28.0% among non-uniform respondents, p < 0.001), declaring they had not received childhood vaccines (18.2% vs 5.4%, p < 0.001) and not agreeing with the statements “it is useful to get vaccinated” (p < 0.001), “getting vaccinated can protect others” (p < 0.001) and “vaccination scares me because of the needle” (p < 0.05). Serial non-demanders declared more frequently not knowing whether it is easy to get trustworthy information on vaccination (44.6 vs 29.5%, p < 0.001), being unaware of their parent's education level (53.4 vs 30.2%, p < 0.001) and speaking another language than French with their parents (39.8 vs 27.4%, p < 0.05) (Appendix Table A.2).

Table A.2

Comparison of Serial demanders, Serial non-demanders and Uniform samples to the Non-Uniform sample.

Respondents characteristics	Total (n = 1458)	Non-uniform (n = 711)	Uniform (n = 747)	Serial non-demanders (n = 88)	Serial demanders (n = 659)
	Column percentage
Individual characteristics
Age
12–13 years old	37.7	36.9	38.6	40.9	38.2
14 years old	47.1	48.7	45.7	42.1	46.1
15–17 years old	15.2	14.5	15.8	17.1	15.6
Gender
Girl	53.5	55.7	51.4	51.1	51.4
Boy	46.5	44.3	48.6	48.9	48.6
Middle school ID				**	***
School 1	8.0	9.4	6.6	18.2	5.0
School 2	26.6	25.2	28.0	15.9	29.6
School 3	10.8	12.2	9.4	22.7	7.6
School 4	28.0	27.4	28.5	17.1	30.1
School 5	26.7	25.7	27.6	26.1	27.8
Education level of the parents				***
Inferior or equal to French baccalaureat	19.2	19.8	18.6	17.1	18.8
Superior to French baccalaureat for at least one parent	50.7	49.9	51.4	29.6	54.3
I don't know, non-applicable	30.1	30.2	30.0	53.4	26.9
Language spoken with parents			*	*	**
Only French	75.3	72.6	77.8	60.2	80.1
Also another language	24.7	27.4	22.2	39.8	19.9
Attitudes towards vaccination
In favour of vaccination (binary)			***	***	***
Disagree or I don't know	23.5	28.0	19.3	72.7	12.1
Agree or strongly agree	76.5	72.0	80.7	27.3	87.9
Overall stated vaccine status			***	***	***
Mostly no	5.7	5.4	6.1	18.2	3.6
Mostly I don't know	72.2	68.0	76.7	69.3	67.8
Mostly yes	22.1	26.6	17.2	12.5	28.5
"It is useful to get vaccinated"				***	***
Disagree	3.2	3.6	3.0	10.2	2.0
Agree	91.9	90.5	93.3	67.1	96.8
I don't know	4.8	6.0	3.8	22.7	1.2
"Getting vaccinated can protect others"				***	**
Disagree	7.2	7.8	6.6	13.6	5.6
Agree	81.9	80.5	83.2	53.4	87.2
I don't know	10.9	11.7	10.2	33.0	7.1
"Vaccination scares me because of the needle"				*
Disagree	60.9	60.3	61.5	50.0	63.1
Agree	34.6	34.9	34.5	39.8	33.7
I don't know	4.4	4.8	4.0	10.2	3.2
"Vaccination scares me because of the substances in the vaccine"					*
Disagree	64.3	62.2	66.4	54.6	67.9
Agree	24.3	26.6	22.1	30.7	21.0
I don't know	11.4	11.2	11.5	14.8	11.1
"Do you find it easy to get trustworthy information on vaccination?"			***	*	***
Difficult	19.6	23.2	16.2	18.1	15.9
Easy	52.4	47.3	57.2	37.4	59.8
I don't know	28.0	29.5	26.6	44.6	24.3

NU: non-uniform sample; U: uniform sample; SND: serial non-demanders; SD: serial demanders; *: p-value compared to NU < 0.05; **: p-value compared to NU < 0.01; ***: p-value compared to NU < 0.001. For the overall stated vaccine status: mostly “no” corresponds to a score from 5 to 8, mostly “I don’t know” from 9 to 12, and mostly “yes” from 13 to 15.

Those in favour of vaccination in general were overrepresented among serial demanders (87.9 vs 72.0% among non-uniform respondents, p < 0.001), as were those agreeing with the statements “it is useful to get vaccinated” (p < 0.001) and “getting vaccinated can protect others” (p < 0.01). Serial demanders were also more likely to find it easy to get trustworthy information on vaccination (59.8 vs 47.3%, p < 0.001) and to not speak another language at home (19.9 vs 27.4%, p < 0.01) (Appendix Table A.2).

Relative risk Ratio of serial demanders and serial non-demanders compared to the non-uniform sample

In multivariate analyses adjusting for the school effect, serial non-demanders compared to non-uniform respondents were less likely to have a favourable opinion on vaccination in general (OR: 0.19 [0.13; 0.27]) (Table 2). Serial demanders compared to non-uniform respondents were more likely to be boys (1.19 [1.07; 1.32]) and in favour of vaccination in general (2.11 [1.51; 2.97]). They were more likely to state that they were vaccinated against childhood diseases and that it is easy to find information on vaccination.

Table 2

Determinants of being a serial non-demander or a serial demander compared to non-uniform respondents. Non-representative sample of 1458 adolescents in France, January–March 2020.

	Total (n = 1458)	Serial non-demanders (n = 88) compared to non-uniform (n = 711)		Serial demanders (n = 659) compared to non-uniform (n = 711)
	Column percentage	RRR	95% CI	RRR	95% CI
Individual characteristics
Age
12–13 year old	37.7	1		1
14 year old	47.1	0.84	[0.43; 1.64]	0.88*	[0.79; 0.99]
15–17 year old	15.2	0.91	[0.33; 2.47]	1.12	[1.00; 1.26]
Gender
Girl	53.5	1		1
Boy	46.5	1.25	[0.98; 1.59]	1.19**	[1.07; 1.32]
Middle school
School 1	8.0	1		1
School 2	26.6	0.39***	[0.36; 0.44]	2.25***	[2.01; 2.52]
School 3	10.8	0.78**	[0.66; 0.92]	1.32***	[1.27; 1.36]
School 4	28.0	0.35***	[0.32; 0.38]	2.17***	[2.07; 2.28]
School 5	26.7	0.47***	[0.41; 0.55]	2.26***	[2.22; 2.31]
Education level of the parents
Inferior or equal to French baccalaureat	19.2	1		1
Superior to French baccalaureat for at least one parent	50.7	1.2	[0.65; 2.21]	0.98	[0.61; 1.57]
I don't know, non-applicable	30.1	2.48	[0.95; 6.50]	0.93	[0.77; 1.12]
Language spoken with parents
Only French	75.3	1		1
Also another language	24.7	2.46	[0.78; 7.78]	0.59	[0.32; 1.11]
Education level of the parents #Language spoken with parents	na
Inferior or equal to French baccalaureat #Also another language	na	1		1
Superior to French baccalaureat for at least one parent #Also another language	na	0.48	[0.15; 1.47]	1.07	[0.41; 2.75]
I don't know, non-applicable #Also another language	na	0.66	[0.15; 2.96]	1.16	[0.37; 3.64]
Attitudes towards vaccination
In favour of vaccination (binary)
Disagree or I don't know	23.5	1		1
Agree or strongly agree	76.5	0.19***	[0.13; 0.27]	2.11***	[1.51; 2.97]
Overall stated vaccine status
Mostly I don't know	5.7	1		1
Mostly no	72.2	3.09	[0.93; 10.3]	0.73	[0.49; 1.08]
Mostly yes	22.1	1.43	[0.80; 2.57]	1.58*	[1.04; 2.38]
"It is useful to get vaccinated"
Disagree	3.2	1		1
Agree	91.9	0.57	[0.14; 2.24]	1.3	[0.65; 2.62]
I don't know	4.8	1.09	[0.34; 3.45]	0.51	[0.24; 1.06]
"Getting vaccinated can protect others"
Disagree	7.2	1		1
Agree	81.9	0.69	[0.31; 1.52]	1.06	[0.92; 1.22]
I don't know	10.9	2.1	[0.93; 4.73]	0.78	[0.44; 1.38]
"Vaccination scares me because of the needle"
Disagree	60.9	1		1
Agree	34.6	1.01	[0.64; 1.61]	1	[0.74; 1.34]
I don't know	4.4	1.56	[0.81; 2.99]	0.84	[0.44; 1.58]
"Vaccination scares me because of the substances in the vaccine"
Disagree	64.3	1		1
Agree	24.3	0.83	[0.41; 1.66]	0.93	[0.79; 1.09]
I don't know	11.4	0.76	[0.31; 1.83]	1.15	[0.76; 1.73]
"Do you find it easy to get trustworthy information on vaccination?"
Difficult	19.6	1		1
Easy	52.4	1.55	[0.57; 4.22]	1.60***	[1.33; 1.92]
I don't know	28	1.37	[0.64; 2.96]	1.36*	[1.01; 1.83]

RRR: Relative Risk Ratios, 95%CI: Confidence interval at the 95%, na: non applicable.

Notes: Estimates obtained from a full multivariate multinomial logistic regression. *: p-value<0.05; **: p-value<0.01; ***: p-value<0.001. For the overall stated vaccine status: mostly no corresponds to a score from 5 to 8, mostly “I don’t know” from 9 to 12, and mostly “yes” from 13 to 15.

Objective 2: Using choice certainty to analyse the preferences of uniform respondents

Description of the certainty scale

The mean certainty was 7.6 across all scenarios among the full sample, with a mean certainty of 7.2 (standard deviation (sd): 2.3) in the non-uniform sample and 8.0 (sd: 2.0) in the uniform sample (Table 3). Respondents with constant certainty were more prevalent in the uniform sample, especially among serial non-demanders (44.3% vs. 2.7% among non-uniforms). Adolescents with a low mean certainty (mean certainty ≤5) were more prevalent among serial non-demanders (21.6% vs. 8.3%), while less prevalent among serial demanders (4% vs. 8.3%). Finally, uniform respondents were more likely to have a high mean certainty (≥9) compared to non-uniform respondents (31.2% vs. 10.7%), and the differences were not significantly different among serial demanders and non-demanders (30.2% vs 38.6%). To summarize, uniform respondents tended to be more certain of their choices and have lower choice certainty variability on average. Figure A.1 of the Appendix shows certainty distribution among serial non-demanders, serial demanders and non-uniforms.

Table 3

Descriptive statistics of the choice certainty level. Non-representative sample of 1458 adolescents in France, January–March 2020.

	Number of choice observations	Mean	Standard Deviation	Q2 (Q1-Q3)	% with constant certainty	% with low mean certainty (0–5)	% with high mean certainty (9–10)
Overall (n = 1458)	14575	7.6	2.2	8 (6–10)	8.1	7.1	21.2
Non-uniform sample (n = 711)	7105	7.2	2.3	8 (6–9)	2.7	8.3	10.7
Uniform sample (n = 747)	7470	8.0*	2.0	8 (7–10)	13.3*	6.0	31.2*
Serial demanders (n = 659)	6590	8.0*	1.9	8 (7–10)	9.1*	4.0*	30.2*
Serial non-demanders (n = 88)	880	7.5*	2.7	8 (5–10)	44.3*	21.6*	38.6*

Notes: n: number of respondents; Q1 = first quartile, Q2 = second quartile, Q3 = third quartile.* P-value test vs non-uniform <0.01. Respondents, including those with unvaried certainty, can have low (0–5), medium (6–8) or high (9–10) mean certainty.

Fig. A.1

Certainty distribution among serial non-demanders, serial demanders and non-uniforms..

Non-uniform sample: adolescents with varied decisions across the choice tasks. Uniform sample: adolescents who always made the same decision across the choice tasks.

Attributes’ impact on choice certainty among uniform respondents

Among serial non-demanders, none of the attributes had significant effect on the certainty of refusal (Table 4).

Table 4

Impact of attributes on the decision certainty to refuse or accept hypothetical vaccination, among serial non-demanders and serial demanders. January–March 2020.

	Serial non-demanders n = 88		Serial demanders = 659
Certainty	Coefficient	95%-CI	Coefficient	95%-CI
Disease
Respiratory disease	ref		ref
Cancer	−0.23	[-0.73,0.27]	−0.01	[-0.10,0.08]
Genital warts	0.00	[-0.13,0.12]	−0.04	[-0.20,0.12]
Safety
No side effect	ref		ref
Scientific surveillance	0.02	[-0.08,0.13]	−0.15	[-0.35,0.05]
Safety other countries	−0.01	[-0.22,0.20]	−0.81***	[-1.02,-0.60]
Benefit/risk	0.07	[-0.33,0.47]	−0.93***	[-1.22,-0.65]
Indirect Protection
Protects only you	ref		ref
Protects other people	−0.02	[-0.41,0.38]	−0.01	[-0.09,0.07]
Elimination	0.09	[-0.29,0.47]	0.09	[-0.03,0.21]
Coverage
Not enough	ref		ref
Already a third	0.11	[-0.01,0.24]	0.28***	[0.14,0.42]
Most adolescents	0.07	[-0.31,0.44]	0.42***	[0.33,0.52]
Other countries 80%	0.20	[-0.02,0.42]	0.33***	[0.20,0.47]

***: p-value<0.001. Note: clustered standard errors at the school level.

Note: the dependent variable is choice certainty (0–10).

Among serial demanders, the safety levels “Other countries” and “Benefit/risk” significantly decreased the certainty of acceptance, while all coverage levels increased it. The “Cancer” level and all “Indirect protection” levels had no significant effect on choice certainty.

Objective 3: Using the vaccine eagerness scale to estimate preferences among the full sample

In the full sample (n = 1458), vaccine eagerness was increased by all Coverage levels and Indirect Protection levels, as well as the “Cancer” level of the disease attribute (Table 5). The “Genital warts” level had no impact on vaccine eagerness compared to a febrile respiratory disease. Safety levels “Safety other countries” and “Benefit/risk” significantly and substantially decreased eagerness compared to the reference (“No side effect”), while the “Scientific surveillance” level had a minor impact.

Some individual characteristics had significant impact on vaccine eagerness: being in favour of vaccination and finding vaccination useful were associated with higher levels of vaccine eagerness, while speaking another language at home and not knowing the education level of parents were associated with lower levels. Variability existed in between schools (Table 5).

Sensitivity analyses

Prevalence of suspected low-quality respondents in each subsample

In the full sample, 118 respondents (8.1%) always indicated the same choice certainty level for all scenarios (constant certainty). This percentage was 2.7% among non-uniform respondents, 9.1% among serial demanders and 44.3% among serial non-demanders (Table 6). This could be the sign of lower data quality in this latter group (Regier et al., 2019).

Table 6

Variability of choice certainty and completion time as quality indicators across subsamples. Non-representative sample of 1458 adolescents in France, January–March 2020.

	Non-Uniform	Uniform		Total
		Serial Demanders	Serial Non-Demanders
	Variability of choice certainty
Invariant certainty level	192.7%	609.1%	3944.3%	1188.1%
Low variation	22832.1%	36755.7%	3236.4%	62743.0%
High variation	46465.3%	23235.2%	1719.3%	71348.9%
Total	711100%	659100%	88100%	1458100%
	Completion time
Completion time <20th percentile	12918.6%	11618.0%	3947.0%	28420.0%
Completion time ≥20th percentile	56381.4%	52982.0%	4453.0%	113680.0%
Total	692100%	659100%	83100%	1420100%

Fast respondents (completion time <20th percentile, 9.38 min) were more prevalent among serial non-demanders (47.0%, compared to 18.6% among non-uniforms and 18.0% among serial demanders).

There were 55 participants combining low certainty variation and short completion time, 21 in the serial non-demanders sample, 25 in the serial demanders, and 9 in the Non-uniform sample.

After exclusion of participants with short response time (analysis B), the effects of levels “Scientific surveillance” (0.11, [0.05,0.17]), “Elimination” (0.17, [0.12,0.21]) and “Other countries 80%” (0.26 [0.12,0.39]) significantly increased the certainty of refusal among serial non-demanders (Table 7). These effects were even more pronounced when participants with any sign of low response quality were excluded (analysis D), with coefficients of 0.18 (non-significant), 0.24 and 0.42, respectively, for “Scientific surveillance”, “Elimination” and “Other countries 80%”.

Table 7

Sensitivity analyses of the effect of attributes on certainty of refusal (0–10) among serial non-demanders, January–March 2020.

Certainty	Sensitivity analyses for Serial non-demanders
	Analysis A (n = 49)		Analysis B (n = 49)		Analysis C (n = 67)		Analysis D (n = 31)
	Coefficient	95% CI	Coefficient	95% CI	Coefficient	95% CI	Coefficient	95% CI
Disease
Respiratory disease	ref		ref		ref		ref
Cancer	−0.42	[-1.32,0.49]	−0.12	[-0.68,0.44]	−0.30	[-0.96,0.36]	−0.20	[-1.06,0.67]
Genital warts	−0.01	[-0.22,0.20]	0.07	[-0.16,0.31]	−0.01	[-0.16,0.15]	0.12	[-0.25,0.48]
Safety
No side effect	ref		ref		ref		ref
Scientific surveillance	0.03	[-0.15,0.20]	0.11***	[0.05,0.17]	0.03	[-0.08,0.15]	0.18	[-0.01,0.36]
Safety other countries	−0.02	[-0.38,0.33]	−0.02	[-0.41,0.36]	−0.02	[-0.30,0.26]	−0.03	[-0.61,0.54]
Benefit/risk	0.11	[-0.65,0.86]	0.01	[-0.52,0.54	0.10	[-0.42,0.61]	0.00	[-0.83,0.83]
Indirect Protection
Protects only you	ref		ref		ref		ref
Protects other people	−0.05	[-0.74,0.63]	0.05	[-0.27,0.36]	−0.03	[-0.51,0.45]	0.07	[-0.44,0.57]
Elimination	0.14	[-0.56,0.84]	0.17***	[0.12,0.21]	0.12	[-0.38,0.61]	0.24***	[0.17,0.31]
Coverage
Not enough	ref		ref		ref		ref
Already a third	0.19	[-0.09,0.48]	0.00	[-0.17,0.17]	0.14	[-0.02,0.29]	−0.01	[-0.39,0.36]
Most adolescents	0.13	[-0.60,0.86]	0.11	[-0.24,0.47]	0.09	[-0.39,0.57]	0.19	[-0.39,0.77]
Other countries 80%	0.37	[-0.08,0.83]	0.26***	[0.12,0.39]	0.26	[-0.02,0.53]	0.42***	[0.23,0.61]

A: excluding participants with invariant certainty; B: excluding participants with low response time; C: excluding participants with both invariant certainty and low response time; D: excluding participants with at least one characteristic. *: p-value<0.05; **: p-value<0.01; ***: p-value<0.001.

Sensitivity analyses on certainty in the serial demanders sample and on vaccine eagerness in the full sample showed similar results compared to the main analyses (Appendix Table A.3 and Table A.4).

Discussion

In this secondary analysis of a single-profile discrete choice experiment with an opt-out, we found that using a certainty scale could provide valuable information on adolescent respondents always choosing the same option (in this case, acceptance or refusal of theoretical vaccination). Adding the choice certainty scale to the single-profile DCE allowed analysing the impact of the attributes on underlying preferences in an innovative way. In particular, it allowed describing sensitivities to particular communication contents even among serial (non) demanders, whose preferences remain unidentified in standard discrete choice models.

Serial non-demanders represented a small part of our sample (6%) and were less likely to be in favour of vaccination in general. They were also more likely to speak another language at home and be unaware of their parents’ education level, which could potentially indicate a lower comprehension of the study or be a proxy for lower socioeconomic status. In analyses including all serial non-demanders, no attribute significantly impacted their certainty of refusal. By contrast, the sample restricted to serial non-demanders with good response quality - probably representing participants with truly high vaccine hesitancy (and excluding no-brainers) - revealed a particular pattern of preferences: communication on the potential to eliminate the disease through vaccination and on high vaccine coverage in neighbouring countries significantly increased the certainty of refusal. Because of the small sample size, we can only hypothesise on interpretation of these results. A first hypothesis is that this group with high vaccine hesitancy refuses any positive communication, in particular one appealing to social conformism or collective responsibility around vaccination. Statements motivating vaccine acceptance in most adolescents could have the opposite effect on this small group. More research is needed to test this hypothesis and see if it could be linked to mistrust towards institutions, which has been linked to vaccine hesitancy (Karafillakis et al., 2016). Our framework of vaccine eagerness will be useful to explore preferences specifically among highly hesitant persons by replicating in other studies using larger sample size. We can also hypothesise that these three attribute levels were not well understood by the subgroup, and could even have been seen as unrealistic for some adolescents with low vaccine knowledge. Another explanation could be that some adolescents simply did not engage with the experiment due to lack of interest, opposition or other personality traits, despite good quality indicators. Few DCEs have surveyed adolescents about their preferences on vaccination (de Bekker-Grob et al., 2010; Hofman et al., 2014; Lavelle et al., 2019; Marshall et al., 2016; Wang et al., 2017), and to our knowledge, none has investigated uniform respondents or serial non-demanders. It is unclear if adolescents have firm and long-lasting opinions on vaccination, or if they endorse anti- or pro-vaccine views outside of their parents’ beliefs. Qualitative studies following DCEs could help better understand adolescents’ motivations toward vaccination, especially for serial non-demanders, which may help tailor immunization communication in specific subgroups.

Results using the vaccine eagerness scale in the full sample were consistent with previous findings among participants with non-uniform decisions (Chyderiotis et al., 2021): the safety attribute level referring to a positive benefit-risk balance decreased vaccine eagerness while levels referring to social conformism increased it. These results suggest that the vaccine eagerness scale could be used in DCEs to explore vaccine hesitancy and interventions to modify it.

Although this study has been conducted before March 2020 and the Covid-19 pandemic, we believe that our results could be used to optimize communication on various vaccines, including Covid-19 vaccines, in particular among adolescents. In practice, we recommend pretesting communication contents highlighting high vaccine coverage figures in neighbouring countries and using safety statements based on ongoing scientific consensus worldwide, while avoiding references to a benefit-risk balance if appropriate. For HPV, we recommend insisting on the protection against cancers and mentioning the possibility to eliminate HPV-induced outcomes.

This study has some limitations. First, this experiment was not conducted in a representative sample of French adolescents enrolled in middle schools. Although the sample size was sufficient, our results might not be generalizable to the entire population of French adolescents. Generalisation to adolescents in other countries must be done carefully, taking into account cultural differences in terms of vaccine preferences between countries and continents. We do not have information on parental opinions on vaccination and how it could have impacted adolescents’ responses to the survey. The small sample size of serial non-demanders limited the power of specific studies on this group. Finally, our vaccine eagerness scale is based on the certainty level of the decision and not the decision itself. The eagerness scale could thus be less accurate than the decision in predicting real life for people with non-uniform decisions. However, attribute levels had similar impact on certainty levels as on theoretical decisions, which reinforces our previous conclusions.

Conclusion

Single-profile DCEs are particularly adapted to preferences around vaccine acceptance, but may be limited by a high percentage of uniform respondents (serial demanders or serial non-demanders). Our results advocate for the use of choice certainty scales specifically in single-profile DCEs, to better estimate the preferences of respondents in vaccine acceptance. Besides, such scales can be used as a quality criterion to exclude individuals which do not vary in their certainty level.

Our results also suggest that despite a high prevalence of uniform responders, our previous findings on vaccination promotion among French adolescents are robust and could be used in communication campaigns targeting this group, with only a very small subgroup of adolescents who could be negatively impacted by some of the optimised communication content.

Ethics

This study is part of the PrevHPV study, conducted under the responsibility of Inserm. It was granted approval by Evaluation Committee of Inserm, the Institutional Review Board (IRB00003888, IORG0003254, FWA00005831) on 10 December 2019. All study participants gave their oral, informed, non opposition to participation, in line with French legal guidelines.

Author statement

JM, JS and SC designed the study. SC and NT organised the data collection, with the help of the members of the PrevHPV consortium (principal investigator: NT). JM and JS conceived the original idea of the manuscript. SC conducted the analyses and wrote the manuscript under JM and JS’s supervision. All the authors contributed to the data interpretation. All the authors revised and approved the final version of the manuscript.

Funding

This study is part of the PrevHPV study, which is conducted with the support of Institut de Recherche en Santé Publique (IReSP) and Institut national de la santé et de la recherche médicale (INSERM), with financial support from ITMO Cancer AVIESAN (Alliance Nationale pour les Sciences de la Vie et de la Santé/National Alliance for Life Sciences & Health) within the framework of the Cancer Plan 2014–2019. The funding sources had no involvement in the study design, the collection, analysis and interpretation of data; in the writing and in the decision to submit the article for publication.

Declaration of competing interest

All authors declare having no conflict of interest.

Table A.5

Brief description of middle schools’ characteristics

	Public/Private	REP*	Urban/Rural
School 1	Public	Yes	Urban
School 2	Private	No	Urban
School 3	Public	No	Rural
School 4	Public	No	Rural
School 5	Public	No	Urban

*REP or “Réseaux d’éducation prioritaires” means schools located in deprived areas can beneficiate from a specific educational program aiming at reducing social and territorial inequalities.