HELPinKids&Adults Knowledge Synthesis of the Management of Vaccination Pain and High Levels of Needle Fear: Limitations of the Evidence and Recommendations for Future Research.

The HELPinKids&Adults knowledge synthesis for the management of vaccination-related pain and high levels of needle fear updated and expanded upon the 2010 HELPinKIDS knowledge synthesis and clinical practice guideline for pain mitigation during vaccine injections in childhood. Interventions for vaccine pain management in adults and treatment of individuals with high levels of needle fear, phobias, or both were included, thereby broadening the reach of this work. The present paper outlines the overarching limitations of this diverse evidence base and provides recommendations for future research. Consistent with the framing of clinical questions in the systematic reviews, the Participants, Intervention, Comparison, Outcome, Study design (PICOAS) framework was used to organize these predominant issues and research directions. The major limitations we identified across systematic reviews were an overall dearth of trials on vaccination, lack of methodological rigor, failure to incorporate important outcomes, poor study reporting, and various sources of heterogeneity. Future research directions in terms of conducting additional trials in the vaccination context, improving methodological quality and rigor, assessment of global acceptability and feasibility of interventions, and inclusion of outcomes that stakeholders consider to be important (eg, compliance) are recommended. Given concerns about pain and fear are known contributors to vaccine hesitancy, improving and expanding this evidence base will be integral to broader efforts to improve vaccine compliance and public health worldwide.

The HELPinKids&Adults knowledge synthesis for the management of vaccination-related pain and treatment of individuals with high levels of needle fear described in the preceding articles in this special issue of the Clinical Journal of Pain updated and expanded upon the 2010 HELPinKIDS knowledge synthesis and clinical practice guideline for the management of acute pain during childhood vaccine injections.1 Specifically, adults and individuals with high levels of needle fear were added, leading to a synthesis of the research evidence that considers both acute and longer-term issues associated with unmitigated pain and is generalizable to a wider variety of individuals and contexts. Gaps were uncovered across the evidence base that are worthy of discussion. Analysis of these shortcomings illuminate important avenues for future research that can serve to improve the quality of the results, and thereby lead to increased confidence in how to optimally manage vaccination pain and related issues across the lifespan. The objectives of this manuscript are to: (1) highlight limitations in the research evidence that span different domains of pain and fear management included in the systematic reviews in this issue; and (2) identify the most important avenues for future investigations in this area. Consistent with the framing of clinical questions in the systematic reviews, the Participants, Intervention, Comparison, Outcome, Study Design (PICOS, Table 1) framework was used to organize these predominant issues and research directions; this information is summarized in Table 2.

TABLE 1

PICOS Criteria for Clinical Questions

TABLE 2

Summary of Limitations of Research Evidence Across Systematic Reviews and Future Research Directions

PARTICIPANTS

Age and Developmental Issues

In undertaking analyses for the different interventions examined in the HELPinKids&Adults knowledge synthesis, age was often identified a priori to be an important factor in judging the effectiveness of a given intervention. As such, age groupings were customized for each analysis. For example, distraction was analyzed for young children (up to 3 y) separately from children (>3 to 12 y), adolescents (>12 to 17 y), and adults (18 y and above). For other clinical questions, broader age groupings (eg, 3 to 17 y) were maintained. Age distinctions were empirically supported based on dramatic biological, cognitive, and social changes occurring across infancy, childhood, and adolescence that can influence the pain experience.2–4 Moreover, the Standards for Research in Child Health, an international initiative established to improve reliability and relevance of pediatric clinical trials, has also advocated for consideration of age.5

Our developmental approach to analyses revealed important differences between age groups in the efficacy of particular interventions that would otherwise not have been revealed. However, the available age divisions often encompassed several developmental periods which may have introduced confounds given developmental differences in pain expression and measurement,6 pain processing,4,7,8 cognitive and coping abilities,9 and potential benefit derived from individual interventions.10,11 For example, with respect to distraction, children between 0 and 3 years were treated as 1 homogeneous group due to the fact that they were often lumped together in included trials. Moreover, evidence was scant or unavailable for particular age groups and populations (adolescents, adults, elderly), thereby limiting our ability to examine effectiveness in specific age groups. We were cautious in our approach to extrapolating results across all individuals; however, we recommend future research that allows more detailed examination of developmental issues.

Individual Comorbidities and Differences

Study samples included in the systematic reviews were variable. Studies could have included individuals with and without chronic illnesses, comorbidities (eg, cancer, depression, autism, cognitive impairments), previous experiences with painful procedures, or any of these that would have resulted in different pain learning histories and thereby influenced ones response to vaccination. Pain during past painful procedures predicts future pain and may influence the likelihood of benefitting from pain management interventions.12 Individual differences in cognitions about pain may also impact one’s ability to benefit from certain evidence-based psychological interventions. For example, catastrophic thinking about pain (ie, the cognitive tendency to magnify and perseverate on the threat value of pain and perceive oneself as being helpless in the face of pain) has been shown to impede analgesic effects of distraction,13 a psychological intervention that has demonstrated efficacy for children and adolescents.14 Such individual differences could have introduced heterogeneity into the data and influenced the magnitude of treatment effects. Recommendations for future research include more detailed reporting of participant characteristics, potential stratification by prognostic criteria such as level of fear, and reporting of results for subgroups with different characteristics.

Heterogeneity Within the Vaccination Context

There was considerable variability in the vaccination procedure among included studies which may have influenced results including: the number and type of vaccines given, how they were given, as well as the setting in which they were given. Variability in operator techniques alone could have led to significant differences in pain, including: injection method (aspiration vs. no aspiration), route of administration (subcutaneous vs. intramuscular), and site of administration (arm vs. leg). All of these factors may affect how an intervention works and we could not examine all of these variables. In fact, authors often failed to provide this information. It is important to note, however, that when treatment effects are revealed in the presence of this variability, it gives more confidence in the results, as this variability is reflective of how interventions are routinely administered in the real world.

Important research gaps were identified, such as the relative painfulness of individual vaccines (which informs their order of administration), the effects of injecting the same versus multiple limbs, as well as aspects about the vaccine supplies themselves (eg, needle size and injection speed) on pain and fear. These technical aspects of vaccination are particularly relevant to study as they are cost-neutral and hence, implementable on a global level.

Indirectness: Reliance on Nonvaccination contexts

Across some of the clinical questions, there were gaps in research evidence with respect to the effect of interventions among individuals undergoing vaccination, and we were sometimes required to draw from research outside of the vaccination context (eg, venipunctures). This was the case for interventions pertaining to individuals with high needle fears, distraction for adults, and muscle tension. Indirect evidence was only considered in the absence of direct evidence to provide guidance across different ages and interventions. Nevertheless, there are potential problems with this approach. For example, young children have reported that discomfort is considerably higher in response to venipunctures than to vaccinations.15 In contrast, multiple vaccines are usually administered in 1 visit, which can lead to more pain. It is also important to consider that the particular cognitions driving needle fears, pain, and distress may differ by individual and procedure type. Individuals may fear, become distressed, or both by drawing or seeing blood, the injection or puncture itself, the medical procedure context, or the insertion of a foreign substance into ones’ body. The focus of the fear would be expected to influence the pain experience and vary as function of the type of procedure (venipuncture, vaccination); therefore, there are several potential limitations in extrapolating findings from nonvaccination to vaccination contexts. These aforementioned issues were not accounted for in the evidence base and their contribution to the variability in the observed effectiveness of different interventions is unknown. More research in the vaccination context is critically needed to provide a specific evidence base that is directly relevant.

INTERVENTIONS

Lack of Trials

The systematic reviews covered a diverse range of interventions spanning procedural, physical, pharmacological, psychological, and process domains. Psychological and physical interventions for needle fear specifically were reviewed separately. Although some interventions (eg, breastfeeding, topical anesthetics, and sweet tasting solutions) were well studied and included large samples, the majority, by comparison, were significantly understudied (eg, adult, adolescent, and infant psychological interventions, education and preparation interventions, interventions for individuals with high needle fears). The lack of data for many interventions limits our ability to have confidence in the findings. Indeed, some of the findings included in this series are based on a single study of low or very low quality. Thus, recommendations based on this limited evidence base could be changed or reversed when these meta-analyses are updated with additional trials in the future. Additional research on these understudied interventions is needed to better characterize their impact on critical and important outcomes.

Heterogeneity in Intervention Delivery

There was considerable heterogeneity in the way that interventions were delivered that may impact results. For example, manual tactile stimulation was delivered by rubbing, tapping, and applying pressure at different time epochs of vaccination. There was even greater variability in psychological interventions, such as distraction, including who delivered the intervention (eg, parent, nurse), the degree of sophistication of the distractor and engagement it evoked (eg, high-technology robots, low-technology toys, verbal comments about non–procedure-related things), and the timing of when the intervention was actually administered (eg, before, during, after the needle). Such variability in intervention delivery can influence its effectiveness; furthermore, given the generally small sample sizes available, differences may have been obscured. Taken together, variability in how interventions were delivered would be expected to have added noise to the data and reduced the precision and confidence in the results. With additional trials of interventions delivered in similar ways, we will be able to examine the efficacy of each method separately.

Outdated Technologies and Cultural Appropriateness

Given rapid changes in electronic technology, the mode of delivery for some interventions in included studies is not consistent with current methods of delivery (eg, video distraction was delivered via a DVD or television rather than an interactive Smartphone or iPad, which would arguably be more engaging); the effectiveness may be different for these different modes of delivery. Related to this issue is the lack of data for the delivery of interventions that are more globally accessible, feasible (eg, music delivered via singing rather than with headphones), and acceptable. Many included interventions may not be globally applicable, in terms of cultural appropriateness, cost, feasibility, political climate, or any of these. This research is critically needed to broaden the reach of this evidence to individuals worldwide.

Underrepresented Interventions

There were also clinical questions initially considered by the HELPinKids&Adults team for inclusion in the systematic reviews that were subsequently excluded because of the lack of studies that adequately evaluated their effects. For example, psychological interventions designed to reframe individuals’ memories about painful procedures following injections were initially included among the trials initially reviewed. However, concerns about indirectness of the evidence (including youth with cancer undergoing lumbar punctures16 and anesthetic injections during restorative dental procedures17) and issues related to deceiving individuals about pain and fear (eg, telling children that they did not cry18) were raised. In addition, complex psychological interventions (eg, hypnosis) were excluded as they are not feasible for clinicians administering routine vaccinations.

Clinical questions pertaining to the environment or setting in which a vaccination occurs were excluded in the HELPinKids&Adults knowledge synthesis. It is likely that aspects of the environment or setting influence ones experience (eg, consider a 10-y-old boy who is vaccinated in a private, calm room in comparison to being vaccinated in a gymnasium where he is surrounded by staring peers and can witness his friends cry and potentially faint). Unfortunately, beyond focus groups and descriptive research,19–28 no trials were identified that systematically studied relevant environmental factors such as: privacy, presence of peers, ventilation, use of separate entrances and exits to the vaccination area, and concealing distress-provoking items. Consideration of setting and situational factors in future research is important, perhaps particularly for school-based vaccination. Related to this, there is currently no literature about how to effectively teach youth about school-based vaccinations. Research is needed to provide guidance about how to optimally educate individuals being vaccinated about vaccination pain and fear management. Furthermore, research is also needed to provide more detailed guidance to relevant individuals (eg, immunizers, parents, individuals being vaccinated) about how to actually implement effective interventions (eg, by expanding immunization manuals to include this information).

Finally, particular pharmacological interventions (ie, nitrous oxide, midazolam, chloral hydrate) were not included in the knowledge synthesis because the context for their use involves acute, urgent procedures (rather than routine vaccinations), or both and specific expertise and monitoring equipment is required to ensure safe use.

Cointerventions

We did not systematically evaluate the effect of cointerventions (eg, the effect of distraction or sucrose in young children in the presence of holding vs. lying supine) and cointerventions were not consistently reported in the trials. We highlighted cointerventions or examined the effects of an intervention with and without them when there were available data and it was deemed important; however, this was not consistent across clinical questions because of limited data and poor reporting. We urge researchers to thoroughly report on the use of cointerventions in trials.

COMPARISONS

Suitability of Comparison Groups

Various comparison groups were accepted in the knowledge synthesis to evaluate the effectiveness of different interventions. We included no treatment control, placebo, and active control groups. The type of comparison chosen depended on the clinical question. Some interventions consistently used strong comparison groups (eg, placebo commonly used in trials of sucrose and topical anesthetics), whereas others (eg, physical and psychological interventions) utilized comparisons that were heterogeneous and raised questions about the validity of the study findings. For instance, a “no treatment” comparison group is subject to bias if it is known by participants that no treatment is being given. This is particularly problematic given that expectations regarding treatment response have been shown to have robust effects on treatment outcomes.29,30 Research suggests that studies that do not use double-blinding substantially overestimate treatment effects, which in turn introduces bias.31,32 Studies included in the knowledge synthesis examining psychological interventions were typically not blinded and included a no treatment control group and therefore were subject to such bias. This may have led to inaccurate conclusions about an intervention’s efficacy and more broadly, can lead to erroneous clinical decision making based on this evidence.33

It is also important to consider that the relevant comparison groups may differ vastly across clinical settings and this variability is often reflective of the real world. For example, researchers in some settings may have compared breastfeeding during vaccination to lying supine, whereas others compared breastfeeding to holding infants. In some cases, the comparison intervention may even have an impact on pain response. Across the interventions included in the knowledge synthesis, the exact nature of the comparison conditions was often unclear, thereby making it difficult to ascertain the extent and impact that this variability had on results. When designing trials, researchers should consider comparisons that both approximate the real world and have methodological rigor. This presents a challenge and often times a paradox for researchers but has important implications for research and practice. For example, although including a water placebo comparison group in trials of sucrose is scientifically sound (ie, prevents threats to internal validity), it does add complexity to trial implementation and may limit generalizability (ie, giving water does not approximate standard clinical care). For research to optimally translate into clinical practice, comparison conditions should be carefully selected.

OUTCOMES

Primary Reliance on Self-Report of Pain

In the field of pain, there is appreciation for the value of incorporating multiple modes of measurement to collectively capture the complexity of the pain experience. Nevertheless, prioritization of certain outcomes as being more critical (eg, self-reported pain vs. observer-rated pain) is often required to facilitate decision making and was done for the HELPinKids&Adults knowledge synthesis. To mitigate the potential to dismiss interventions that might be associated with some benefits, we also considered secondary important outcomes (eg, observer-rated distress) in certain situations (eg, for interventions with young children whose self-report of pain may be unreliable and problematic6,34–37).

It is not clear how to optimally measure pain if not using self-report. In the knowledge synthesis, we used currently acceptable and valid measures.1,38 Recent research suggests that electroencephalography may provide a more sensitive indicator of nociceptive response to vaccination among infants up to 1 year than traditional behavioral measures39 and other research highlights the incongruence between behavioral measures and electroencephalography.40,41 This raises important questions about the optimal way of measuring the effectiveness of pain interventions across the lifespan and more fundamentally, what aspect of the pain experience it is that we are measuring. However, feasibility of assessment methods and “observability” of effects of interventions are important considerations to facilitate “buy in” of interventions and practice change.

Heterogeneity in Outcome Measurement

There was remarkable variability in the measurement of effectiveness across interventions in the knowledge synthesis. Systematic reviews included trials employing different observational and self-report measurement tools and methodologies (eg, video, real time), as well as assessment time periods (eg, ranging from seconds to minutes before and following the injection) and assessors (eg, observer, parent, individual). It is likely that the methods used contributed to variability in the results and the overall findings.

In addition, the effects of different interventions were measured at different time epochs. Pain management interventions might differentially impact immediate versus more delayed responses.42 We therefore separated out the different phases of the procedure (preprocedural, acute procedural, procedural recovery, or combinations of these) to isolate the effect of interventions across the various time epochs. Using established methods, we also combined outcome measures to allow all data pertaining to an outcome to inform the summary statistics, thereby eliminating bias from picking any individual tool or approach. This method may have also reduced the variability around each estimate. Nevertheless, the psychometric properties of various measures of an outcome were likely not equivalent and amalgamation of outcomes from multiple informants may have added noise to the data. Observer judgments are invariably influenced by the raters’ personal relationship to and history with the individual being immunized. The subjectivity of pain and fear and known lack of agreement between raters (eg, parents and clinicians43) could have introduced error and additional variability, thereby masking treatment effects. Nevertheless, given that treatment effects were often found in the presence of this variability, we can have more confidence in these findings.

Lack of Data on Important Outcomes

Critically, we noted a striking lack of data for many outcomes identified by the HELPinKids&Adults team as important including: vaccination compliance, procedure outcomes (duration of procedure, success of procedure), memory, satisfaction, preferences, and fidelity with interventions.44 These outcomes extend beyond pain intensity and represent the broader priorities of individuals invested in vaccination. We urge researchers to include these outcomes in future intervention studies to address the needs of stakeholders who ultimately determine if evidence will be used. Uptake of knowledge is compromised if the evidence does not address stakeholders’ questions and concerns. Pain researchers need to be cognizant of the aspects of vaccination that nonpain researchers deem to be important.

One of the major research gaps identified in this field is the lack of research on the long-term impact of our interventions on individuals’ subsequent pain and fear-related cognitions and behaviors. For example, although providing the most painful vaccination last appeared to result in decreased distress, there is research to suggest that the way in which painful procedures end has important implications for subsequent memory development and medical compliance.45 Related to this, we did not examine the effect of interventions on delayed pain (ie, pain that occurs hours to days following injection). It is possible that some interventions have effects that impact delayed pain (eg, injection of vaccines in specific body regions). Looking beyond immediate pain is important and should be investigated.38 Finally, examination of patient preferences is also needed to determine what it is that individuals being vaccinated want from pain management and which interventions they deem to be acceptable. Incorporating this information into trial design will ultimately impact treatment effectiveness and satisfaction with the vaccination experience.

STUDY DESIGN

Lack of Methodological Rigor

We included only randomized-controlled trials (RCTs) and quasi-RCTs in the HELPinKids&Adults knowledge synthesis. While these designs are of the highest quality compared with other designs, overall, there was a lack of methodological rigor for critical outcomes across included studies, as assessed by the Cochrane risk of bias tool and GRADE. In general, quality was rated as low to very low primarily due to small numbers of participants, high risk of bias frequently due to lack of blinding, and heterogeneity in results. Reporting of important aspects of methodology (eg, sequence generation and concealment) and study design was lacking or generally poor and invariably reduced quality assessments. Some of this may be due to date of publication (pre-CONSORT), but also frequently occurred in new studies. In fact, it is acknowledged that the word quasirandomized is often used interchangeably with quasiexperimental and the meanings are not clear. We attempted to contact authors of trials to gather additional details about studies; occasionally this led to exclusion of studies from the review. It is possible that some studies were included that should not have been. In addition, we noted that in several cases, trial details did not match registry information. This raises questions about the reasons for these inconsistencies and how to best approach them in systematic reviews.

The majority of studies included in the systematic reviews had high risk of bias. Additional RCTs of existing interventions are needed where the quality of evidence is poor and few studies exist. Furthermore expansion to interventions targeting other aspects of the vaccination experience (eg, delayed pain, postvaccination cognitions of individuals, and significant others) is also needed. The field needs to move beyond “single-dose” studies to provide a broader scope of the long-term impact of consistent pain management. This will be aided by the use of prospective, longitudinal designs. With additional trials and knowledge syntheses, the field will move toward understanding moderators and mechanisms of change in our interventions as well as tailoring interventions based on individual preferences and needs to maximize benefit.

When designing trials, we strongly urge researchers to follow CONSORT and Cochrane methodology to ensure low risk of bias and increase confidence in results. Greater efforts to minimize bias are critically needed. Researchers should use true randomization rather than alternate assignment, as well as adequate allocation concealment and blinding of participants, personnel, and outcome assessment. Although this can at times be challenging, particularly in the case of psychological interventions, it is scientifically and ethically justified. Indeed, to not do this may promote the use of ineffective interventions and halt advances in the field. There are excellent examples of this in the field of pain that can serve as examples46–48; however, creativity and new methods may be needed depending on the context.

Consideration of Evolving Research Climate

In addition, researchers need to consider the current and evolving research climate in the areas of vaccination and pain, and broadly clinical trials. For example, it has been suggested that use of placebo as a comparison group is unethical when evidence for a therapeutic intervention’s efficacy exists.49–52 Indeed, in the context of vaccination, researchers have argued that pain management interventions should become the standard of care.38 These movements will have implications for future trials and systematic reviews. Alterations in study design (eg, comparing 2 treatments as opposed to comparing a treatment with control) may change the parameters with which we use to judge clinical significance.

CONCLUSIONS

In summary, the HELPinKids&Adults knowledge synthesis examined the effectiveness of interventions across a wide range of domains including procedural, physical, pharmacological, psychological, and process for vaccine-related pain and fear in individuals with a high level of needle fear. The synthesis is very timely due to outbreaks of vaccine preventable diseases across different geographic regions53,54 due to underimmunization of populations and loss of herd immunity. The reasons for vaccine hesitancy are complex and context specific55; however, concerns about pain and fear are well-documented contributors to vaccination noncompliance.56,57 Effective management of vaccine-related pain and needle fear hold promise for improving compliance and health across the lifespan. The HELPinKids&Adults knowledge synthesis is an important step in that direction. The current paper outlines some of the critical limitations of the evidence base and important gaps in our knowledge. We strongly urge researchers to design high-quality trials incorporating the recommendations offered in this manuscript. The importance and impact of managing vaccination pain and fear extend to all individuals at every stage of development. Improving and expanding the evidence base regarding interventions aiming to reduce vaccination pain and fear will be integral to efforts to improve compliance and public health worldwide.