Challenges in designing, conducting, and reporting oral health behavioral intervention studies in primary school age children: methodological issues.

Often within oral health, clinical outcome measures dominate trial design rather than behavioral outcome measures, and often there is a reliance on proxy self-reporting of children's behavior with no corroboration through triangulation of measures. The complexity of the interventions involved in oral health intervention is often overlooked in trial design, and more flexible pragmatic designs that take account of the research context may be more appropriate. Some of the limitations in oral health behavioral intervention studies (trials) in primary school age children were reported in a recently published Cochrane review. This paper aims to critically discuss the findings of a recent Cochrane review in terms of the methodological implications that arise for future design, development, measurement, and reporting of oral health trials in primary school age children. Key components of the UK Medical Research Council's framework for the design and evaluation of complex interventions are discussed in relation to using taxonomies of behavior change. This paper is not designed to be a definitive guide but aims to bring learning from other areas of public health and health promotion into dental public health. Ultimately, the aim is to aid the design of more successful interventions that produce long-term behavioral changes in children in relation to toothbrushing and nighttime sugar snacking.

Background: rationale for oral health intervention in primary school age children

Globally, 60%–90% of children in industrialized countries have caries,1 with a worldwide average of decayed, missing, and filled teeth (DMFT) of 2.0 at age 12 years.2 As such, dental caries is the most prevalent chronic disease among children worldwide.3 The complexity of oral health issues and preventative strategies are recognized by Kwan et al4 within a World Health Organization (WHO) report which highlights the interlinking, complex (eg, economic, psychosocial and behavioral) and specific factors (eg, provision of safe water and optimal exposure to fluorides), related to oral health.

The introduction of fluorides delivered both systemically and topically has positively impacted dental caries rates in children, as well as preventative strategies. Even with systemic fluoridation, there is still a need within dental public health to continue to improve the effectiveness of behavioral interventions: ie, those that incorporate the use of fluoride toothpaste.

Oral health promotion is vital across the life course.5,6 Primary school age (4–12 years) has been highlighted as being particularly important for three reasons: behavioral habit formation, skill acquisition, and susceptibility. Considering these factors, children are developing independence in relation to their oral hygiene behaviors (toothbrushing) and dietary behaviors with respect to cariogenic foods and drinks (sugar snacking behaviors). In addition, young children can be particularly vulnerable to cariogenic substances, due to the mix of primary and mixed dentition and newly erupted teeth having more porous enamel prior to enamel maturation finishing.7 Further, habitual toothbrushing behaviors are increasingly difficult to impact by the time adolescence is reached.8 Therefore, there is a need to improve the effectiveness of behavioral interventions targeting primary school age children in producing sustained behavioral changes in toothbrushing and sugar snacking.

The determinants of oral health behaviors are complex, as they comprise both daily toothbrushing routines and sugar snacking behaviors.9 Toothbrushing routines and sugar snacking behaviors are practiced mainly within home, school, and community environments for primary school age children and are predominantly influenced by significant social relationships and behaviors of those around them, such as parents,10,11 especially mothers,12,13 siblings,14 and peers, and other social, economic, and cultural factors. Toothbrushing usually takes place within a home environment that, in itself, may be complex due to variability in social and domestic family structure (eg, lone parent families)15 and wider influences such as grandparents (who may have different attitudes and beliefs about toothbrushing than the parents).16 Similarly, dietary behavior is likely to be impacted by influences from school and community, as well as the home.17

What is known regarding effectiveness of oral health behavioral intervention studies (trials) in primary school age children?

Reviews by Kay and Locker18,19 and subsequently by Watt20 reported little evidence of measurable gains in oral health, in either adults or children, as a result of dental health education interventions. Outcomes within oral health trials often focus on clinical measures as the primary outcome, such as plaque or DMFT, rather than the behaviors of interest, namely toothbrushing and sugar snacking. In this regard, it is still the case that despite many new oral health interventions being implemented globally, few produce a long-term behavior change in their recipients.21–24 A recent Cochrane review25 to assess the effectiveness of the WHO’s Health-Promoting Schools initiative for improving health and well-being of students and their academic achievements found only one study linked to oral health26 (67 included studies). As such, it could not determine whether oral health initiatives conducted under the WHO’s Health-Promoting Schools have a positive impact, but it did find it to be effective in other areas (eg, physical activity, tobacco use), so it suggests that lessons could be learnt from other areas.

Despite the challenges around changing behavior, it is well reported that school-based dental education and promotional programs have previously improved knowledge but usually only in the short term.27 Similarly, a recent Cochrane review9 of four school-based oral health programs (evaluated through randomized controlled trials [RCTs]) that had both dietary (cariogenic food) and toothbrushing components found that there was some limited evidence of interventions having an impact on children’s plaque and their oral health knowledge. The review9 also found that there was insufficient evidence from these studies to conclude the efficacy of combined dietary and toothbrushing interventions to reduce caries in primary school age children.

Within the Cooper et al9 review, the included studies were also analyzed in terms of the behavior change techniques (BCTs)28 used within the interventions. This was done using Abraham and Michie’s28 taxonomy of 26 BCTs, which aims to introduce common terminology and definitions across interventions to improve reporting and subsequent design of future interventions. Subsequent work in this area has resulted in the development of BCT Taxonomy v1, which comprises 93 distinct BCTs that can be used to help identify specific components within an intervention, aiding replication, reporting, and understanding in often complex interventions.29

In addition to the analysis in the Cochrane review,9 the composition of BCTs in primary school interventions designed to prevent dental caries (evaluated through RCTs) has also been analyzed by Adair et al.30 The results30 indicate through the five interventions considered (six papers) a median of three BCTs could be identified (range two to six), and only eight of the possible 26 BCTs in the taxonomy were utilized.31 The focus of these BCTs was on knowledge (eg, knowledge-based BCT); “provide general encouragement; model or demonstrate behaviour; teach to use promotes and cues; prompt practice”.28 None of these included interventions reported being based on a behavioral theory. Moreover, from these two reviews, it is not possible to identify which BCTs are linked to certain outcomes.9,30 Michie et al,31 in reviewing behavior change intervention linked to quitting smoking, healthy eating, and physical activity in low-income groups, found that those interventions that were effective tended to have fewer BCTs (8.22) compared with those with a larger number of BCTs (12.75). Michie et al31 also found in their review that BCTs most commonly focused on providing information and facilitating goal setting.

These analyses of RCTs9,30 highlight that, to date, there has been an overemphasis in intervention design in this field on knowledge-based BCTs rather than other elements of the behavior change taxonomy that are more “active”. Active components could include skill-based techniques in the correct context (toothbrushing in the home) or reinforcement of behaviors that could enact a change in behavior in relation to either toothbrushing or sugar snacking. A further challenge within this area is the current dearth of information around identifiable BCTs within an intervention.32,33 To aid this there is a need to improve the rigor of reporting within studies in terms of the contents of interventions to allow BCTs to be identified easily and to include more active components in the interventions themselves. Through an improved knowledge of effective components within interventions, it is anticipated that a greater proportion would produce a lasting behavior change.33,34

The aforementioned reviews9,20 have focused on RCT interventions. However, the non-RCT literature also informs and helps explain the methodological issues around the challenges in designing, conducting, and reporting oral health behavioral intervention studies in primary school age children. For many interventions, the common delivery location is the school. Although this setting allows the correct information to be disseminated to a larger audience and can make it easier to implement brushing programs, it is not the natural location of the behavior. In addition, many current behavioral interventions have unequal weight for the “active” component in schools (eg, lessons, activities, and support) and often more “passive” components in the home (eg, leaflets, written guidance for parents). In line with this, Pine35 outlines six levels of interventions for school oral health programs from passively targeting the child (eg, through written material) to actively targeting the child (eg, through personalized interaction and reinforcement). Through ensuring there is an understanding of the behavior being targeted (eg, in terms of both identifying the behavior and how it needs to change)32 and also through the consideration of the hierarchy presented by Pine,35 it is likely that more informed, balanced, and “active” behavioral interventions can be designed, which should ultimately produce a greater likelihood of behavior change occurring.

In terms of examples of good reporting behavioral interventions, the Fit for School36 program in the Philippines is an example of a non-RCT oral health intervention that provides access to all the program material and also the evaluations that have occurred through its website. It is suggested that the level of transparency provided by the website is needed for all future interventions to help improve the methodology within some trials, the ability to determine common and effective BCTs, and the ability to better understand what components of interventions are effective.

As with previous reviews,9,18,19 a search conducted for this paper of non-RCT primary school studies found there was evidence of some interventions having a positive impact on behavior, although few found behavioral changes that were maintained over a longer period. The non-RCT literature highlighted many of the same issues as the RCT reviews,9,20 particularly the need for better reporting of the content and make-up of interventions, as well as greater insight into any process evaluation.37

Alongside this it is anticipated that through a greater understanding of primary school-based behavioral interventions regarding oral health and other areas with common antecedents (eg, sugar snacking and obesity), it may be possible to identify complementary effective intervention elements. For example, the study by Peters et al38 identified five effective elements (“use of theory; addressing social influences, especially social norms; addressing cognitive-behavioural skills; training of facilitators and multiple components”) for inclusion in school programs across three domains (substance abuse, sexual behavior, and nutrition), which suggests these topics could be combined into an integrated program.

Ensuring behavioral components are integral, equally weighted, and delivered by an appropriate multidisciplinary team

Designing trials of behavioral interventions in oral health needs to reflect the complexity of the behaviors being addressed, namely daily toothbrushing routines and sugar snacking behaviors (discussed previously).9

The UK Medical Research Council (MRC)34 defines complex interventions as those that comprise:

Several interacting components within the experimental and control interventions

A range and difficulty of behaviors required by those delivering and receiving the intervention

A number of groups or organizational levels targeted by the intervention

A number and variability of outcomes

A permitted degree of flexibility or tailoring of the intervention.

In addition to toothbrushing skills and daily routines, oral health interventions must also address the dietary intervention requirements regarding sugar snacking (abstinence from sugar snacking at night), which adds another layer of complexity to the intervention picture.

One of the main challenges when conducting trials is the need to develop a clear understanding of the study context and how this may fluctuate throughout the period of the trial.34 Trial fidelity is not always easy to achieve with a complex behavioral intervention, due to “ecological fluctuations” in the study environment (such as changes in delivery due to ongoing learning or necessary tailoring to suit the participants or cross-contamination between control and intervention sites), which may influence the intervention.39,40 The MRC stated:

[…] controls must be put in place to limit unplanned variation [in the intervention]. But some interventions are deliberately designed to be adapted to local circumstances. How much variation can be tolerated depends on whether you are interested in efficacy or effectiveness.34

One of the problems with intervention reporting is that the contextual detail and reporting of delivery of trials in this field of public health is weak.9 Few studies are using the available reporting guidelines, eg, the CONSORT statement for RCTs41,42 and cluster RCTs,43,44 the QUORUM statement for meta-analyses,45 the TREND statement for nonrandomized evaluations,46 the STROBE statement for observational epidemiological studies,47 and other methods for qualitative research.48,49 Furthermore, the reporting of treatment fidelity is also limited and needs improving throughout.50,51

Involving the home environment is vital when designing an oral health intervention for young children, due to the influence of parental habit on children and the natural location of the behavior.5

The trial has to be both delivered and/or maintained by parents in the home – as is the case with school-based oral health interventions that require parents to follow up elements of the program after school (eg, practising toothbrushing routines).52 For oral health trial evaluation, getting into the home environment to take measurements can be difficult, and this requires careful thought about the use of self-report or objective measures of behavior. Oral health research with young children often relies on reported behavior by parents as proxy, adding an additional layer of interpretation into the behavior and the outcomes of studies.53 Proxy measures relying on parental reporting of their child’s behavior may be inaccurate. For example, Martins et al54 reported low agreement between observed toothbrushing in children in comparison with that reported by their mothers.

The behavioral components of toothbrushing and sugar snacking should be considered as an integral intervention package and given equally weighted importance within the intervention program. Cooper et al9 have previously reported that in the evaluated published RCTs, tooth-brushing has usually been the predominant intervention component, with sugar snacking being implemented as an “add-on” rather than an integral intervention component:

Studies reported frequent supervised toothbrushing sessions and in some interventions parents were encouraged to take an active role in supervising their child’s toothbrushing however, this intensity of intervention was not replicated for cariogenic food/drink components.9

This may be due to the previous lack of involvement of relevant dietary health professionals (dieticians or health economists) in the design of oral health intervention trials. These are usually designed and led by dental health professionals with some involvement of other professional groups such as teachers. In the review by Cooper et al,9 no study reported trial intervention design or delivery by dietary professionals.

In addition, participant involvement early in intervention development is vital for intervention success,34,55 as then the design of the intervention can take into account the range of issues such as social and economic context, familial attitudes, and beliefs regarding relevant health behaviors, barriers and facilitators of behavior. The MRC have highlighted a number of practical benefits of involving users including: enhanced recruitment and retention; improved community support; and potentially a “better understanding of the process by which change is achieved”.34

Recommendations for intervention design processes

There are some key principles of good practice that should be followed when planning, designing, and implementing complex behavioral trials. For example, the MRC framework34 states that when developing an intervention to trial you need to ensure:

There is clarity regarding the outcome (eg, achieving twice-daily toothbrushing or reducing consumption of cariogenic drinks/food at night)

A sound theoretical basis of behavior change has been used to systematically develop the intervention (eg, behavior change taxonomy)28

The intervention can be fully described so it can be replicated and intervention fidelity assured

It is based on a knowledge of intervention effectiveness and cost-effectiveness (previous systematic reviews)

It can be scaled up and implemented beyond just a research setting (translational effect).55

The MRC34 also recommend using a phased development approach (see Figure 1) to trial design, development, and testing, and complex trials will require consideration of the most appropriate methodological approach to take when testing effectiveness: RCTs, cluster RCTs, quasi-experimental alternatives, and mixed method evaluation.34 Watson et al56 have previously highlighted that “it is important to recognise a complex community-based intervention can take years to develop to the point it can reasonably be expected to have a worthwhile effect.” Hence, it may be detrimental to trial an intervention before it is properly designed and thoroughly piloted (see Figure 1). In childhood obesity interventions, year-on-year improvement in intervention outcomes have been reported, thus highlighting the danger of “writing off a potentially efficacious intervention if experimental trial is carried out too early”.56

Figure 1

Phased development of behavioral oral health interventions for primary school age children, informed by the Medical Research Council (MRC) framework for developing and evaluating complex interventions.

Notes: Adapted with permission from the Medical Research Council. Developing and evaluating complex interventions: new guidance [webpage on the internet]. London: Medical Research Council; 2008. Available from: http://www.mrc.ac.uk/documents/pdf/complex-interventions-guidance/. Accessed October 6, 2014.34 Adapted with permission from Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008; 337:a1655.68 Adapted with permission from SAGE Publications. Watson P, Dugdill L, Murphy R, Knowles Z, Cable N. Moving forward in childhood obesity treatment: a call for translational research. Health Educ J. 2013;72(2):230–239.56

Abbreviation: RCT, randomized controlled trial.

Trial evaluation should include process as well as impact and outcome data.37 This allows the exploration of how and why an intervention might be working and which elements of the intervention may be the “active” components (ie, those that are responsible for the outcomes under investigation). As Oakley et al57 have previously stated, process evaluations “would improve the science of many randomised controlled trials”. Furthermore they can include, alongside details around setting and implementation:

[…] views of participants on the intervention; study how the intervention is implemented; distinguish between components of the intervention; investigate contextual factors that affect an intervention; monitor dose to assess the reach of the intervention; and study the way effects vary in subgroups.57

The need for behavioral outcome measures with long-term follow-up

Given the complex determinants of oral health behavior, together with the focus on clinical measures rather than the behaviors of interest, there is a need for trials to realign their focus. Trials need to ensure that they address the primary aim of the intervention, which is often about behavior change and acquiring mastery of effective toothbrushing routines into daily life and reduction in nighttime sugar snacking behaviors. A key challenge for the field is to develop more robust measures of these behaviors in the home environment, eg, through the use of data logging toothbrushes,58 to collect objective data of toothbrushing, or dietary/video diaries to measure sugar snacking.59

Few oral health studies measure outcomes over the longer term, which may be due to the challenge of tracking children through school-based or community trials. Consequently, there is a gap in the evidence base regarding what is known regarding longer-term impacts (over 1 year) of oral health trials in children. Trials also need to incorporate longer-term follow-up measures over 1 year or more to assess the sustainability of the intervention.

The dearth of cost-effectiveness measures

To make sound decisions regarding commissioning of future health services, measures of both effectiveness and cost-effectiveness are required from intervention trials.55 Once a trial is at the stage of full testing (see Figure 1), it is imperative that cost-effectiveness measures are built into the trial design whenever possible, as this allows policymakers to decide if it is feasible to afford the larger-scale rollout of an intervention. There are various types of economic analysis that could be considered in addition to cost-effectiveness analysis, such as cost–benefit analysis, return on investment analysis, and economic modeling.

Implications for trial management and reporting

The authors recommend that future behavioral oral health trials must be designed by a relevant multidisciplinary team (involving dental professionals, dietary professionals, and teachers), in conjunction with parents, and children (to ensure relevance to the target population).55 The intervention should be designed with equally weighted components addressing both toothbrushing routines and cariogenic food/drink intake. Outcome measures should include behavioral as well as clinical measures, and, if possible, objective measures (eg, toothbrushing behaviors) should be incorporated. Triangulating measures across different outcomes will add trustworthiness to the trial design. Process measures of intervention delivery will help to assess what components of the intervention have an effect and how the intervention is impacting on the outcomes of interest. Economic analysis will provide useful additional evidence for policymakers and commissioners when deciding which interventions to roll out on a larger scale. In conjunction with this, process evaluation can help those designing trials to understand how “the actions taken by the ‘human components’ of CHIs [complex health interventions] are influenced by the context in which the intervention takes place”60 and any impact this may have on an intervention.

A further challenge within evaluation of oral health behavioral interventions is the lack of uniform way (eg, through the use of common core indicator sets) of evaluating reported behavior and behavioral impacts. This leads to authors using many different survey methods/tools,52,61,62 which often rely on self-report, and presently only clinical methods of data collection are standardized.63 In 2010, the COMET initiative (http://www.comet-initiative.org/) was started to help develop, report, and adapt core outcomes sets for different areas of health that should be collected and reported as a minimum in trials to help with standardization and comparison across studies.64 Currently, for dentistry and oral health, there are 21 studies in the database, all of which have a clinical focus. Further, with complex interventions, interpreting findings can be difficult without accounting for the contextual factors of the program.65

As there is a lack of consistency in current study reporting, future trials must be encouraged to use standardized reporting methods,34 both in relation to the description of interventions (eg, through the use of the BCT Taxonomy v1) and the reporting of the evaluation. It is also worth future authors considering using relevant guidelines available on the EQUATOR website (http://www.equator-network.org) that have been developed to increase the accuracy and transparency of health research reporting.66

The continued development of health-related public–private health partnerships (eg, between toothpaste manufacturers and global nongovernmental organizations) will allow a sharing of skills, experiences, and resources within the design, delivery, and evaluation of oral health behavioral interventions targeting primary school children. Buse and Tanaka,67 through their experiences of global health public–private partnerships, highlight a current issue around a lack of publicly accessible evaluations within partnerships, but they also recognize the triumph as many of these partnerships focus on low/middle-income countries to help tackle issues linked to noncommunicable disease. Buse and Tanaka67 suggest that global health public–private partnerships are likely to remain a major facet but highlight some challenges that need to be overcome: eg, around the need for evidence of informed decisions that build on lessons from previous experiences.

Conclusion

This paper has highlighted some of the key issues that should be considered when designing behavioral interventions that aim to improve the oral health of primary school children. In addition, some of the current gaps in trial design are highlighted, such as the need to improve understanding regarding the effective components of interventions, the need to continue to analyze current interventions to identify BCTs, and also how potentially similar behaviors could successfully be grouped into interventions (eg, sugar snacking components could be combined in interventions that address both obesity and dental public health). This paper is not designed to be a definitive guide but aims to bring learning from other areas of public health and health promotion into dental public health. Ultimately, the aim is to aid the design of more successful interventions that produce long-term behavioral changes in children in relation to oral health and nighttime sugar snacking.