A framework for measurement and harmonization of pediatric multiple sclerosis etiologic research studies: The Pediatric MS Tool-Kit.

BACKGROUND: While studying the etiology of multiple sclerosis (MS) in children has several methodological advantages over studying etiology in adults, studies are limited by small sample sizes.
OBJECTIVE: Using a rigorous methodological process, we developed the Pediatric MS Tool-Kit, a measurement framework that includes a minimal set of core variables to assess etiological risk factors.
METHODS: We solicited input from the International Pediatric MS Study Group to select three risk factors: environmental tobacco smoke (ETS) exposure, sun exposure, and vitamin D intake. To develop the Tool-Kit, we used a Delphi study involving a working group of epidemiologists, neurologists, and content experts from North America and Europe.
RESULTS: The Tool-Kit includes six core variables to measure ETS, six to measure sun exposure, and six to measure vitamin D intake. The Tool-Kit can be accessed online ( www.maelstrom-research.org/mica/network/tool-kit ).
CONCLUSION: The goals of the Tool-Kit are to enhance exposure measurement in newly designed pediatric MS studies and comparability of results across studies, and in the longer term to facilitate harmonization of studies, a methodological approach that can be used to circumvent issues of small sample sizes. We believe the Tool-Kit will prove to be a valuable resource to guide pediatric MS researchers in developing study-specific questionnaire.

Introduction

Multiple sclerosis (MS) is primarily a disease of adulthood, with peak incidence between 30 and 50 years of age,[1-3] although MS in children has recently become increasingly studied. Pediatric-onset multiple sclerosis (PoMS) cases are a small proportion (3%–10%) of all those diagnosed with MS.[4] While etiological risk factors identified in studies of adult-onset MS[5,6] have also been shown to be associated with PoMS,[7,8] more research is needed to inform on the etiology of MS in pediatric populations. PoMS provides a unique opportunity to study MS etiology, relative to adults, because cases are younger, and thus, (1) key exposures occur closer in time to MS onset and (2) the time period in which to search for risk factors is shorter. In addition, studies suggest that MS risk is determined in childhood and early adolescence.[9-15]

A common methodological problem faced in PoMS epidemiological studies is small sample sizes. Because PoMS is rare, the number of cases that can be obtained in individual studies is low and requires long periods of time to accrue to have sufficient statistical power to precisely estimate main effects or to explore interaction between risk factors. Harmonization, a methodology used to combine data collected in multiple studies, provides a potential solution to small sample sizes. The methodology used for harmonization focuses on the use of a common set of core variables, which serve as a framework to conduct pooled analyses.[16]

We developed the Pediatric MS Tool-Kit (Tool-Kit) for pediatric MS researchers to design study-specific questionnaires based on variables which (1) have been selected using a rigorous methodological process, (2) enhance comparability of results across studies, and (3) are amenable to future harmonized analyses. This paper describes the methodology that was used to develop the Tool-Kit and provides an overview of how the Tool-Kit can be used in pediatric MS research.

Methods

We solicited input from the International Pediatric MS Study Group (IPMSSG)[17] using an online survey to select three risk factors. We searched PubMed in March 2013 (search date range: 2000–2013) for epidemiological studies that examined the etiology of MS to generate a list of etiological factors, which was then filtered using the following predefined criteria: (1) an association was found with the risk of MS, in at least one high-quality study; (2) the timing of exposure is relevant to PoMS; and (3) the risk factor can be measured using a self-report questionnaire. The survey was distributed in May 2014 to 138 IPMSSG members. Respondents were asked to report whether, in their view, each risk factor was (1) a priority, (2) important, but not a priority, (3) not important for future research, or (4) I don’t have an opinion.

A systematic review of measurement property studies was conducted to summarize the available evidence on the validity and/or reliability of relevant questions/questionnaires/scales. Standardized methodology[18] was used to perform a review of each risk factor. We searched three electronic databases (PubMed, EMBASE, and CINAHL) in 2014–2015 using a validated measurement properties search strategy.[19] Quality assessment was completed using the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist.[20,21]

A Delphi study[22,23] was then conducted to select and define a set of core variables for each risk factor. A working group (WG) of 11 researchers from Canada, the United States, the United Kingdom, the Netherlands, and Italy was assembled, including epidemiologists, pediatric MS neurologists, adult MS neurologists, and content experts. The WG included nine researchers actively involved in MS research who are also IPMSSG members. We also invited three content experts, selected from among authors identified through the measurement property reviews. The WG was divided into three sub-groups, one for each risk factor, with some overlapping membership and six members in each group. Prior to the start of the Delphi study, the WG met for a 2-day face-to-face meeting during which background knowledge to facilitate participation was provided.

The Delphi study had four rounds that were developed with knowledge gained from the measurement property systematic reviews. Delphi rounds 1, 2, and 4 were completed anonymously online and round 3 was face-to-face. In round 1, a research question for each risk factor was defined and a set of criteria were selected to guide selection of core variables. In round 2, the WG was presented with a list of broad areas that were relevant for each risk factor, for which various variables could be defined. For example, for summer sun exposure, potential variables include duration of sun exposure, frequency of sun exposure, sun exposure during certain periods of the day, and so on.

We initially planned for all rounds to be online; however, because selection of core variables proved to be a complex process, in round 3 the WG actively engaged in guided discussions during a 2-day face-to-face meeting. Given the goal was to limit the number of core variables, the WG also defined a set of ancillary variables—variables that provide important supplementary information about exposure, but were not deemed core. In round 4, the WG gave approval for the proposed Tool-Kit variables. Each Tool-Kit variable includes a variable description, harmonizable response options, and data coding.

We also evaluated the content validity of the core variables using the COSMIN checklist.[20,21] Each WG member independently rated (highly relevant, somewhat relevant, not relevant) whether the core variables (1) refer to relevant aspects of the construct being measured, (2) are relevant for the target study population (e.g. age, sex, disease characteristics, country, setting), (3) are relevant for the purpose of the measurement instrument (e.g. predicting exposure), and (4) together comprehensively reflect the construct being measured. Variables that were rated as relevant (highly or somewhat) remained as core variables, and those that were rated as “not relevant” were subsequently classified as ancillary variables.

Age epochs were also defined. Differences in the potential for exposure based on the child’s main activities and changes in activities that represent potential changes in exposure were considered; but we also wanted to select a small number of age epochs to ensure questionnaires were not too long.

Results

Forty-two risk factors previously implicated in MS risk were identified, among 88 publications (of 1400 records), subsequently reduced to 12 risk factors that are relevant for pediatric populations: body size or body mass index, environmental tobacco smoke (ETS), head injury or traumatic brain injury, history of infectious mononucleosis, penicillin use, physical activity, prenatal and perinatal factors, sibling exposure and attending daycare, stressful life events, sun exposure, vaccinations, and vitamin D intake. Forty-eight IPMSSG members completed the survey (35% response). When the responses “a priority” and “important” were combined, sun exposure (96%), vitamin D intake (94%), and ETS (93%) were most highly endorsed (Figure 1).

Figure 1.

Results of the Risk Factor Survey, indicating the percentage of respondents who rated each risk factor as (1) a priority for future research and either (2) a priority for future research or important, but not a priority for future research.

Among the reviews of measurement property studies, we identified 152 publications on ETS questionnaires, 35 on sun exposure, and 13 on vitamin D intake. For vitamin D intake, we also searched country-specific food composition databases and government reports to identify country-specific food sources of vitamin D. Much of the extant measurement property literature focused on questionnaires that measure current or recent exposure; however, we did not find validated questionnaires to assess long-term exposure histories, as required in case-control studies. This is a major gap we identified in the measurement properties literature.

A research question for each risk factor is presented in Table 1 and the eight variable selection criteria in Table 2. In the final Tool-Kit, there are six core and three ancillary variables to measure ETS, six core and six ancillary variables to measure sun exposure, and six core and five ancillary variables to measure vitamin D intake (Table 3). An example of a sun exposure core variable is provided in Table 4. The Tool-Kit variables can be accessed online (www.maelstrom-research.org/mica/network/tool-kit). Our preliminary assessment suggests that the Tool-Kit core variables have good content validity.

Table 1.

Research questions for the three risk factors that are included in the Tool-Kit.

Risk factor	Research question
Environmental tobacco smoke	Everything else being equal, are children who have been exposed to higher levels of environmental tobacco smoke at increased risk of MS compared with children who have been exposed to lower levels of environmental tobacco smoke?
Sun exposure	Everything else being equal, are children who have been exposed to lower levels of sun at increased risk of MS compared with children who have been exposed to higher levels of sun?
Vitamin D intake	Everything else being equal, are children with lower intake of vitamin D (through supplementation) at increased risk of MS compared with children with higher intake of vitamin D (through supplementation)?

MS: multiple sclerosis.

Table 2.

Eight criteria for selecting the core variables to be included in the Tool-Kit.

Selection criteria
The variable is necessary to answer the research question
The variable helps to better interpret or understand the level of exposure to the risk factor
The variable is a potential confounder or effect modifier
The variable can be collected using proxy-report (i.e. parent/guardian) via self-administered and/or interview-administered questionnaire
The variable can be collected in a valid and reliable way, given the required retrospective nature of the data collection
The level of detail that is asked to recall is reasonable given the retrospective nature, time, and resources available
The variable is of high enough prevalence in the source population to ensure sufficient statistical power
The variables and response options should be selected to enhance cross-cultural validity

Table 3.

Tool-Kit core and ancillary variables for the three risk factors.

Core variables	Ancillary variables
Environmental tobacco smoke (ETS) exposure
1. Home ETS exposure ladder[a] 2. Childcare ETS exposure ladder[a] 3. Frequency of smoking by the child’s mother4. Frequency of smoking by the child’s father5. Frequency of smoking by others who lived with the child6. Residential history	1. Evidence that previous smoker(s) lived in child’s home2. Smoking status of close family members and/or friends3. Type of tobacco products consumed by individuals who lived with the child
Sun exposure
1. Residential history2. Frequency of daily outdoor activities during daylight Hours3. Duration of time outdoors on weekends during summer4. Duration of time outdoors on weekdays during Summer5. Duration of time outdoors on weekdays during winter6. Duration of time outdoors on weekends and school holidays during winter	1. Frequency of travel to sunny destinations during winter2. Skin color3. Sun sensitivity4. Frequency of sun protection use: sunscreen5. Frequency of sun protection use: wearing a shirt with sleeves6. Frequency of sun protection use: staying in the shade or under an umbrella
Vitamin D intake
1. Child’s use of dietary supplements2. Frequency that the child used dietary supplements3. Duration of time that the child used dietary supplements4. Child’s use of dietary supplements that contain vitamin D5. Frequency that the child used dietary supplements that contain vitamin D6. Duration of time that the child used dietary supplements that contain vitamin D	1. Brands of dietary supplements that were commonly used by the child2. Use of dietary supplement was recommended by a health care professional3. Child’s use of cod liver oil4. Frequency that the child’s used cod liver oil5. Duration of time that the child used cod liver oil

The ETS exposure ladders incorporate sources and locations of exposure, as well as smoking rules in the home.

Table 4.

Example of a core variable included in the Tool-Kit.

Table	Sun exposure
Variable name	Frequency of daily outdoor activities during daylight hours
Label	Frequency of outdoors activities
Description	● Classifies the frequency of the child’s usual daily outdoor activities during daylight hours● Self-report by parent(s) or both child and parent(s)● Ask for all relevant age epochs
Value type	Text
Missing	9999
Unit	Not applicable
Category codes and labels	3: Almost always outdoors2: More often outdoors1: More often indoors0: Almost always indoors9999: Don’t know/can’t recall

The following age epochs were selected: (1) baby (birth–1 year), (2) toddler/preschool (2–4 years), (3) child/primary or elementary school age (5–12 years), and (4) teenager/high school age (13–18 years). In utero was also identified as an important epoch; however, given the need to modify variable definitions to focus on mother’s activities/behaviors, it was not included at this time.

Discussion

We developed the Pediatric MS Tool-Kit, which is a novel contribution to the field of pediatric MS etiological research. The Tool-Kit aims to enhance the methodological rigor of newly developed PoMS etiologic studies by proposing a measurement framework to facilitate the design of study-specific questionnaires. The short-term goals of the Tool-Kit are to enhance exposure measurement in individual PoMS studies, by proposing a set of rigorously selected and defined variables that measure priority risk factors, and to enhance comparability of study results, by proposing the use of a common measurement framework. The long-term goal of the Tool-Kit is to enhance the potential for collaboration through data sharing and consequently larger sample sizes in harmonized analyses. The Tool-Kit provides a set of core and ancillary variables that are intended to be used to measure children’s ETS exposure, sun exposure, and intake of vitamin D. The core variables are those that were selected for harmonized analyses. As the WG proposed a number of important variables to comprehensively measure exposure, these additional variables are provided as ancillary variables. The Tool-Kit will reduce the time and resources required to design study-specific questionnaires.

Researchers can use the information in the Tool-Kit to create a questionnaire that is specific to their target population. The exact questions, however, are not provided in the Tool-Kit. We chose to provide information about the variable to develop the questions, rather than exact wording of individual questions. This approach is referred to as flexible prospective harmonization.[24] While the use of flexible prospective harmonization may hamper comparability among studies, it may be more appropriate for etiological research given study investigators are most knowledgeable about their study context and target population. In addition, flexible prospective harmonization will enable relevant data collected in existing PoMS studies to be harmonized with data collected in new studies, which would not be possible had we employed a more stringent harmonization methodology.

The need for, and value of, prospective harmonization has been recognized by the research community and is demonstrated by several examples of large prospective harmonization initiatives such as the National Institute of Neurological Disorders and Stroke (NINDS) Common Data Elements (CDE),[25,26] European Prospective Investigation into Cancer and Nutrition (EPIC) study,[27] the PhenX Toolkit,[28] and Canadian Partnership for Tomorrow project.[29] The NINDS has recently developed CDE for MS that include variables focused on demographics, clinical assessments, imaging, and neuropsychology/cognition.[25,26] The NINDS also used a WG model to select and define the MS CDE. The MS CDE also include a core set of variables, which they define as essential information applicable to any study, and have also defined three sets of CDE which they classify as supplemental–highly recommended, supplemental, or exploratory.

The Tool-Kit variables can be accessed online at www.maelstrom-research.org/mica/network/tool-kit. The information presented online includes the proposed Tool-Kit variables (i.e. name, type, and description), response options, and data coding. To be harmonizable, we recommend that the information online be used to develop individual questions to include in study-specific questionnaires. Ideally, at least one question should be developed for each variable, and the question should be worded so that it links to the variable description. If multiple questions are used, it will be important to ensure the Tool-Kit variables can be derived from the questions used.

The proposed response options for each variable are meant to be used as is, although modification is possible; however, for data to be harmonizable, researchers should ensure the response options in the Tool-Kit can be generated from modified response options. For example, if deemed more appropriate by the study investigators, finer response options may be used, but the new response options should be collapsible into those provided in the Tool-Kit. We do not recommend that investigators exclude any of the response options that are proposed in the Tool-Kit, as they form the basis for harmonization and were developed through a rigorous methodological process.

In addition to retaining the response options as proposed in the Tool-Kit, the age epochs should also be used as is in order for data to be harmonizable. However, unlike the response options, collapsing age epochs is much more methodologically difficult and may render the data non-harmonized. The key is to ensure the Tool-Kit variables and response options can be generated from the data collected in a study.

Once the questionnaire is developed, we recommend that the actual questions and their response options be compared to the variable descriptions and response options in the Tool-Kit by an individual who was not involved in developing the study questionnaire, and if the questionnaire is used in a language other than English, this individual should be fluent in English and in the language used in the questionnaire.

The Tool-Kit is a methodological resource for questionnaire design and is not a repository of data. While we will maintain an inventory of studies that have used the Tool-Kit variables, the decision to share data, at the time of a proposed harmonized analysis, is left to the discretion of the individual study investigators. As the long-term goal of the Tool-Kit is to provide the opportunity for collaboration through harmonization, researchers using the Tool-Kit variables will be asked to provide us with some basic information about their study (e.g. study name, sample size, variables used) to be displayed on the Tool-Kit webpage. To facilitate this process, we ask when using the Tool-Kit to develop a study-specific questionnaire that this publication is cited appropriately.

The main limitation, in light of the rigorous process used to select the core variables, is the possibility that the variables selected have poor measurement properties. We were unable to identify a validated questionnaire to use in a case–control study, and thus we used an expert consensus-seeking approach to select the Tool-Kit variables. The Tool-Kit has not yet been tested in a “real-life” research setting. While we show that the core variables have good content validity, continued evaluation of the measurement properties of the Tool-Kit variables will be imperative to its utility and success. We are open to collaborating with researchers wanting to use and assess the measurement properties of the core variables in their specific research settings.

We believe the Tool-Kit will prove to be a valuable resource to guide pediatric MS researchers in developing study-specific questionnaires. Rigorous epidemiological and expert consensus methods were utilized to develop the Tool-Kit variables and we engaged the pediatric MS research community to ensure that what we developed is relevant. We invite content area experts to take the opportunity to expand the Tool-Kit to develop additional core variables for the other priority MS risk factors.