Measuring Oral Mucositis of Pediatric Patients with Cancer: A Psychometric Evaluation of Chinese Version of the Oral Mucositis Daily Questionnaire.

OBJECTIVE: Oral mucositis is a frequent clinical condition that has been shown to affect pediatric cancer patients. Oral Mucositis Daily Questionnaire (OMDQ) is one of the few available patient-reported outcome measures to assess the extent and impact of oral mucositis. The objectives of the study were to translate the Mouth and Throat Soreness-Related Questions of the OMDQ into Chinese (OMDQ MTS-Ch) for children and adolescents aged 6-18 years receiving chemotherapy and to evaluate its psychometric properties.
METHODS: This was part of a multicenter, prospective cohort study involving two phases. Phase I involved forward-backward translation to fit the cognitive and linguistic age level of the children and adolescents, followed by face and content validation, together with pretesting. In Phase II, which evaluated the internal consistency, test-retest reliability, and discriminant validity, a total of 140 patients completed the OMDQ MTS-Ch for 14 days.
RESULTS: The OMDQ MTS-Ch had satisfactory face and content validities. The Cronbach's alpha coefficient of the OMDQ MTS-Ch was 0.984. All of the corrected item-total correlations were higher than 0.90. The test-retest intraclass correlation coefficient between consecutive days for the OMDQ MTS-Ch items ranged from 0.576 to 0.983; the only value that was not over 0.70 was that for the paired study days 7 and 8 for the item of talking. The mean area-under-the-curve OMDQ MTS-Ch item scores were significantly different among patients with different degrees of mucositis severity (P < 0.001), supporting the discriminant validity.
CONCLUSIONS: It has been shown that the OMDQ MTS-Ch has a good level of reliability and discriminant validity and can be completed by children aged ≥6 years and adolescents on a daily basis to measure mucositis and its related functional limitations.

Introduction

Oral mucositis is a frequent and debilitating symptomatic condition that has been shown to affect pediatric cancer patients in chemotherapy, hematological stem cell transplant (HSCT), and molecularly targeted therapy settings.[1] Oral mucositis can impose psychosocial burdens on patients and their families and can have a profound impact on patients’ oral functional status and clinical outcomes.[23] A review of the recent major guidelines and literature identified a very few preventive interventions that can be reliably effective to prevent oral mucositis in children and adolescents in cancer settings.[4] Currently, only cryotherapy or low-level light therapy is suggested to be applied to cooperative children receiving chemotherapy or HSCT conditioning regimens with a high risk of mucositis.[5] Literature has indicated that the level of knowledge about the assessment and management of oral mucositis is inconsistent, particularly in the pediatric oncology setting.[56] More pediatric-specific clinical studies are warranted to develop effective strategies on the prevention and management of oral mucositis. Nevertheless, psychometrically sound and simple form of instruments is required to conduct effective clinical studies of oral mucositis.[7]

Numerous clinician rating scales have been used in the assessment of oral mucositis, such as the World Health Organization Scale,[8] the National Cancer Institute – Common Terminology Criteria for Adverse Events Scale,[9] and Oral Assessment Guide.[10] However, these scales are required patients’ hospital visits and clinical evaluations on oral cavity. Patient-reported outcome measure (PROM) has been increasingly recognized as an important approach to assess the extent and impact of oral mucositis. Oral Mucositis Daily Questionnaire (OMDQ),[11] Oral Mucositis Weekly Questionnaire,[12] Patient-Reported Oral Mucositis Symptom Scale,[13] and Children's International Mucositis Evaluation Scale[14] are the few available PROMs, for which psychometric properties have been established that allow subjective reporting of symptoms and dysfunction associated with cancer therapy-induced oral mucosal toxicity. These oral mucositis-specific PROMs use slightly different criteria and items, and their lengths vary. For example, OMDQ considers the impact of mouth and throat soreness on swallowing, drinking, eating, talking, and sleeping,[11] while Children's International Mucositis Evaluation Scale evaluates difficulty in swallowing saliva, eating, and drinking because of mouth or throat pain.[14] Comparing the OMDQ with other PROMs, the panel of Mouth and Throat Soreness-Related Questions of the OMDQ (OMDQ MTS) would be less demanding for pediatric patients and allow patients’ daily self-assessment of oral mucositis.[11]

The OMDQ MTS, together with an item on diarrhea, had been evaluated as a proxy measure for children with parents of 59 children aged <12 years[15] and as a self-report measure for 15 children aged 12 to <18 years in chemotherapy and HSCT settings.[16] Its test-retest reliability and construct validity were supported by high correlation of two measurements 24 h apart and at least moderate correlation with comparators for most items of the OMDQ. While these studies may be impacted by generalizability issues due to the small sample size and single site, they have contributed to the understanding of the potential uses of the OMDQ as a PROM in the pediatric setting.[1516] At present, the psychometric evaluation of the OMDQ is limited to English-speaking populations. An exploration of the use of the OMDQ with diverse populations and psychometric evaluation would help facilitate the widespread adoption of the OMDQ into clinical studies and routine oral mucositis assessment, in particular, the Chinese-speaking populations who constitute approximately 16% of the world's population. The objectives of the study were therefore to translate the OMDQ MTS into Chinese (OMDQ MTS-Ch) for older children and adolescents receiving chemotherapy and to evaluate the psychometric properties of this version.

Methods

With the approval of the Institutional Review Board, this study was part of a multicenter prospective cohort study designed to examine the associations between risk factors and oral mucositis specific to patients who were 6–18 years of age and undergoing induction or consolidation chemotherapy for the treatment of hematological malignancies or solid tumors.[17] The details of sampling and investigation have been reported elsewhere.[17] The phases in the validation procedures involved two phases: First, to perform forward-backward translation; second, to perform psychometric evaluation of the OMDQ MTS-Ch. Phase I involved iterative forward-backward translation to fit the cognitive and linguistic age level of pediatric patients (6–12 years) and adolescents (13–18 years). This was done by a linguistics expert and two bilingual investigators/research assistants. There was also expert evaluation of the content relevance and semantic equivalence, as well as separate focus group interviews with six pediatric and adolescent cancer patients to check on the most appropriate wording, and pretesting with ten pediatric and adolescent patients with oral mucositis.

The psychometric properties were established in Phase II, in which 140 patients aged 6–18 years completed the OMDQ MTS-Ch daily for 14 days, with assistance from their parents if needed for the pediatric patients. Patients and their parents were trained in using the OMDQ MTS-Ch before data collection to ensure the quality of oral mucositis assessment.[17] The 14-day assessment period was selected to capture the clinical time-course of chemotherapy-induced oral mucositis, which begins 3–5 days after the initiation of chemotherapy and peaks at 7–10 days, after which it slowly resolves.[1718] The OMDQ MTS-Ch contains a question on measuring mouth and throat soreness and five questions on assessing the resulting limitations on daily functional activities (swallowing, drinking, eating, talking, and sleeping), with responses that ranged from 0 (no soreness or limitation) to 4 (extreme soreness or inability to carry out a function). The area under the curve (AUC) representing oral mucositis severity was calculated over 14 days for each question of the OMDQ MTS-Ch for individual subjects, using the trapezoidal approximation, with possible scores ranging from 0 to 56 (a higher score indicating greater severity of oral mucositis over 14 days). AUC is a time–response curve over the period of observation and is one of the summary measures that can effectively summarize the longitudinal data.[19]

Referencing the approach of the original study,[11] the daily compliance rate was calculated for each question of the OMDQ MTS-Ch to assess feasibility. Longitudinal compliance was assessed by calculating the proportion of patients who completed 80% of their assessments over the study period for each question of the OMDQ MTS-Ch. Cronbach's alpha coefficient was used to assess the internal consistency reliability on day 7. An alpha within the range of 0.70–0.95 was accepted as satisfactory for internal consistency.[20] For test-retest reliability, intraclass correlation coefficients (ICCs) were calculated on paired study days 7 and 8 and days 13 and 14 for each OMDQ MTS-Ch question. All values >0.70 for ICC were accepted as a satisfactory level for test-retest reliability.[21] The discriminant validity was evaluated by comparing the difference in the mean AUC OMDQ MTS-Ch scores among patients with different degrees of oral mucositis severity with the Kruskal–Wallis test. Patients were grouped by their maximum degrees of oral mucositis across 14 days; MTS ≤1 as the absence of oral mucositis, MTS = 2 as mild oral mucositis, MTS = 3–4 as severe oral mucositis. We hypothesized that patients with severe oral mucositis had higher mean AUC OMDQ MTS-Ch scores than those without or with mild oral mucositis. Scale-level analysis of each question of the OMDQ MTS-Ch on days 7 and 8 was evaluated by the floor and ceiling effects, with a percentage of 70% subjects scored “0 (no soreness or limitation)” and “4 (extreme soreness or inability to carry out a function)” being considered a high floor and ceiling effect, respectively, and that particular question was considered for removal.

Results

Phase I

The OMDQ MTS-Ch had satisfactory face and content validities. Results showed that the content relevancy and semantic equivalence content validity index for each question ranged from 0.87 to 1. The results of pretesting for children and adolescents receiving chemotherapy revealed adequate comprehensibility of the OMDQ MTS-Ch.

Phase II

The characteristics of patients have been described previously.[11] About half of the patients were 6–12 years of age (54%), and 63% were boys. Slightly more than half of them (56%) were diagnosed with hematological malignancies and 28% were treated with adriamycin-based chemotherapy [Table 1]. The daily compliance rates for each question of the OMDQ MTS-Ch ranged from 90% to 95%. As for the longitudinal compliance, all of the patients completed ≥80% of their assessment over the whole study period for each question in the OMDQ MTS-Ch. The patients did not report any significant problem with item comprehension.

Table 1

Subjects’ characteristics (n=140)

Characteristics	f (%)
Age (years)
6-12	75 (53.6)
13-18	65 (46.4)
Gender
Male	88 (62.9)
Female	52 (37.1)
Education level (n=135)
Primary	76 (56.3)
Secondary	59 (43.7)
Cancer diagnosis
Solid tumor	62 (44.3)
Hematological malignancy	78 (55.7)
Chemotherapy regimen
Adriamycin-based	39 (27.9)
Methotrexate-based	25 (17.9)
Combined etoposide, methotrexate, cytarabine, and/or adriamycin	32 (22.9)
Etoposide-based	18 (12.9)
Cytarabine-based	13 (9.3)
Other anthracyclines-based	13 (9.3)

Reliability analysis of the OMDQ MTS-Ch is shown in Table 2. The Cronbach's alpha coefficient of the OMDQ MTS-Ch was 0.984. All of the corrected item-total correlations were higher than 0.90, indicating that the OMDQ MTS-Ch questions were strongly correlated. Table 3 also shows that the alpha coefficients, if items were deleted, were comparable to the overall alpha coefficient. As shown in Table 2, the test-retest ICC between consecutive days for the OMDQ MTS-Ch questions ranged from 0.576 to 0.983; the only value that was not in excess of 0.70 was that for the paired study days 7 and 8 for the MTS talking item.

Table 2

Reliability analysis of the Mouth and Throat Soreness-Related Questions of the Oral Mucositis Daily Questionnaire into Chinese (n=140)

OMDQ MTS-Ch	Internal consistency
	Squared multiple correlation	Corrected item-total correlation	Alpha if item deleted
MTS	0.923	0.942	0.979
MTS swallowing	0.943	0.965	0.974
MTS drinking	0.948	0.973	0.973
MTS eating	0.953	0.956	0.975
MTS talking	0.917	0.903	0.981
MTS sleeping	0.937	0.936	0.977

OMDQ MTS-Ch: Mouth and Throat Soreness-Related Questions of the Oral Mucositis Daily Questionnaire into Chinese

Table 3

Intraclass correlation coefficient of the Mouth and Throat Soreness-Related Questions of the Oral Mucositis Daily Questionnaire into Chinese (n=140)

OMDQ MTS-Ch	ICC (95% CI)
	Days 7 and 8	Days 13 and 14
MTS	0.897 (0.83-0.94)	0.930 (0.88-0.96)
MTS swallowing	0.793 (0.65-0.88)	0.911 (0.85-0.95)
MTS drinking	0.928 (0.88-0.96)	0.975 (0.96-0.99)
MTS eating	0.937 (0.89-0.96)	0.983 (0.97-0.99)
MTS talking	0.576 (0.29-0.75)	0.967 (0.95-0.98)
MTS sleeping	0.972 (0.95-0.98)	1.000 (1.00-1.00)

ICC: Intraclass correlation coefficient, OMDQ MTS-Ch: Mouth and Throat Soreness-Related Questions of the Oral Mucositis Daily Questionnaire into Chinese, CI: Confidence interval

As shown in Figure 1, patients with more severe oral mucositis (MTS score 3–4) had higher mean AUC OMDQ MTS-Ch scores than those without oral mucositis (MTS score ≤1) or with mild oral mucositis (MTS score 2). The Kruskal–Wallis test showed that the mean AUC OMDQ MTS-Ch scores were significantly different among patients with different degrees of oral mucositis severity (P < 0.001), supporting the discriminant validity.

Figure 1

Differences in mean area-under-the-curve oral mucositis daily questionnaire mouth and throat soreness-Chinese Scores by the maximum mucositis

The percentage of subjects scored “0 (no soreness or limitation)” and “4 (extreme soreness or inability to carry out a function)” ranged from 4.3%–69.3% to 3.6%–67.1%, respectively. A high floor or ceiling effect was not observed for any items of OMDQ MTS-Ch.

Discussion

The OMDQ MTS was developed to provide a simple method for self-reporting of oral mucositis and its impact on oral function. A focus group interview with experts in oral mucositis in the pediatric setting revealed the need for a mucositis instrument that is simple, quick to complete, and easy to use in almost all children.[22] In this study, a rigorous iterative forward-backward translation was undertaken in developing the Chinese version of the OMDQ MTS, taking into account the cognitive and linguistic age level of children in middle childhood and adolescents. It could be argued that young children may lack the cognitive ability to report oral mucositis and a parent proxy report may be required. Nevertheless, cognitive development theory suggests that children between 6 and 12 years of age in fact have begun their development of concrete thinking, while adolescence (12–18 years of age) marks the development of logical thinking.[23] The present study revealed high daily and longitudinal compliance rates of 90%–95% and 100%, respectively, suggesting that the OMDQ MTS-Ch is a feasible tool for self-monitoring of the clinical progression of oral mucositis by patients 6 years of age and older. In addition, none of the children or adolescents in this study found any items of the OMDQ MTS-Ch confusing or difficult to answer. The compliance rates in this study were actually higher than those of the previous studies of the original scale, in which adult patient compliance rates were about 80%–95%.[11]

The OMDQ MTS-Ch questions had a high degree of internal consistency (correlation coefficients >0.9), thus supporting its reliability, in that the questions concerning MTS and related oral limitations in swallowing, drinking, eating, talking, and sleeping are measuring the same construct. In comparison with the original validation with the adult population, both the corrected item-total correlation coefficients (0.903–0.973) and the alpha coefficients if item deleted (0.973–0.981) for the OMDQ MTS-Ch were higher than those for the Pearson's correlation coefficients obtained in the original study (0.5–0.8).[11] Nevertheless, the differences in the statistical tests being used for reliability analysis between the present and the original studies may contribute to the different magnitudes of internal consistency. Pearson's correlation analysis was used by Stiff et al., 2006,[11] while the current study employed Cronbach's alpha coefficient which is a function of the average of the pairwise Pearson's correlations across the k classes and is often used in psychometric evaluation to measure the reliability of scale items. Our study also demonstrated good test-retest reliabilities, which were comparable to those in the original studies with adult[11] and pediatric populations.[16] It is noteworthy that the ICC of 0.576 for the MTS talking item between days 7 and 8 was unable to reach the acceptable level of 0.7 for test-retest reliability, while the ICC for the same item between days 13 and 14 was high. In the Tomlinson et al. 2011's study, the Spearman's correlation coefficients for the MTS drinking (0.511) and eating (0.539) items were low.[15] One probable explanation for the relatively low value of the ICC for the MTS talking item is the probable steep improvement in talking function in some cases corresponding to the reduction of oral mucositis severity from day 7 to day 8;[15] furthermore, talking would be the least affected activity unless there was severe oral mucositis. Nevertheless, caution is needed for interpretation of test-retest reliability across different studies as different statistical tests were used for the determination of coefficients of repeatability for the OMDQ.[111516] The ICC used in the present study is always a measure of stability or agreement of measurements in a short period or made by multiple observers measuring the same quantity. The current study also showed that the OMDQ MTS-Ch has adequate discriminant validity when applied to the pediatric cancer setting. Patients with severe oral mucositis throughout its clinical course scored significantly higher on all of the OMDQ MTS-Ch items than patients with mild or without oral mucositis. There was no floor and ceiling effect on OMDQ MTS-Ch items.

Conclusion

It has been shown that the OMDQ MTS-Ch has a good level of reliability and discriminant validity as a PROM that can be completed by children aged 6 or above and adolescents on a daily basis to measure oral mucositis severity and its related functional limitations. Further validation is needed to examine its convergent, divergent, and predictive validity.

Financial support and sponsorship

This study was supported by a grant from the Health, Welfare, and Food Bureau of Hong Kong (Project No. 05060471).

Conflicts of interest

There are no conflicts of interest.