Development and Validation of the Korean Rome III Questionnaire for Diagnosis of Functional Gastrointestinal Disorders.

BACKGROUND/AIMS: A self-report questionnaire is frequently used to measure symptoms reliably and to distinguish patients with functional gastrointestinal disorders (FGIDs) from those with other conditions. We produced and validated a cross-cultural adaptation of the Rome III questionnaire for diagnosis of FGIDs in Korea.
METHODS: The Korean version of the Rome III (Rome III-K) questionnaire was developed through structural translational processes. Subsequently, reliability was measured by a test-retest procedure. Convergent validity was evaluated by comparing self-reported questionnaire data with the subsequent completion of the questionnaire by the physician based on an interview and with the clinical diagnosis. Concurrent validation using the validated Korean version of the Short Form-36 Health Survey (SF-36) was adopted to demonstrate discriminant validity.
RESULTS: A total of 306 subjects were studied. Test-retest reliability was good, with a median Cronbach's α value of 0.83 (range, 0.71-0.97). The degree of agreement between patient-administered and physician-administered questionnaires to diagnose FGIDs was excellent; the κ index was 0.949 for irritable bowel syndrome, 0.883 for functional dyspepsia and 0.927 for functional heartburn. The physician's clinical diagnosis of functional dyspepsia showed the most marked discrepancy with that based on the self-administered questionnaire. Almost all SF-36 domains were impaired in participants diagnosed with one of these FGIDs according to the Rome III-K.
CONCLUSIONS: We developed the Rome III-K questionnaire though structural translational processes, and it revealed good test-retest reliability and satisfactory construct validity. These results suggest that this instrument will be useful for clinical and research assessments in the Korean population.

Introduction

Since the historical meeting of experts in Rome in 1988, several revisions have been made to standardize the diagnostic criteria of functional gastrointestinal disorders (FGIDs).1-3 This standardization has enabled international comparisons of epidemiology, clinical features and clinical outcomes of FGIDs. The medical terms of the Rome criteria are in English, and the majority of self-administered symptom questionnaires are intended for use with English-speakers.4 A simple translation of a published questionnaire usually fails to contain the original meaning because of language and cultural differences.5 Therefore, a systematic translation and the cross-cultural adaptation process are essential for research using questionnaires.6 None of self-administered questionnaires related to FGIDs have undergone a structural validation study in Korea. Thus, the aim of this study was to develop, validate and confirm the reliability of the Korean version of the self-administered symptom questionnaire based on the Rome III criteria (Rome III-K).

Materials and Methods

Development of the Questionnaire

Stepwise processing was used to develop the Rome III-K. First, translation of the Rome III criteria and wording were performed according to guidelines.6,7 Two physicians were involved: a native Korean physician with long-term residence in the USA (Hong YS) and another Korean gastroenterology-subspecialist physician (Jung HK). Based on the comments of Richard Brislin,8 back-translation of the translated draft demonstrated that the target version was functionally equivalent to the Rome III. A translation committee included the above 2 physicians, a psychologist experienced in questionnaire-based studies, a teacher of Korean and 2 native non-medical people. This committee reviewed the translation and back-translation and confirmed that the questionnaire drafted was developed at the level of understanding of sixth grader in elementary school. Several pre-tests with native Korean outpatients provided feedback regarding the readability and comprehensibility of the questionnaire to the translation committee. After these pilot studies, we compiled the final edition of the Rome III-K to be validated. It was composed of 35 questions mainly regarding the presence or frequency of symptoms, including the diagnosis of irritable bowel syndrome (IBS), functional dyspepsia (FD), functional heartburn (FH), functional constipation, functional diarrhea and functional bloating.

Questionnaire Pilot Study

The draft of the Rome III-K was tested on patients presenting at the gastroenterology clinic of Ewha Womans University, Mokdong Hospital. The patients may have complained of gastrointestinal symptoms or had no such symptoms. All patients were native Korean speakers (age, 33-71 years). Patients were requested to fill out the questionnaire, and one gastroenterology-subspecialist judged the FGID clinical diagnosis in a subsequent interview with the patient. The agreement between the clinical diagnosis and diagnosis based on the self-administered questionnaire was the key outcome. Tests of feasibility included completeness of the questionnaire, completion time, length and relevance. Feedback from patients on readability was reviewed by the translation committee. The translation committee revised the questionnaire several times to enhance its comprehensibility.

Validation Study

Participants for the validation study were enrolled from outpatients visiting gastroenterology clinics. Overall, 5 institutes (Ewha Womans University, Mokdong Hospital, Konyang University Hospital, Seoul Samsung Medical Center, Gangnam Severance Hospital and Seoul Asan Medical Center), 4 located in Seoul and one located in Daejeon, were engaged in the survey for content validity and reliability. Each institute had 1 gastroenterologist who interviewed and examined the patients. Recruitment took 7 months from August 2011 to February 2012. During the first visit, the participating gastroenterologists inferred whether the patient had minimal comprehensibility to complete the questionnaire, and asked the relevant people (age, 19-70 years) whether they were willing to take the questionnaire. Those who provided verbal consent were provided with details of the aim and methodology of the study, and gave written consent. After the participants completed the questionnaire, the attending gastroenterologists, who were blinded to the participants' responses, interviewed the participants and subsequently filled out the questionnaire. The content validity of the self-administered Rome III-K was investigated using the physician-administered Rome III-K as the gold standard. The patients may have undergone endoscopy or ultrasound before their next visit. However, those who had started medical therapy or who already were in the course of medications that would affect gastrointestinal symptoms were excluded to avoid confounders. Mental health patients, alcohol or drug abusers, patients suffering from any medical disease that could affect comprehensibility or readability, such as dementia and stroke, were also excluded. The test-retest method was used to assess reliability of the Rome III-K. The retest was the same questionnaire that was completed by the enrolled persons at the second visit, usually 1 week after the first visit. Another survey for additional convergent validity and discriminant validity was performed in an independent study population. The clinical diagnosis judged by a physician (Jung HK), who was blinded to the patients' responses to the questionnaire, was used as the reference to assess convergent validity. We obtained responses for the Korean version of Short Form-36 Heath Survey questionnaire (SF-36-K) concurrently, which is known to be valid and reliable for measuring general health-related quality of life (HR-QOL).9 Participants diagnosed with an FGID according to the Rome III-K are supposed to have lower HR-QOL.10 Therefore, we conducted this concurrent validation to assess the validity of the Rome III-K indirectly through the validated SF-36-K.

Statistical Methods

The majority of Rome III-K question items were categorical or dichotomous items, and the test-retest reliability of the Rome III-K was assessed by Cronbach's α coefficient. Coefficient > 0.7 indicates acceptable consistency. The coefficient for each item is presented as a median with a 95% confidence interval. Convergent validity was judged with the physician-administered questionnaire as the reference. The self-administered questionnaire and physician-administered questionnaire have the same key outcome of the presence or absence of a FGID, which can be calculated according to the Rome III criteria. For example, we diagnosed IBS if the response of abdominal pain was present for the last 3 months (item 1), with a frequency of symptoms > 3 days per month (items 1 and 2) associated with 2 or more subsets of the following items (items 2-6) ranked > 1, and with symptom onset at least 6 months prior to the day of examination (item 1-1). The overall ratios of agreements for the presence of FGIDs between the 2 questionnaires were expressed as a percentage and verified using Cohen's κ coefficient. Convergent validity was expressed as percentage agreement and assessed statistically using McNemar's test. Another survey for additional convergent validity was assessed using the clinical diagnosis of a physician as the reference value. The Wald- Wolfowitz two-sample run test was used to determine whether the subjects without or with FGIDs had an identical SF-36 score distribution. If the overall rate of administered items was < 80% (< 28 of 35 questions), such data were defined as a low responder and excluded from any analyses.

Ethical Approval

All institutes engaged in the validation process approved this study via their own institutional review boards. All participants submitted a written informed consent form at enrollment.

Results

Demographic Characteristics

Overall, 297 subjects agreed to participate inthe reliability and content validity survey. Among them, 65 were excluded with the response rate of < 80%, a duration too long for the test-retest, or dropping out on the second visit.

Finally, 232 subjects were enrolled (median age, 47.5 years; range, 19-73 years; male:female, 1:1.7).

Feasibility

The questionnaire was developed at the sixth grade elementary school reading level. We measured the time required to complete the Rome III-K in a subset of participants, and the mean measured run-time was about 12 minutes (12.0 ± 4.0 minutes).

Reliability

The median visit interval between the test and retest was 7.0 days (range, 1-19 days). Almost all items had acceptable to excellent consistency with a Cronbach's alpha > 0.7 (range, 0.71-0.97) (Table 1). The median Cronbach's alpha of all IBS items was 0.73 (range, 0.51-0.87), which indicated excellent consistency; FD was 0.84 (range, 0.60-0.86); functional constipation was 0.83 (range, 0.73-0.86); functional diarrhea was 0.83; nausea was 0.83; vomiting was 0.81; and bloating was 0.89. Four questions of suboptimal value were related to IBS or FD. One question for IBS (features of accompanying decreased bowel movement) and one question regarding FD (epigastric pain not relieved by defecation) showed poor consistency.

Table 1

Cronbach's α Coefficient for Reliability of the Korean Version of Rome III Questionnaire for Functional Gastrointestinal Disorders

The data are presented as medians (95% confidence interval); FGIDs, functional gastrointestinal disorders; BM, bowel movement.

Validity

The degree of agreement between patient-administered and physician-administered questionnaires to diagnose FGIDs was > 95% for all kinds of disorders (Table 2). The κ index of each disorder also confirmed that all agreement degrees were significant. Subtyping of IBS or FD was accurate. Thirty-six of 194 participants were diagnosed with IBS in both patient-administered and physician-administered questionnaires, and the concordance rate of IBS subtyping was as high as 94.4% among IBS patients (34/36) (2 diarrhea-predominant patients were classified as mixed subtype according to the patient-administered questionnaire). Subtyping of patients with FD by self-reported responses was also highly concordant with the physician-completed results, as all corresponding patient subtypes were equivalent (17/17).

Table 2

Convergent Validity Judged Using a Physician-completed Questionnaire as the Reference

aP < 0.01 for all, by Cohen's κ.

We compared the physician's clinical diagnosis with the self-administered responses using a number of subjects (n = 74). McNemar's test verified that there was no significant difference for determining the presence or absence of all kinds of FGIDs in the population (Table 3). The percentage agreement between physicians' clinical diagnosis and diagnosis by patients' self-reported questionnaires was excellent: IBS (80.9%), FH (87.2%), functional constipation (90.9%), functional diarrhea (95.1%) and functional bloating (87.7%). However, a marked discrepancy was observed in the diagnosis of FD (34%), although the difference was not significant.

Table 3

Convergent Validity Judged Using Clinical Diagnosis as the Reference

aMcNemar's test verified that the clinical diagnoses by physician and self-diagnoses based on the Korean version of Rome III questionnaire were not significantly different.

Rome III-K was sufficiently valid to distinguish patients with FGIDs from normal subjects with higher HR-QOL as determined by validation using the SF-36-K. HR-QOL tended to be lower in patients suffering from FGIDs compared to that in subjects without FGIDs in 7 domains of the SF-36, with the exception of physical functioning (Table 4). Scores of patients with FD did not decline in some additional domains (role physical, bodily pain and social functioning). Age, gender ratio and presence of on-going medical problems were not significantly different between the normal group and each FGID; the FH group was significantly older than the normal group (52.5 ± 14.0 vs. 44.0 ± 14.7 years, P = 0.037 by Mann-Whitney test). Similar patterns of difference in HR-QOL were also observed when subjects with overlapping FGIDs were compared with non-overlapping patients. Seven of 8 SF-36 domains were reduced in patients with overlapping syndromes (Table 5). Age, gender ratio and the presence of on-going medical problems were not different between the 2 groups.

Table 4

Short Form-36 Health Survey Scores in Subjects Without and With the 3 Functional Gastrointestinal Disorders According to the Korean Version of Rome III Questionnaire

SF-36, Short Form-36 Health Survey; IBS, irritable bowel syndrome; FD, functional dyspepsia; FH, functional heartburn.

Wald-Wolfowitz two-sample run test (aP < 0.01 and bP < 0.05).

Table 5

Short Form-36 Health Survey Scores in Subjects Without and With Overlapping Functional Gastrointestinal Disorders

SF-36, Short Form-36 Health Survey.

aWald-Wolfowitz two-sample run test.

Discussion

We demonstrated that the Rome III-K is a reliable and valid self-reported tool for diagnosis of FGIDs. A validated questionnaire related to the diagnosis of FGIDs is crucial to assess the prevalence of FGIDs in epidemiological studies or to enroll patients in clinical trials. Systematic translation and independent back-translation are essential for accurate translation of the original source. Without such processes, cross-language studies are vulnerable to the risk that between-group differences, a different prevalence for example, are an artifact of differences in the adequacy of translation.5 We produced several reconciled versions and a reliable version of the Rome III questionnaire in Korean. We have demonstrated that the Rome III-K is valid for diagnosis of FGIDs.

Many symptoms related to FGIDs, including pain with bowel habit change, early satiation and heartburn, are difficult to express clearly in Korean and can be easily misunderstood. Anatomical misunderstanding between the epigastrium and substernal area is one example. The Rome III-K contains a small illustration to minimize such anatomical misunderstandings, and the satisfactory reliability shown by the test-retest method suggests that this questionnaire was translated accurately. However, the question "onset of pain associated with decreased frequency of stool" had poor reliability. This suboptimal result might be related to a lower positive response rate among patients. Cronbach's α was also < 0.6 for the question "epigastric pain not relieved by defecation." It was not clear whether this poor reliability was related to significant overlap with IBS. A Portuguese validation of the Rome III questionnaire for FD showed a similar level of reliability,11 whereas a Malay translation of the Rome III questionnaire for IBS had relatively higher test-retest reliability compared with that in the present study.12

Convergent validity was excellent when we used the physician-completed questionnaire as the reference. The percent agreement between the physician-completed and self-administered Rome III-K was > 97% for all FGID items. FD showed excellent agreement at 97.9%; however, that between clinical diagnosis and self-reported ROME III-K decreased to 66.0%. This reduction can be explained partly by the fact that the Rome III definition of FGIDs is stricter than physician's clinical diagnosis. The Rome III criteria were developed for selecting pathogenetically homogenous patients for research.13 Therefore, strict definitions of symptom onset and duration were included. The major changes in the Rome III criteria compared to the Rome II criteria were symptom onset and duration; these were intended to decrease recall bias. Interestingly, the reliability of symptom onset and duration of each IBS and FD subscale had a Cronbach's α coefficient > 0.8 in the present study. Therefore, these questions may be important for discrimination of FGIDs from organic gastrointestinal symptoms. The clinical diagnosis of FD showed a marked discrepancy with the self-reported diagnosis, as 34.0% of the study population was diagnosed differently. However, this discrepancy was minimized to 2.1% when we interviewed the participants and diagnosed FD based on the formal questionnaire. A questionnaire-based FD diagnosis may be far more accurate than a physician's diagnosis for several reasons. The Rome III criteria for epigastric pain syndrome (EPS) are relatively strict, and a physician is more likely to fail to remember the criteria without a formal questioning form. Additionally, a marked overlap in non-erosive reflux disease with EPS was reported recently in Korea, even though the authors used an insufficiently validated questionnaire.14 Heartburn or acid regurgitation is the most bothersome symptom in only 24.6% of patients with reflux esophagitis, according to a nationwide endoscopy-based study in Korea.15 More than 13% of patients with reflux esophagitis complained of epigastric pain as the most bothersome symptom in that report. Therefore, it is difficult to distinguish FH from EPS according to the Rome III criteria in Koreans complaining of epigastric pain, and our data suggest a relatively low accuracy for the clinical diagnosis of FD. The Rome III-K contained only questions related to typical symptoms, heartburn and acid regurgitation, for the diagnosis of FH. In contrast, the absence of extraesophageal symptoms is the main disadvantage of the Rome III-K. Therefore, we developed questions regarding extraesophageal symptoms, including cough, asthma, globus pharyngitis, hoarseness and chest pain, to enhance the sensitivity of FH; however, these items have yet to be validated. These data confirm the convergent validity of the Rome III-K. The substantial degree of agreement with clinical diagnosis is comparable to questionnaires in other Asian languages.12

Discriminant validity is another pillar of construct validity, as introduced by Campbell.16 However, no standardized value is available for discriminant validity. In addition, it is theoretically difficult to validate when there is no previously validated tool that measures the same concept or disorder. Because the Rome III-K is the first systematically translated Korean questionnaire for diagnosis of FGIDs, we adopted a concurrent validation process. The previously validated SF-36-K can discriminate low HR-QOL in participants suffering from FGIDs. One report of a Korean population demonstrated that the HR-QOL is significantly impaired in subjects with IBS, uninvestigated dyspepsia and FH; this finding is consistent with data from other countries.10 Another study from Japan demonstrated worse HR-QOL in patients with an overlap of these 3 diseases.17 In the present study, almost all SF-36 domains were impaired in participants who were diagnosed with one of these FGIDs according to the Rome III-K. Furthermore, participants with overlapping disorders tended to reveal a worse QOL. These results suggest that the Rome III-K can discriminate the 2 groups of participants. The weakness of our study is the lack of demographic data regarding QOL (marital status, presence of religion, level of education, or economic status), which may be a confounder regarding discriminant validity.

We conclude that the Rome III-K questionnaire, developed using a structural translational process, had acceptable test-retest reliability and good construct validity. Nationwide epidemiological studies and clinical trials should now be conducted using this first Korean instrument for diagnosis of FGIDs.