Validation of health screening questionnaire used for screening gastrointestinal disorder in worker's special health examination for night time work.

BACKGROUND: Since the night time work was introduced as a 'harmful factor' for the worker's special health examination (WSHE) in 2014, the validation of the questionnaire used for screening gastrointestinal (GI) disorder has not been conducted. The purpose of this study is to verify the validity of the questionnaire using the data of specific health screening cluster.
METHODS: We used WSHE screening data for 3 years, from 2014 to 2016, in health screening cluster. The subjects who had received upper GI endoscopy in opportunistic screening and WSHE simultaneously regardless of the results of the questionnaire were selected. We tested the validity of the questionnaire using upper GI endoscopy as a gold standard.
RESULTS: This study was conducted on 5,057 examinees in 2014, 8,352 examinees in 2015, and 10,587 examinees in 2016. The validity of the questionnaire for each year was as follows: sensitivity 12.3% (95% confidence interval [CI], 11.1-13.4), specificity 88.6% (95% CI, 87.2-90.1), accuracy 41.1% (95% CI, 39.8-42.5) in 2014, sensitivity 5.9% (95% CI, 5.2-6.5), specificity 93.6% (95% CI, 92.7-94.4), accuracy 38.6% (95% CI, 37.6-39.6) in 2015, sensitivity 6.0% (95% CI, 5.4-6.5), a specificity of 9.42% (95% CI, 93.4-95.0), accuracy of 34.2% (95% CI, 33.3-35.1) in 2016. In generally, questionnaire showed sensitivity of 10%, specificity of 90%, and accuracy of 40%.
CONCLUSIONS: Despite the purpose of WSHEs aiming to identify target disease early, the sensitivity of the questionnaire for GI disease was too low as 10%. The reasons for this are the problem of the question itself, and the problem of ambiguous target disease. In the future, the questionnaire should be improved to meet the purpose of the WSHE, and further correction of the target disease should be made.

BACKGROUND

Night time work is related to several health and safety issues. In addition to causing work accidents [1], night time work also causes breast cancer [2], sleep disorders [3], cardiovascular disease [4], and type 2 diabetes [5]. Night time work may also cause disorders of digestive enzyme secretion [6], a disorder of acid-base balance, and eventually gastrointestinal (GI) disorder such as peptic ulcer and reflux disease [67]. Previous epidemiological studies also consistently report the association of night time work and GI disorder [8]. Moreover, functional GI disorders have also been reported to be associated with night time work [9].

Those adverse effects of night time work let the Korean government has added ‘night time work’ as a harmful factor of worker's special health examination (WSHE) since 2014 as a part of the policy to protect workers [10]. The purpose of WSHE is to prevent and make an early diagnose the occurrence of nervous, cardiovascular, GI, and endocrine disorder in workers who work at night. The WSHE consists of 2 steps. First, screening is done through the questionnaire items, and the second test is conducted when an abnormality is found in this item. In the case of GI disorder, the same method is used, and the second test of GI disorder is upper GI endoscopy. Although the first screening test of the WSHE is based on the questionnaire, it has not been tested whether the items in the questionnaire correctly plays a role in screening the target diseases since WSHE for night time work introduced in 2014. Therefore, in this study, we focus on the GI disorder and obtain the validity of the screening questionnaire item of the GI disorder (health screening questionnaire GI disorder [HSQGI]).

METHODS

Background

The WSHE conducted for night time worker is based on the method of selecting the subject of the second examination (upper GI endoscopy) after the first examination using the questionnaire and medical interview. Therefore, upper GI endoscopy is not performed for subjects who have indicated that they have no abnormalities in the HSQGI, generally. However, the health screening cluster where this study was conducted has a unique program called opportunistic screening’, and this health checkup performed upper GI endoscopy without specific conditions. Using these advantages of the health screening center, we selected the subjects who performed the Opportunistic screening and WSHE simultaneously. Using the results of both the first examination of WSHE and the upper GI endoscopy of opportunistic screening, we were able to obtain the validity of HSQGI.

Study population

The results of HSQGI and basic information among WSHE for 151,301 night shift workers performed in 7 centers (Jongno, Yeouido, Gangnam, Suwon, Gwangju, Daegu, and Busan) from 2014 to 2016 were collected. The results of each year's screening were considered to be independent because of the relatively short duration of treatment and due to the nature of GI disorder that respond well to treatment. Each study was considered as an individual study. The study was conducted for examinees who received opportunistic screening and WSHE simultaneously. Subjects who did not have adequate upper GI endoscopy due to residual food in stomach or overreacting by sedatives were excluded from the study. In addition, we also excluded cases in which the answer to the questionnaire was missing value. Finally, 5,057 examinees in 2014, 8,352 examinees in 2015, 10,587 examinees in 2016 were included in the main study. The selection of the final subject according to exclusion criteria is shown in Fig. 1.

Fig. 1

Flow chart of subject selection.

WSHE: worker's special health examination; GI: gastrointestinal; HSQGI: health screening questionnaire gastrointestinal disorder.

HSQGI decision logic

At the first screening examination, a medical questionnaire and a physician's medical interview are performed. If the examinee answers “dyspepsia” in the questionnaire, it will lead to upper GI endoscopy which is the secondary screening examination. The above-mentioned “dyspepsia” refers to a case where one of the following 3 conditions is met in HSQGI [10].

• Bothersome postprandial fullness, occurring after ordinary-sized meals lasts more than 6 months and frequency is more than 2 days per week

• Early satiation that prevents finishing a regular meal persists for more than 6 months and frequency is more than 2 days per week

• Abdominal pain, sore symptoms lasting more than 6 months, frequency more than one day a week.

The full item of HSQGI can be found in Appendix 1.

Endoscopic findings

Upper GI endoscopy results were classified as ‘non-specific’ and ‘GI disorder’ as 2 variables. The classification of upper GI endoscopy results was based on the criteria of Table 1. The criterion table was prepared by reflecting opinions of 4 physicians who are well-experienced in the judgment of WSHE for night time workers. In the case of gastritis, simple superficial gastritis, or simple erythematous gastritis, which does not cause any specific symptoms, is classified as non-specific when there is no other abnormal findings.

Table 1

Classification of endoscopic findings

Non-specific		GI disorder
Normal	Submucosal tumor (benign)	Erosive gastritis	Acute gastric mucosal lesion
Glycogen acanthosis	Esophageal candidiasis	Raised erosion	Gastritis
Ectopic pancreas	Esophageal subepithelial lesion	Chronic gastritis	Duodenitis
Vocal cord nodule	Paraesophageal hernia	Inflammatory polyp	Gastric/duodenal ulcer
Laryngeal papilloma	Esophageal diverticulum	Barrett's esophagus	Malignancy
Laryngeal lipoma	Gastric diverticulum
Xanthoma	Post gastrectomy
Esophageal papilloma	Post subtotal gastrectomy

GI: gastrointestinal.

Statistical analysis

Sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) of the WSHE items were obtained to evaluate whether the HSQGI could be used as a screening tool for GI disorder. Sensitivity was determined by the percentage of those who had GI disorder findings on the endoscopic examination and those who answered “dyspepsia” to the HSQGI. The specificity was calculated as the ratio of the subjects who showed non-specific findings on the endoscopic examination and those who responded normally to the HSQGI. The accuracy was determined by the percentage of subjects (true positive and true negative) who matched the results of the endoscopic examination and the HSQGI. PPV and NPV were obtained using the following formula:

The prevalence was calculated as the ratio of the subjects who showed ‘GI disorder’ findings on upper GI endoscopy. All analyses were conducted using SAS statistical software, version 9.4 (SAS Institute Inc., Cary, NC, USA).

Ethics statement

The original study was approved by the Institutional Review Board (IRB) of Seoul St. Mary's Hospital (IRB No.: KC17RESI0477).

RESULTS

The general characteristics of the study population are as follows Table 2. The average age of the examinees in each year was 40.3 years in 2014, 41.5 years in 2015, 42.9 years in 2016. Among the target population, males accounted for 80%, and about one third of the respondents said that they are currently smoking. The proportion of people who drank over a week was 34.0% in 2014, 34.2% in 2015 and 29.5% in 2016. The average working time at the current job was about 9 years, and the percentage of people who worked for 12 years or more was the highest.

Table 2

General characteristics of the examinees of WSHE performed in specific health screening clusters

Characteristics		2014	2015	2016
Age (years)		40.1 ± 10.0	41.2 ± 9.6	42.4 ± 9.4
	20–29	762 (15.1)	1,024 (12.3)	1,050 (9.9)
	30–39	1,703 (33.7)	2,670 (32.0)	2,996 (28.3)
	40–49	1,616 (32.0)	2,742 (32.8)	4,134 (39.1)
	Above 50's	976 (19.3)	1,916 (22.9)	2,407 (22.7)
Sex
	Male	4,105 (81.2)	7,223 (86.5)	8,776 (82.9)
	Female	952 (18.8)	1,129 (13.5)	1,811 (17.1)
Smoking
	Never-smoker	1,837 (36.7)	2,820 (34.1)	3,859 (36.7)
	Ever-smoker	1,283 (25.7)	2,469 (29.9)	3,139 (29.9)
	Current-smoker	1,880 (37.6)	2,978 (36.0)	3,517 (33.5)
Alcohol*
	Adequate	3,274 (65.5)	5,358 (64.8)	7,312 (69.5)
	Overdrink	1,726 (34.5)	2,909 (35.2)	3,203 (30.5)
Employment period (years)		9.4 ± 8.2	9.9 ± 8.9	9.3 ± 8.6
	Below 2	810 (16.0)	1,511 (18.1)	2,042 (19.3)
	2–5	1,120 (22.2)	1,736 (20.8)	2,156 (20.4)
	5–12	1,453 (28.7)	2,078 (24.9)	2,654 (25.1)
	Over 12	1,674 (33.1)	3,027 (36.2)	3,735 (35.3)
Total		5,057	8,352	10,587

Data are expressed as mean ± standard deviation or number (%).

WSHE: worker's special health examination.

*If the amount of alcohol consumed per week is over soju (Korean traditional alcohol) 2 bottles for men under 65 years old, 1 bottle for men over 65 years old and for women under 65 years old, and 0.5 bottle for women under 65 years old, defined overdrinking. And when the amount of drinking per week was less than that, it was classified as “appropriate”.

Table 3 shows the distribution of upper GI endoscopy abnormalities according to HSQGI responses. In all 3-year screenings, the number of people diagnosed with GI disorder by endoscopy was higher than those with Non-specific diagnoses. Nonetheless, the number of people confirmed with HSQGI dyspepsia was significantly less than that of normal people. Among the people who were confirmed with dyspepsia in HSQGI, the percentage of GI disorder was higher than those who were confirmed as non-specific in all 3-year screenings. The validity of the HSQGI based on the endoscopy is shown in Table 4.

Table 3

Distribution of GI disorder identified by upper GI endoscopy according to HSQGI results, 2014–2016

HSQGI		Upper GI endoscopy		Total
		Non-specific	GI disorder
2014
	Normal	1,694 (33.5%)	2,760 (54.6%)	4,454
	Dyspepsia	217 (4.3%)	386 (7.6%)	603
		1,911	3,146	5,057
2015
	Normal	2,916 (34.9%)	4,928 (59.0%)	7,844
	Dyspepsia	201 (2.4%)	307 (3.7%)	508
		3,117	5,235	8,352
2016
	Normal	3,193 (30.2%)	6,768 (63.9%)	9,961
	Dyspepsia	196 (1.9%)	430 (4.1%)	626
		3,389	7,198	10,587

GI: gastrointestinal; HSQGI: health screening questionnaire gastrointestinal disorder.

Table 4

Validity of HSQGI in WSHE

Year	Sensitivity	Specificity	Accuracy	PPV	NPV	AUC value
2014	12.3% (11.1–13.4)	88.6% (87.2–90.1)	41.1% (39.8–42.5)	64.0% (60.2–67.8)	38.0% (36.6–39.5)	0.505
2015	5.9% (5.2–6.5)	93.6% (92.7–94.4)	38.6% (37.6–39.6)	60.4% (56.2–64.7)	37.2% (36.1–38.2)	0.503
2016	6.0% (5.4–6.5)	94.2% (93.4–95.0)	34.2% (33.3–35.1)	68.7% (65.1–72.3)	32.1% (31.1–33.0)	0.501

HSQGI: health screening questionnaire gastrointestinal disorder; WSHE: worker's special health examination; PPV: positive predictive value; NPV: negative predictive value; AUC: area under curve.

The sensitivities of ‘HSQGI’ for GI disorder confirmed by Upper GI endoscopy were 12.3% (95% confidence interval [CI], 11.1–13.4), 5.9% (95% CI, 5.2–6.5) and 6.0% (95% CI, 5.4–6.5), in 2014, 2015 and 2016, respectively. On the other hand, the specificity is relatively high at almost 90%, which is 88.6% (95% CI, 87.2–90.1), 93.6% (95% CI, 92.7–94.4), and 94.2% (95%:CI, 93.4–95.0), in 2014, 2015 and 2016, respectively.

Accuracy is reported to be 41.1% (95% CI, 39.8–42.5) in 2014, 38.6% (95% CI, 37.6–39.6) in 2015, and 34.2% (95% CI, 33.3–35.1) in 2016. Despite the high specificity due to the relatively low sensitivity, the accuracy was about 40%.

The usefulness, i.e., PPV and NPV of the HSQGI were 64.0% (95% CI, 60.2–67.8) and 38.0% (95% CI, 36.6–39.5) in 2014, 60.4% (95% CI, 56.2–64.7) and 37.2% (95% CI, 36.1–38.2) in 2015, and 68.7% (95% CI, 65.1–72.3), and 32.1% (95% CI, 31.1–33.0) in 2016, respectively. PPV and NPV were in the almost 60% and 40%, respectively.

DISCUSSION

The validity test for the HSQGI used in WSHE was performed with the medical records and questionnaire from 5,057, 8,352, and 10,587 subjects who received both upper GI endoscopy in opportunistic screening and WSHE simultaneously. The HSQGI in WSHE showed a sensitivity of 10%, a specificity of 90%, an accuracy of 40%, a PPV of 60% and a NPV of 30% for predicting the target diseases in upper GI system.

Among the validity of HSQGI, the specificity was high value of 90%, but the sensitivity was low as 10% so as not to meet the purpose of screening. A previous study for the validity of questionnaires through endoscopic results reported that the symptom questionnaire had a 92% sensitivity and 19% specificity for Gastroesophageal reflux disease [11], in addition, the other study reported the sensitivity and specificity of the symptom questionnaire (GerdQ) for Gastroesophageal reflux disease was 71.7% and 72,2% respectively [12]. The sensitivity and specificity of the symptom questionnaire, which targets peptic ulcer in the same way, has been reported to be 88% and 61%, respectively [13]. There might be several reasons for those low sensitivity of questionnaire in WSHE in the present study. We can first consider the high prevalence of chronic gastritis in Korea. According to a study of the subjects who visited for the purpose of health checkup, 85.9% of them had more than one endoscopic ‘gastritis’ finding [14]. High-prevalence of gastritis among Koreans is consistently reported in previous studies, which is attributed to high rate of Helicobacter pylori infection, high salt intake and excessive drinking. Many of these endoscopic gastritis is asymptomatic, and it is sufficient to observe without further treatment. However, this characteristic of asymptomatic gastritis confirmed by upper GI endoscopy was not considered in this study and it increased false negative by being classified as abnormal. If a definition of gastritis as a GI disorder was limited to gastritis requiring immediate treatment, or H. pylori-associated gastritis, sensitivity may have increased. However, the data used in this study did not contain such additional information in endoscopic findings. Second, a functional dyspepsia (FD) can be considered for increasing numbers of negative results. The design of this study used endoscopic findings as a gold standard in defining the GI disorder, which means that although FD is a GI disorder that WSHE wanted to screen, many FD cases misclassified to false positives because of their non-specific endoscopic findings. If the clinical procedure used to diagnose FD, which excludes the possibility of organic causes at first, as a gold standard in this study, the results may have changed. The cases that were classified as false positive will be reclassified as true positive, thus the sensitivity may increase. However, since this method is a clinical diagnostic process for symptomatic patients used in practical field, using that in WSHE does not satisfy the condition that screening should be considered cost-effective. Third, actual clinical process may be different from the guidelines. In this study, the guidelines of the Occupational Safety and Health Research Institute, were applied as HSQGI decision logic, as mentioned in the method above [10]. However, this is only a guideline, and the final judgment at the actual clinical field is left entirely to the physician in charge. Therefore, even if the subject responds at the HSQGI that he or she has no symptoms, which do not comply with the positive criteria in the guideline, an endoscopy may be implemented if the physician who has had the examination thinks that an endoscopy is necessary for the subject. Besides, in such cases, it is more likely be positive findings than the endoscopic cases conducted only with HSQGI. Thus, if these cases had been considered in study, the sensitivity might have increased. Nevertheless, despite these limitations, in this study, to reduce variability among physicians who examine subject in actual clinical fields as much as possible, a standard guideline was selected to HSQGI decision logic.

Nevertheless, it is obvious that the sensitivity of HSQGI is too low and the item needs to be improved. If so, the question is how much sensitivity should be increased. Generally, in order to increase the sensitivity, the specificity should be sacrificed, so the appropriate level is determined considering the severity and curability of the target disease [15]. The target disease of WSHE is GI disorder which includes gastric cancer to functional dyspepsia, which coverage is very wide and vague. For example, to screen for gastric cancer, sensitivity should be significantly increased even if the specificity is lowered because the false negatives should be reduced. However, sensitivity is not required for other target diseases, gastritis or functional dyspepsia as much as gastric cancer. Therefore, it is important to improve the questionnaire, but it should be preceded by clearly selecting the target disease of WSHE. Furthermore, these target diseases should include diseases such as ulcer, reflux, and functional dyspepsia that proven to be associated with night time work. However, of these 3 diseases, the 2 preceding diseases, i.e., the organic dyspepsia, are clearly diagnosed and the treatment method is reliable, so there is no problem in inclusion in the target disease. However, in the case of FD, it is necessary to reconsider whether it is included as target disease. FD does not meet the conditions for successful screening because pathophysiology is not well established and no treatment has been established [16]. In addition, the psychosocial factors such as stress are more likely to cause the disease [17], so rather than treating it as GI disease, it may be better to treat as a nervous system disease such as sleep disorder or a separate category. If this separation is made, the target disease of the GI disorder of WSHE can be clearly identified as an organic disease such as ulcer or reflux, and the sensitivity target and the item can be improved accordingly. In this study, we evaluated again the validity of HSQGI with reflux disease and ulcer as target disease. which was consistent low with the sensitivity of 10% (Appendixes 2 and 3). The improvement of the question seems to be certainly needed, and it can be based on many questionnaires already developed [111213]. For example, in order to rule out reflux disease, a GerdQ questionnaire that has already been validated through many studies may be added to the current HSQGI. While current HSQGI asked about a sore symptom vaguely thorugh only one question, the addition of the GerdQ questionnaire may provide a more specific approach to reflux symptoms by asking about sleep discomfort caused by symptoms, whether they are injected by symptoms, and how often they are reversed [12]. It may also be considered for the addition of specific question about family history related to ulcers disease or whether the symptoms are relieved by the ordinary meal as in the GLADYS questionnaire [13].

Some endoscopic findings of chronic gastritis such like simple superficial gastritis and simple erythematous gastritis were excluded. As mentioned above, we expected that false negatives would be overestimated due to the high prevalence of chronic gastritis in Koreans. Simple superficial gastritis alone does not cause any significant symptoms and is sufficient for follow-up observation without special treatment. These diseases are classified as non-specific because they are not considered to be within the GI disorder range that WSHE wants to find.

This study was conducted for those who receive opportunistic screening and WSHE at the same time. Unlike WSHE, opportunistic screening is not a mandatory test prescribed by law, and is a type of 'welfare within the company' provided by employers for the health of workers. Therefore, companies offering opportunistic screening are more likely to be workers-friendly than companies that do not, and more likely to be with well-established systems for worker health. Considering these features of opportunistic screening, bias may have occurred in the selection process of the subject. To determine whether this bias was present, we conducted a subgroup analysis. The best method for subgroup analysis is to compare the validation obtained by dividing into 2 groups: opportunistic screening group and only-WSHE group. However, as described above, validation can not be obtained in only-WSHE group because GI endoscopy is not performed unless an abnormality is found in HSQGI. Therefore, we analyzed the size of the business by considering it as a proxy for the degree of worker health friendliness of the company. In previous study [18], it was found that the health status of workers varied according to the size of the workplace. The analysis conducted by dividing workplaces with less than 50 employees, workplaces with more than 50 employees and less than 300 employees, workplaces with more than 300 employees, showed 10% lower sensitivity and 90% higher specificity as like the main result of this study (Appendix 4). It may reflect that the welfare-friendly aspects of opportunistic screening do not bias the research results.

The HSQGI consists of 6 individual items, however, we did not receive the data for each of the 6 individual items of HSQGI, and only data on dyspepsia according to the HSQGI logic mentioned above were received due to data registry of health screening cluster. Therefore, the accuracy of the subtypes of the diseases each item was seeking could not be determined.

This study was also not performed on all subjects who performed WSHE in Korea. Nonetheless, the tests were performed in 7 multi-center health examination clusters distributed across nationwide and might be a sufficient to show representative with sufficient subject numbers.

The endoscopic findings as the gold standard in this study were not a histopathological findings but a descriptive findings of the practitioner. Despite the high utility of upper GI endoscopy, there are differences among practitioners for unclear lesions [19]. Because of these limitations, biopsy is performed for definitive diagnosis [20]. However, biopsy was not performed for every procedure in the upper GI endoscopy performed in this study.

The WSHE for night time workers was conducted for 3 years after the implementation. It is the first study to obtain the validity of the questionnaire screening for GI symptoms in WSHE. In addition, this study did not have to integrate the 3-year test because the number of subjects per year was sufficient enough to consider each year test as a separate test.

CONCLUSIONS

Since 2014, WSHE has been implemented for night time workers, but there was no validity test for the screening tools used in WSHE. In this study, the validity of the diagnostic tool as a target disease for GI disorder was determined and low sensitivity was obtained that did not fit the purpose of screening. In order to achieve the purpose of screening, it is necessary to develop a more sensitive item with a clear selection of target disease.