Reliability and Validity of a Nationwide Survey (the Korean Radiation Workers Study).

BACKGROUND: This study aimed to evaluate the reliability and validity of the self-administered questionnaire for Korean radiation workers.
METHODS: From May 24, 2016, to June 30, 2017, 20,608 participants completed the questionnaire, providing information on sociodemographics, lifestyle, work history and practices, medical radiation exposure, and medical history, which was linked to the National Dose Registry and the National Cancer Registry. The validity of the questionnaire was evaluated using the responses of 20,608 workers, and reliability was evaluated using the responses of 3043 workers who responded to the survey twice.
RESULTS: Responses concerning demographic characteristics and lifestyle showed reliability with a moderate-to-high agreement (kappa: 0.43-0.99), whereas responses concerning occupation and medical radiation exposure had a wide range of agreement (kappa: 0.05-0.95), possibly owing to temporal variability during employment. Regarding validity, responses to the question about the first year of employment had an excellent agreement with the national registry (intraclass correlation coefficient = 0.9); however, responses on cancer history had a wide range of agreement (kappa: 0.22-0.85).
CONCLUSION: Although the reliability and validity of the questionnaire were not distinguished by demographic characteristics, they tended to be low among participants whose occupational radiation exposure was minimal. Overall, the information collected can be reliable for epidemiological studies; however, caution must be exercised when using information such as medical exposure and work practices, which are prone to temporal variability.

Introduction

A self-administered survey is an important tool and a common approach to collect data in public health research because it is easy to use, and it is feasible and inexpensive to apply [1]. Although it is subject to several biases such as recall bias, nonresponse bias, and social desirability bias [2,3], when correctly used, it can be useful in obtaining surrogate measures of occupational or environmental exposure to hazardous substances [4] and risk confounders such as smoking and socioeconomic status, thereby enabling more precise risk assessment of hazards of interest.

Ionizing radiation is a well-known human carcinogen, and its health effects have been reported over the last 100 years [5]; however, the health effects in a low dose range (≤100 mSv, regardless of whether the dose is received in a single exposure or cumulative exposure) are unclear owing to not only a limited sample size but also uncertainties about exposure assessment (e.g., unrecorded doses and measurement errors) and limited information about potential confounders. In this regard, studies on radiation workers who are typically exposed to low dose radiation are essential, and the collection of reliable information, including surrogate measures of radiation exposure and potential confounders, is vital to estimate more precise radiation-induced health risks. This information can be obtained through surveys or national registries and can be used for dose reconstruction, organ dose estimation, and potential confounders in risk estimation. However, among the published studies on radiation workers to date, few have collected the abovementioned information; most were concerned with medical diagnostic fields [6,7]. Limited information is available concerning nuclear workers [[8], [9], [10]], although they have various work procedures depending on their workplace. In addition, there is a lack of evaluation of the reliability and validity of the information collected through self-administered surveys for epidemiological studies.

Previously, a cohort of Korean radiation workers in various occupations was constructed in a 2016-2017 nationwide survey. Comprehensive individual information was obtained through the survey, including demographic characteristics, occupational characteristics, and lifestyle factors. The data were then linked with the National Dose Registry and the National Cancer Registry [11]. The present study evaluated the reliability and validity of the self-administered questionnaire and identified key demographic and occupational characteristics associated with its reliability and validity.

Materials and methods

Study population

This study was derived from a nationwide survey for the Korean radiation workers study; the details of the study design and method of participant recruitment have been described in the study protocol and the baseline study [11,12]. In brief, a nationwide survey was conducted between May 24, 2016, and June 30, 2017—through the annual mandatory radiation safety education courses for radiation workers—to enroll study participants in the cohort and collect baseline information of individual workers, including demographics, occupational characteristics, and lifestyle factors. Among the 32,157 workers who responded to the survey, 20,608 were identified in the National Dose Registry and the National Cancer Registry, which are used to evaluate the validity of survey responses. Of those 20,608 workers, 3043 responded to the survey twice, and their data were used to evaluate the reliability of survey responses. The average time interval between the first and the second survey was 231 days (range: 3–399 days), and nearly all participants (98.6%) had a time interval greater than 31 days. This study received ethical approval from the institutional review board of the Korea Institute of Radiological and Medical Sciences (K-1603-002-034), and all study participants provided written informed consent. All study procedures were carried out in accordance with relevant guidelines and regulations.

Data collection

A self-administered questionnaire was developed by referring to previous studies about Korean [13] and American [14] radiologic technologists. The questionnaire comprised 20 items, including work history (e.g., calendar year of hire and employment duration), work practices (e.g., wearing a dosimeter, separation of workplace from radiation source, and wearing protective equipment), medical radiation exposure (e.g., X-ray, intraoral or panoramic radiography, and computed tomography), medical history (e.g., cancer, hypertension, cataract, and diabetes), lifestyle (e.g., smoking status and alcohol consumption), and demographics (e.g., sex, marital status, and educational level). The survey data were linked to personal dose equivalents of individuals, which is the dose equivalent in tissue at a depth of 10 mm in the body and is noted as Hp(10). Data were collected from 1984 to the first quarter of 2017, including information about facility-based occupations from the National Dose Registry, and cancer incidence data (International Classification of Diseases-10 codes) from 1988 to 2017 from the National Cancer Registry via individual personal identification numbers. The sources of dosimetry and cancer incidence data are detailed in the baseline study [11].

Statistical analysis

Reliability refers to the agreement between two responses (i.e., the first and the second survey at different time points) of the same questionnaire [15]. Reliability for categorical variables between the first and second survey responses was measured by percentage agreement and Cohen's kappa statistics [16]. Weighted kappa statistics were used for responses considered as ordinal variables [17]. A kappa value of 0–0.2 was considered as a poor agreement, 0.2–0.4 a fair agreement, 0.41–0.6 a moderate agreement, 0.61–0.8 a substantial agreement, and 0.81–1.0 an almost perfect agreement [18]. For continuous variables (e.g., calendar year of hire and employment duration), we calculated the intraclass correlation coefficient (ICC) [19]. An ICC value of 0–0.39 was considered as a poor agreement, 0.4–0.59 a fair agreement, 0.6–0.74 a good agreement, and 0.75–1.0 an excellent agreement [20].

Validity refers to the accuracy of survey responses [15]. In this study, survey questions about the first year of employment and cancer history (top five incidence cancers in Korea), which can be identified from the national registries, were used for the evaluation of validity using the same methods as for the reliability analyses. In addition, sensitivity, specificity, positive predictive value, and negative predictive value were calculated to measure the degree of accuracy of survey responses to medical history [21]. Statistical analyses were performed using SAS 9.4 (SAS Institute, Cary, NC) and SPSS 23 (IBM Corp., Armonk, NY, USA).

Results

Demographic characteristics

Participants' characteristics are summarized in Table 1. Most participants were men (87%), and more than one-third were in their 30s. Overall, the characteristics of participants who were included in the reliability analysis did not deviate highly from those of the total enrolled participants (i.e., participants included in validity evaluation), although the proportion of industrial radiographers was higher than that of nuclear power plant workers in the data set for reliability evaluation.

Table 1

Participants' characteristics per the evaluation of reliability and validity

Characteristics∗	Reliability evaluation (n = 3043)		Validity evaluation (n = 20,608)
	n	%	N	%
Sex
Men	2647	87.0	17,831	86.5
Women	396	13.0	2777	13.5
Age (years)
≤29	709	23.3	5171	25.1
30–39	1180	38.8	7441	36.1
40–49	742	24.4	4857	23.6
≥50	412	13.5	3139	15.2
Occupation
Public institution	192	6.3	676	3.3
Education institution	147	4.8	2010	9.8
Military	34	1.1	165	0.8
Industrial radiography	811	26.7	3517	17.1
Industry	537	17.6	3886	18.9
Research institution	105	3.5	1139	5.5
Nuclear power plant	709	23.3	6328	30.7
Medical institution	508	16.7	2887	14.0
Calendar year of hire
Mean (standard deviation)	2006.85 (9.41)		2007.91 (8.94)
Occupational radiation exposure†
Exposed	2070	68.0	12,707	61.7
Unexposed	973	32.0	7901	38.3

Information was identified from the National Dose Registry.

Cumulative personal dose equivalent greater than 0.1 mSv (≤0.1 mSv was recorded as “below recording level”) was considered “exposure.”

Reliability

The reliability of the demographics and lifestyle questions is presented in Table 2. The kappa values indicated substantial to almost perfect agreement, except for the questions about secondhand smoke, physical exercise, and sleeping hours, which showed relatively low reliability (kappa values 0.43–0.55), indicating moderate agreement.

Table 2

Reliability of the questions on demographic and lifestyle factors

Questionnaire item	First response		Second response		Agreement	Kappa statistic (95% CI)
	n	%	n	%
Sex
Men	2611	85.8	2625	86.3	97.6%	0.99 (0.98–1.00)
Women	387	12.7	397	13.1
Missing	45	1.5	21	0.7
Educational level
High school or below	751	24.7	760	25.0	94.3%	0.95 (0.94–0.96)
College or above	2206	72.5	2246	73.8
Missing	86	2.8	37	1.2
Marital status
Single	1216	40.0	1182	38.8	93.7%	0.94 (0.93–0.96)
Married	1747	57.4	1825	60.0
Missing	80	2.6	36	1.2
Smoking status
Nonsmoker	1126	37.0	1146	37.7	87.4%	0.82 (0.80–0.84)
Ex-smoker	561	18.4	575	18.9
Current smoker	1331	43.7	1300	42.7
Missing	25	0.8	22	0.7
Secondhand smoke
No	626	20.6	665	21.9	77.5%	0.43 (0.39–0.47)
Yes	2350	77.2	2314	76.0
Missing	67	2.2	64	2.1
Alcohol consumption
No	453	14.9	469	15.4	91.5%	0.71 (0.67–0.75)
Yes	2571	84.5	2558	84.1
Missing	19	0.6	16	0.5
Regular exercise
No	1317	43.3	1348	44.3	75.8%	0.55 (0.52–0.58)
Yes	1679	55.2	1652	54.3
Missing	47	1.5	43	1.4
Sleeping hours
≤6 hours	1239	40.7	1281	42.1	62.9%	0.49 (0.46–0.52)∗
7 hours	1210	39.8	1211	39.8
≥8 hours	577	19.0	539	17.7
Missing	17	0.6	12	0.4

CI, confidence interval.

Weighted kappa statistics.

The reliability of the questions for work history, including calendar year of hire, employment duration, and employment status, indicated excellent agreement with ICC or kappa values ranging from 0.84 to 0.95, whereas the reliability of most questions for work practices including the use of radiation sources and wearing protective equipment indicated fair-to-moderate agreement (Table 3, Table 4).

Table 3

Reliability of the questions on work history

Questionnaire item	First response		Second response		ICC (95% CI)
	Mean	SD	Mean	SD
Calendar year of hire	2006.63	9.29	2006.79	9.21	0.95 (0.95–0.96)
Employment duration∗ (years)	8.84	8.68	8.40	8.59	0.95 (0.94–0.95)

CI, confidence interval; ICC, intraclass correlation coefficient; SD, standard deviation.

The last year of employment was set for 2016.

Weighted kappa statistics.

Table 4

Reliability of the questions on work practices

Questionnaire item	First response		Second response		Agreement	Kappa statistic (95% CI)
	n	%	n	%
Radiation source
Sealed isotope	888	29.2	990	32.5	60.3%	0.60 (0.58–0.63)
Unsealed isotope	289	9.5	313	10.3
Radiation-generating device	766	25.2	774	25.4
Not sure	510	16.8	402	13.2
None	330	10.8	345	11.3
Missing	260	8.5	219	7.2
Distance from radiation source
<1 m	505	16.6	497	16.3	63.6%	0.57 (0.54–0.60)∗
1 to <3 m	638	21.0	697	22.9
≥3 m	1651	54.3	1705	56.0
Missing	249	8.2	144	4.7
Wearing a dosimeter
Always	1823	59.9	2155	70.8	57.0%	0.39 (0.35–0.43)∗
Sometimes	342	11.2	313	10.3
Almost never	368	12.1	171	5.6
Missing	510	16.8	404	13.3
Separation of workplace from radiation source
Yes	1104	36.3	1189	39.1	39.1%	0.44 (0.39–0.48)
No	1024	33.7	1025	33.7
Missing	915	30.1	829	27.2
Use of lead apron
Never	1671	54.9	1728	56.8	52.3%	0.49 (0.46–0.53)∗
≤74%	490	16.1	495	16.3
≥75%	336	11.0	342	11.2
Missing	546	17.9	478	15.7
Use of lead goggles
Never	1918	63.0	1983	65.2	50.6%	0.33 (0.28–0.38)∗
≤74%	276	9.1	321	10.6
≥75%	192	6.3	174	5.7
Missing	657	21.6	565	18.6
Use of lead gloves
Never	1886	62.0	1955	64.3	50.4%	0.33 (0.29–0.38)∗
≤74%	258	8.5	297	9.8
≥75%	261	8.6	239	7.9
Missing	638	21.0	552	18.1
Use of thyroid protector
Never	1948	64.0	2016	66.3	52.2%	0.38 (0.33–0.42)∗
≤74%	252	8.3	274	9.0
≥75%	182	6.0	179	5.9
Missing	661	21.7	574	18.9
Night shifts
None	1382	45.4	1380	43.4	73.2%	0.73 (0.71–0.75)∗
<1 year	418	13.7	382	12.6
1–5 years	721	23.7	741	24.4
≥6 years	477	15.7	507	16.7
Missing	45	1.5	33	1.1

CI, confidence interval; ICC, intraclass correlation coefficient; SD, standard deviation.

Weighted kappa statistics.

The reliability of the question on medical radiation exposure in the last 3 years indicated a wide range of agreement depending on the types of medical procedures (Table 5). Basic medical procedures such as chest X-rays, which are provided to workers through the national health examination every 1 or 2 years, had relatively low agreement compared with other procedures, such as computed tomography and thyroid ultrasonography, which usually proceed with a specific diagnostic purpose.

Table 5

Reliability of the questions on medical radiation exposure

Questionnaire item	First response		Second response		Agreement	Weight kappa statistic (95% CI)
	n	%	n	%
X-ray (chest, abdomen, head, or limbs)
Never	389	12.8	374	12.3	49.2%	0.31 (0.28 to 0.34)
1 time	566	18.6	580	19.1
2 times	497	16.3	480	15.8
≥3 times	1522	50.0	1556	51.1
Missing	69	2.3	53	1.7
Intraoral or panoramic radiography (face and mouth)
Never	1775	58.3	1745	57.3	53.5%	0.39 (0.36 to 0.42)
1 time	590	19.4	651	21.4
2 times	248	8.2	304	10.0
≥3 times	205	6.7	191	6.3
Missing	225	7.4	152	5.0
Computed tomography
Never	2171	71.3	2216	72.8	68.3%	0.44 (0.40 to 0.48)
1 time	480	15.8	481	15.8
2 times	111	3.7	119	3.9
≥3 times	48	1.6	63	2.1
Missing	233	7.7	164	5.4
Fluoroscopy
Never	2625	86.3	2723	89.5	80.5%	0.18 (0.11 to 0.24)
1 time	83	2.7	71	2.3
2 times	20	0.7	15	0.5
≥3 times	14	0.5	15	0.5
Missing	301	9.9	219	7.2
Nuclear medicine imaging or therapy
Never	2624	86.2	2697	88.6	81.0%	0.30 (0.22 to 0.37)
1 time	96	3.2	114	3.8
2 times	15	0.5	14	0.5
≥3 times	10	0.3	13	0.4
Missing	298	9.8	205	6.7
Mammography
Never	2592	85.2	2664	87.6	80.4%	0.54 (0.47 to 0.61)
1 time	77	2.5	80	2.6
2 times	23	0.8	32	1.1
≥3 times	21	0.7	24	0.8
Missing	330	10.8	243	8.0
Interventional radiography
Never	2695	88.6	2774	91.2	83.0%	0.08 (0.01 to 0.14)
1 time	35	1.2	31	1.0
2 times	5	0.2	10	0.3
≥3 times	5	0.2	12	0.4
Missing	303	10.0	216	7.1
Radiation therapy (external radiotherapy or brachytherapy)
Never	2710	89.1	2794	91.8	83.9%	0.05 (−0.06 to 0.16)
1 time	16	0.5	20	0.7
2 times	6	0.2	7	0.2
≥3 times	7	0.2	10	0.3
Missing	304	10.0	212	7.0
Thyroid ultrasonography
Never	2333	76.7	2359	77.5	81.0%	0.66 (0.63 to 0.69)
1 time	359	11.8	353	11.6
2 times	117	3.8	128	4.2
≥3 times	145	4.8	162	5.3
Missing	89	2.9	41	1.4

CI, confidence interval.

The reliability of the questionnaire was also evaluated by sex, age, occupation, educational level, employment status, and occupational exposure (Supplementary Tables S1–S6). Although the reliability somewhat differed depending on sex, age, and types of facilities, a specific tendency to be characterized by these factors was not observed (Supplementary Tables S1–S3). However, overall, a higher agreement of the responses was observed among participants with higher education levels (i.e., college or above) as compared to those with lower education levels (i.e., high school or below), particularly on work practices and medical radiation exposure (Supplementary Table S4). In addition, participants with a secure employment status (i.e., regular job position) or occupational exposure showed a higher agreement of responses on work practices than those with an irregular job position or non-exposure to occupational radiation (Supplementary Tables S5, S6).

Validity

The self-reported responses about the first year of employment (i.e., calendar year of hire) were compared with the first year of radiation dose records from the National Dose Registry, indicating a high agreement between the survey responses and the dose registry (ICC = 0.90; Table 6). The agreement on cancer history between self-reported responses and the National Cancer Registry had high specificity (99.8%–99.9%) and negative predictive values (99.9%–100.0%); however, sensitivity (57.1%–89.7%) and positive predictive values (15.4%–81.4%) ranged widely depending on cancer type. The agreement was highest in thyroid cancer (kappa = 0.85) and lowest in lung cancer (kappa = 0.22; Table 7).

Table 6

Validity of the questions on work history

Questionnaire item	Survey∗		Registry		ICC (95% CI)
	Mean	SD	Mean	SD
Calendar year of hire	2006.85	9.41	2007.91	8.94	0.90 (0.88–0.91)

CI, confidence interval; ICC, intraclass correlation coefficient; SD, standard deviation.

Response at the second survey.

Table 7

Validity of the questions on cancer history

Cancer history	Yes (self-reported)	Cases (registry)	Agreement	Sensitivity	Specificity	PPV	NPV	Kappa statistic (95% CI)
Stomach	73	56	91.8%	78.6	99.8	60.3	99.9	0.68 (0.59–0.77)
Lung	26	10	91.8%	40.0	99.9	15.4	100.0	0.22 (0.04–0.40)
Colon	58	33	91.8%	72.7	99.8	41.4	100.0	0.53 (0.40–0.65)
Breast	35	14	91.1%	71.4	99.9	28.6	100.0	0.41 (0.23–0.58)
Thyroid	129	117	91.7%	89.7	99.9	81.4	99.9	0.85 (0.81–0.90)

CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

The validity of the self-reported responses to calendar year of hire was evaluated by sex, age, occupation, educational level, employment status, and occupational exposure (Supplementary Table S7). Overall, excellent agreement for the question on calendar year of hire was observed in all subgroups stratified by these factors; however, the agreement among participants unexposed to occupational radiation was relatively low compared with other subgroups (ICC = 0.60).

Discussion

This study evaluated the reliability and validity of a self-administered questionnaire developed for an epidemiological study of Korean radiation workers. We found that the reliability of self-reported responses to questions about demographics and lifestyle factors was moderate-to-high (kappa: 0.49–0.95); however, the reliability of responses to the questions about work practices including wearing protective equipment and a personal dosimeter and medical radiation exposure had a wide range of agreement (kappa: 0.05–0.73). Low reliability values (kappa: ~0.49) were mostly observed for questions to which responses were likely to be related to temporal variability depending on changes in job tasks and opportunities of regular medical checkups. For example, newly hired workers who participated in the first survey might not have been assigned to radiation-related tasks yet owing to the incompletion of a mandatory radiation education course. Indeed, overall responses to the questions on work practices in the second survey showed higher reliability than those in the first survey, and when evaluation was restricted to participants with an employment period ≥2 years, the reliability values of most items of work practices increased, particularly for wearing a personal dosimeter, that showed a kappa value of 0.45. For medical radiation exposure, most radiation workers in Korea are provided with opportunities to undergo common diagnostic procedures (e.g., chest X-rays) through a regular medical checkup every year in the national workers' general health examinations [22]. This may have resulted in the low percentage agreements of responses between the surveys conducted in different years. In contrast, uncommon procedures such as fluoroscopy, interventional radiography, and radiation therapy showed high percentage agreements (>80%) but had low kappa values (0.05–0.18). This is mainly owing to the nature of the calculation of a kappa value—that a question with a highly unbalanced response (e.g., questions with dichotomous response options to which few participants [or most participants] give a particular response) yields limited kappa values despite a high level of percentage agreement [23].

Our findings related to the reliability evaluation were comparable to those of previous occupational studies. The high levels of reliability for questions on the first year of employment and employment duration in this study (ICC: 0.95 and 0.95, respectively) were comparable with findings from studies of diagnostic radiologic technologists (ICC: 0.87–0.99) [6] and industrial workers such as coal miners [24], shipyard workers [25], and capacitor manufacturing workers [26]. In addition, after combining a few levels of responses (e.g., recategorization from ordinal responses to binary responses) to make comparisons with other studies more applicable, overall kappa values for demographics, lifestyle, work practices, and medical exposure were not highly deviated from those in other studies (Supplementary Table S8).

The reliabilities of questions on demographics and lifestyle were not distinguished by sex, age, and types of facilities, similar to that observed in other occupational studies [[27], [28], [29]]. In general, sociodemographic characteristics are associated with the reliability of survey responses, particularly in studies of the general population [30,31]; however, they influenced the reliability in this study less, which might be attributed to homogeneity of the participants selected from a specific population belonging to similar occupational fields. However, the reliability of questions on work practices was higher for participants with job characteristics, such as having a secure position and a radiation exposure-related duty, indicating that workers with unstable job positions or less involved radiation-related duties are likely to have temporal variability for their job tasks during employment.

For the validity evaluation of the self-administered questionnaire, an excellent agreement on the first year of employment was shown in this study, similar to that observed in another occupational study with an agreement of 0.93 [25]. Similar to the reliability evaluation, the agreement was relatively lower among workers unexposed to occupational radiation, implying that inclusion of workers whose radiation exposure doses are below the recording level may increase uncertainties in dose estimation (e.g., dose reconstruction model) when using a self-administered questionnaire. A wide range of agreement on cancer history between survey responses and the National Cancer Registry was observed depending on agreement measures, such as sensitivity, specificity, and positive predictive values—which are highly influenced by the prevalence of cancer. Considering that participants were active workers at the time of the survey (i.e., assumed to be healthy), the low positive predictive values are mainly owing to the low prevalence rates of cancer; the overall agreement on cancer history in this study was comparable to that in other studies [[32], [33], [34], [35]], particularly for mostly non-life-threatening cancers with higher prevalence rates (e.g., thyroid cancer; Supplementary Table S9).

This study had some limitations originating from the nature of self-administered surveys. First, approximately 50% of the entire target population participated in this study, which may not represent the entire population and thus may result in selection bias related to radiation exposure and disease status. However, considering that the surveys were conducted nationwide on active workers who were assumed to be healthy, and distributions of occupations and radiation doses among these participants did not highly deviate from the study population [36], the selection of participants can be assumed to be not highly correlated with exposure and disease status. Second, the evaluation of the reliability of survey responses can be affected by the time interval between surveys. While, in general, around 1 month can be adequate to examine the reliability of survey responses [37,38], the average time interval between the surveys in our study was approximately 7.5 months. As time passes, the blurring of past information [39] and the possibility of true changes in one's lifestyle and job duties increases, resulting in decreased reliability; thus, our reliability measures might be underestimated. In addition, the interval should be long enough to prevent memory effects [40]. Approximately 1.4% of the participants had a time interval of less than 1 month; thus, memory effects would be expected to be minimal. Finally, available periods of the dose registry and cancer registry used for the validity evaluation are limited. Since the dose registry was available only from 1984, it was difficult to identify the actual first calendar year of hire for workers who responded to the question about the first year of their employment with the option “before 1984.” However, only 2.3% of the participants reported that their first calendar year of hire was before 1984. Furthermore, when these participants were removed from the evaluation, the ICC values remained almost the same. Similarly, the cancer registry was available from 1988; therefore, we could not verify the cancer history of participants who answered that they were diagnosed with cancer before 1988. However, the average age of this study's participants was 12.6 years in 1988, and the percentage of participants aged more than 40 years in 1988 was 0.3%; thus, the influence of unidentified cases on the validity evaluation of cancer history can be assumed to be minimal.

Despite these limitations, this study is unique because it provides reliability and validity assessments of a self-administered questionnaire on radiation workers, particularly in the nuclear industry where this type of a study is rarely conducted. Moreover, the evaluation items are comprehensive, including demographics, lifestyle factors, and occupational characteristics, which provide useful information for radiation epidemiological studies.

In summary, although information on demographics and lifestyle factors collected in the self-administered questionnaire proved to be reliable for use as potential confounders for the assessment of radiation-induced health risks, occupation-related information had a wide range of reliability, possibly owing to temporal variability during employment, particularly for workers whose occupational radiation doses were below the recording level. Thus, one must exercise caution when using information from workers with minimal occupational radiation doses as baseline levels, in reconstructing radiation doses or estimating organ doses. Although the data used in this study were derived from a specific occupational cohort of radiation workers, our findings can be applied to other occupational studies using self-administered questionnaire to identify baseline characteristics of participants, including demographics, lifestyle, and exposure levels of harmful agents. Future questionnaires need to include more detailed items such as job position or job title to identify whether changes in responses on surveys, particularly for occupational characteristics, actually occurred because of changes in job duties during employment.

Conflicts of interest

The authors declare they have no conflict of interests.