Validity and reliability of the Japanese version of the diabetes knowledge test among in-patients with type 2 diabetes.

AIMS/
INTRODUCTION: The diabetes knowledge test (DKT) is unavailable in Japan. In this study, we developed and evaluated a Japanese version of the DKT (J-DKT) for in-patients with type 2 diabetes before and after receiving diabetes education.
MATERIALS AND METHODS: The J-DKT contains 12 questions (0-12 points) to assess knowledge regarding diabetes, its complications, and diabetic nutrition therapy. During the median 10 days of hospitalization, 107 patients with type 2 diabetes received diabetes education (20 min private lessons every day from physicians, two nutrition counselling programs from dietitians, and a 2 h group session conducted by physicians, dietitians, and nurses). The J-DKT was administered on admission and before discharge. To confirm the J-DKT's reliability, we assessed the internal consistency using Cronbach's α (≥0.70 was considered acceptable). To evaluate its validity, we investigated changes in the J-DKT total scores after the education programs and examined the differences in the scores among groups classified based on patient characteristics such as age, diabetes-related hospitalization history, and hospitalization duration.
RESULTS: The J-DKT total scores increased from 5 to 8 (P ˂ 0.01) after the education programs. The J-DKT before and after the program showed a Cronbach's α of 0.48 and 0.73, respectively. Except for age, baseline characteristics such as history and period of hospitalization for diabetes were not associated with the J-DKT scores after the education program.
CONCLUSIONS: The validity and reliability of the J-DKT after the diabetes education program were acceptable in this study.

INTRODUCTION

The number of patients with type 2 diabetes is increasing all over the world, including in Japan . The goal of diabetes management is to prolong a healthy life and to improve the quality of life among patients with diabetes. The American Diabetes Association, the European Association for the Study of Diabetes, and the Japanese Diabetes Society (JDS) have recommended that diabetic patients should receive education regarding medical nutrition and exercise therapy as well as knowledge of diabetes and its comorbidities , , , .

The diabetes knowledge test (DKT), which was developed in the United States (US‐DKT), has been useful in measuring the extent to which patients with diabetes understand diabetes as a disease and its comorbidities in addition to nutrition and exercise therapy . The US‐DKT has been employed in the United States for patients with either type 1 or 2 diabetes . Some cross‐sectional studies have reported that patients with diabetes with high scores in the US‐DKT achieved better glycemic control , .

We developed the Japanese version of DKT (J‐DKT) as an original one for three reasons. First, the US‐DKT comprises items on diet and lifestyle habits specific to the United States (e.g. baked potato) . Japanese people often prefer eating steamed rice over baked potatoes . Therefore, the J‐DKT should be developed in a way that it collects information relevant to the diet and lifestyle habits in Japan. Second, the US‐DKT contained items that may not be appropriate to evaluate a patient’s knowledge of diabetes before and after disease education, e.g. For a person in good control, what effect does exercise have on blood glucose? The answer is ‘Lowers it’. Some items in the US‐DKT were considered easy, and most patients with diabetes were able to select the correct answer before participating in a hospital‐based education program, which raises concerns on the possibility of ceiling effects. The diabetes education program has been reported to continue for a longer duration when administered in a hospital setting in Japan , , and advanced knowledge is frequently provided. Third, the guidelines of the JDS should be considered when designing the J‐DKT. In addition, the number of older patients is higher in Japan than in other countries . The JDS has recommended that patients with diabetes should be informed about the comorbidities that occur frequently in older patients, such as dementia, falls/fractures, sarcopenia, or malignant tumors, as well as information about preventing the vascular complications associated with diabetes .

In this study, we established 12 items of the J‐DKT: a full score is 12 points (Figure 1). Some items were similar to those in the US‐DKT (e.g. Hemoglobin A1c is a measure of your average blood glucose level for the past: 6–12 weeks). The J‐DKT requires knowledge regarding diabetes, its complications and diabetic nutrition therapy. We created the J‐DKT for patients with type 2 diabetes who did not use insulin because the number of insulin users was reported to be approximately 8% of all patients with diabetes, according to the claims database of 121 hospitals . Therefore, in this study, questionnaires for insulin users were not included. However, the J‐DKT can be applied to both users and nonusers of insulin.

Figure 1

Japanese version of the diabetes knowledge test.

In this report, we developed the J‐DKT and evaluated its validity and reliability in Japanese patients with diabetes who were hospitalized and participated in diabetes education programs.

PATIENTS AND METHODS

Study design and ethical considerations

This multicentre and cross‐sectional study was registered with the University Hospital Medical Information Network (UMIN ID: 000034329). The primary endpoint was to investigate the validity and reliability of J‐DKT among in‐patients with type 2 diabetes. The study was approved by the ethics committee of Saiseikai Yokohama‐Shi Nanbu Hospital and Odawara Municipal Hospital on November 12, 2018 (approval number; 2018‐D27). This study was conducted in accordance with the tenets of the Declaration of Helsinki. Written informed consent was obtained from all participants.

Participants

Of the target participants, 114 patients were eligible and 107 were enrolled in this study (96 patients in Saiseikai Yokohama‐Shi Nanbu Hospital and 11 patients in Odawara Municipal Hospital). The participants were Japanese in‐patients (aged ≥20 years; native language, Japanese) with type 2 diabetes who were consecutively treated at each hospital from November 2018 to November 2020. The exclusion criteria were stroke history, dementia, diabetic ketoacidosis, and psychiatric conditions such as depression, schizophrenia, developmental disorder, or attention deficit‐hyperactivity disorder.

Assessments of blood tests and characteristics

Fasting blood tests were performed a day after hospitalization for patients with type 2 diabetes. The HbA1c levels were measured using high‐performance liquid chromatography (HPLC‐723G11; Tosoh, Inc., Tokyo, Japan). The serum creatinine levels were measured using the modified Jaffe method. The estimated glomerular filtration rate was calculated as: (mL/min/1.73 m2) = 194 × serum creatinine−1.094 × age−0.287 (×0.739 for females) . During hospitalization, the levels of fasting serum C‐peptide and urinary albumin‐to‐creatinine ratio (ACR) were measured at a central clinical laboratory (SRL, Inc., Tokyo, Japan). We investigated all participant characteristics, including age, gender, body mass index, diabetes duration (years), dementia, and psychiatric diseases on admission.

J‐DKT and interventions during hospitalization

The J‐DKT was developed in Japanese and translated into English by two bilinguals who used Japanese and English languages via ulatus service (https://www.ulatus.jp/; accessed on February 17, 2021), with the first author confirming the consistency of each question (Figure 1). All patients completed the J‐DKT (in Japanese) on admission and before discharge. During hospitalization, (median, 10 days), physicians educated the patients for 15–20 min every day regarding diabetes and its complications, how to exercise according to their body composition and lifestyles, how to prepare meals to achieve better glycemic control and body weight, and to manage diabetes therapy during days when they were sick. After receiving the education, their knowledge regarding diabetes were assessed by nurses twice for 15–20 min. The patients also received nutrition counselling from nationally registered dietitians twice during their hospitalization. They also participated in a 2 h group session on diabetes that was administered by nationally registered physical therapists, dental and diabetes nurses, and physicians.

Statistical analysis

All statistical analyses were performed using IBM SPSS 26 Software for Windows (SPSS, Inc., Chicago, IL, USA). Continuous variables were presented as mean ± standard deviation and median [first quartile, third quartile] when they did and did not follow a normal distribution, respectively. Categorical variables were presented as numbers. Cronbach’s coefficient α was used to calculate reliability, and the item‐total correlations were also evaluated for each item. Using the McNemar test, each item score on the J‐DKT on admission and before discharge were compared. The J‐DKT scores were defined as the total scores of the J‐DKT, and the J‐DKT scores on admission and before discharge were compared using the Wilcoxon signed‐rank test. As validation, the J‐DKT total scores before discharge among the following groups were compared via the unpaired Student’s t‐test: by gender (male/female), history of hospitalization for diabetes and insulin use. The J‐DKT scores before discharge were compared among the groups using the Mann–Whitney U test.

The ΔJ‐DKT scores were defined as the difference in the J‐DKT from discharge to admission. The correlations of the J‐DKT scores before discharge and ΔJ‐DKT scores were assessed by age, duration of diabetes, days in hospital, and HbA1c levels on admission using univariate linear regression analysis and multivariate analysis. HbA1c levels, eGFR, and ACR were assessed as the correlates of J‐DKT scores on admission via univariate linear regression analysis.

The J‐DKT scores on admission were compared between the groups identified to have a history of hospitalization for diabetes. The J‐DKT scores on admission and before discharge were also compared between the two hospitals. The J‐DKT scores on admission were compared among the patients with and without any previous history of coronary artery diseases. The J‐DKT scores before discharge and ΔJ‐DKT were compared between patients aged >65 years and younger patients. All the above comparisons were done using the Mann‐Whitney U test. P < 0.05 was considered significant.

RESULTS

Characteristics of participants

Of the 114 patients with type 2 diabetes, 107 were enrolled in this study. The characteristics of participants (77 males and 30 females; median age, 66 years [55, 75]) are shown in Table 1. Data are presented as median [first quartile, third quartile] or means (standard deviation). The median HbA1c of all participants was 10.0% [8.8%, 11.6%]. For all participants, the mean body mass index was 26.0 kg/m2 (4.3), fasting median C‐peptide was 1.37 ng/mL [0.78 ng/mL, 1.83 ng/mL] and median ACR value was 20.2 mg/gCr [6.5 mg/gCr, 89.6 mg/gCr]. Most patients (95/107, 89%) were hospitalized for the first time to optimize their diabetes treatment and to receive diabetes education. The median hospitalization period of all patients was 10 days, and their median diabetes duration was 5 years [1, 15]. In total, 23 patients (21%) had received insulin therapy. Of the insulin users, 43% (10/23) had started insulin therapy within a month before hospitalization. The median number of oral medications for diabetes was one [0, 2].

Table 1

Patients’ baseline characteristics

	(n = 107)
Male/Female	77/30
Age (years)	66 [55, 75]
BMI (kg/m2)	26.0 ± 4.3
HbA1c in the hospital (%)	10.0 [8.8, 11.6]
Fasting serum CPR (ng/mL)	1.37 [0.78, 1.83]
Duration of diabetes (years)	5 [1, 15]
eGFR (mL/min/1.73 m2)	66.0 ± 22.9
Albumin‐to‐creatinine ratio (mg/gCr)	20.2 [6.5, 89.6]
History of coronary artery disease	14 (13.1%)
History of hospitalization for diabetes (n, %)	12 (11.2%)
Days in the hospital	10 [8, 11]
Diabetes therapy
Number of tablets for diabetes	1 [0, 2]
Insulin user (n, %)	23 (21.5%)
Duration of insulin use (years)	1 [0, 13]

Data are expressed as median [first quartile, third quartile], mean ± standard deviation or numbers. BMI, body mass index; CPR, C‐peptide immunoreactivity; eGFR, estimate glomerular filtration rate; HbA1c, glycated hemoglobin.

Reliability

As shown in Table 2, the Cronbach’s α of the J‐DKT on admission and before discharge were 0.48 and 0.73, respectively (≥0.70 implies good internal consistency). The J‐DKT scores before discharge showed that the item‐total correlation was >0.20 on each item, except for item no. 12. These results indicate that the J‐DKT scores before discharge yielded good reliability and the questions are discriminating well. On the other hand, the J‐DKT performed on admission showed a Cronbach’s α of 0.48, and the item‐total correlation coefficients for 7 of 12 questions were <0.20.

Table 2

Cronbach α, percentage of correct answers and item‐total correlation in the Japanese version of the Diabetes Knowledge Test (J‐DKT)

Item	On admission		Before discharge		P
	Percent correct (%)	Item‐total correlation	Percent correct (%)	Item‐total correlation
	Cronbach α: 0.48		Cronbach α: 0.73
1	32	0.34	51	0.42	˂0.01
2	35	0.16	47	0.34	˂0.05
3	23	0.14	61	0.30	˂0.01
4	11	0.05	33	0.38	˂0.01
5	32	0.11	50	0.36	˂0.01
6	58	0.19	88	0.55	˂0.01
7	50	0.26	79	0.43	˂0.01
8	55	0.07	82	0.50	˂0.01
9	47	0.10	70	0.38	˂0.01
10	77	0.22	83	0.31	0.27
11	43	0.26	79	0.39	˂0.01
12	51	0.25	69	0.15	˂0.01
	Total score: 5 [3, 7]		Total score: 8 [6, 10]		˂0.01

P values were evaluated to compare each item between the J‐DKT on admission and before discharge. The total score was expressed as median [first quartile, third quartile].

Validity

As displayed in Table 2, the total scores of the J‐DKT significantly improved from 5 [3-7] on admission to 8 [6-10] before discharge (from 5 [3-7] to 8 [6-10] in Saiseikai Yokohama‐Shi Nanbu Hospital and from 5 [4.5–7] to 8 [6.5–12] in Odawara Municipal Hospital, with no statistical difference between the two hospitals). The percentage of correct answers on each item and total scores before discharge were significantly higher than those on admission, except for item no. 10 (Table 2). These results suggest that the J‐DKT reflected the acquisition of diabetes‐related knowledge. There were no differences in the J‐DKT scores before discharge in the contexts of gender, history of hospitalization for diabetes, and insulin use (Table 3). Hence, the J‐DKT can be given to a wide range of patients with diabetes regardless of their gender, history of education, or therapeutic intervention. Next, we examined the associations of the J‐DKT scores before discharge with the baseline characteristics of age, diabetes duration, hospitalization period (days) and HbA1c levels using univariate and multivariate linear regression analysis (Table 4). The analysis showed a negative association of the J‐DKT scores with age after adjustment with gender, hospitalization period, and insulin use. Thus, the J‐DKT scores could be influenced in older patients with type 2 diabetes. The difference in J‐DKT scores before discharge from those on admission was defined as ΔJ‐DKT, which reflects acquisition of diabetes‐related knowledge during hospitalization. The ΔJ‐DKT showed a negative association with age, but not diabetes duration, hospitalization period, and HbA1c levels (Table 5).

Table 3

J‐DKT scores before discharge

Baseline characteristics		Median total score [first quartile, third quartile]
Gender	Male	9 [6, 10]
	Female	8 [6, 10]
	Difference	P = 0.47
History of hospitalization for diabetes	Yes	8 [5, 10]
	No	8 [6, 10]
	Difference	P = 0.78
Insulin use on admission	Non‐users	8 [6, 9]
	Users	9 [6, 10]
	Difference	P = 0.29

J‐DKT, Japanese version of Diabetes Knowledge Test.

Table 4

Associations of J‐DKT scores before discharge with age, diabetes duration, hospitalization period, and HbA1c levels

Association	Univariate		Multivariate
	B (95% CI)	P	B (95% CI)	P
Age (years)	−1.0 (−0.13, −0.70)	˂0.001	−1.0 (−0.13, −0.06)	˂0.001
Diabetes duration (years)	−0.08 (−0.13, −0.03)	˂0.01	−0.04 (−0.08, 0.01)	0.13
Hospitalization period (days)	−0.08 (−0.30, 0.13)	0.45	0.06 (−0.14, 0.26)	0.57
HbA1c levels (%) in the hospital	0.21 (−0.03, 0.44)	0.08	−0.06 (−0.29, 0.18)	0.63

HbA1c, glycated hemoglobin; J‐DKT, Japanese version of Diabetes Knowledge Test.

Table 5

Association of ΔDKT scores with age, diabetes duration, hospitalization period, and HbA1c levels

ΔDKT	Univariate		Multivariate
Age (years)	−0.05 (−0.08, −0.01)	˂0.01	−0.05 (−0.09, −0.01)	˂0.05
Diabetes duration (years)	−0.04 (−0.08, −0.01)	0.15	0.02 (−0.04, 0.07)	0.52
Hospitalization period (days)	0.01 (−0.20, 0.22)	0.91	0.01 (−0.20, 0.22)	0.92
HbA1c levels (%) in the hospital	0.18 (−0.05, 0.41)	0.13	0.06 (−0.18, 0.31)	0.61

HbA1c, glycated hemoglobin; J‐DKT, Japanese version of Diabetes Knowledge Test; ΔDKT, difference of J‐DKT scores before discharge from those on admission.

DISCUSSION

To our knowledge, this study is the first to report the use of a Japanese version of the DKT, which comprises 12 items (0–12 points), for patients with type 2 diabetes. The J‐DKT had sufficient internal consistency and reliability (Cronbach’s α of J‐DKT before discharge = 0.73 [≥0.70]) and good discrimination (item‐total correlation ≥0.20 except for no. 12) after participating in diabetes education programs. Because the scores of the J‐DKT did not vary among the groups based on gender, diabetes‐related hospitalization history, and insulin use, the test is universally applicable to patients with type 2 diabetes. The J‐DKT scores, or ΔJ‐DKT, were not associated with the duration of diabetes, hospitalization period (days), and HbA1c levels, although age was a variable factor that influenced the scores and acquisition of knowledge. Therefore, the J‐DKT would be useful to assess the efficacy of diabetes education programs for Japanese in‐patients before discharge, but the effects of aging should be considered. Aging has been reported to be the best predictor of cognitive decline even in absence of dementia , . Cognitive abilities such as memory performance or reasoning ability decline during aging, particularly in individuals aged >65 years . The J‐DKT scores before discharge and ΔJ‐DKT of patients with type 2 diabetes aged >65 years were lower than those of younger patients (6.9 ± 2.7 vs 9.2 ± 2.1 for J‐DKT before discharge, P ˂ 0.01, and 2.3 ± 2.6 vs 3.4 ± 2.5 for ΔJ‐DKT, P ˂ 0.01). Thus, the J‐DKT may also be useful in identifying cognitive impairment.

To prove the validity of J‐DKT, we evaluated whether J‐DKT scores reflect diabetes‐related knowledge in terms of whether the J‐DKT scores improved after participating in the diabetes education program. In the present study, J‐DKT scores were not normally distributed before and after the program. Evaluation of the increased diabetes‐related knowledge was challenging because of the abnormal distribution. However, the percent correct of all items except no. 10 increased during diabetes hospitalization via the McNemar test. Increased percent correct of each item contributed to increased total J‐DKT scores. Therefore, participating in the program might have improved the J‐DKT scores.

The item‐total correlation of no. 12 was ≤0.20 before discharge. Item no. 12 is ‘Which is Because participants had to choose an incorrect answer, some participants might have misinterpreted the question. To evaluate its reliability, the test–retest reliability of the J‐DKT should be evaluated. However, in this study, participants took the test twice during hospitalization, for a median of 10 days, to evaluate the acquisition of diabetes‐related knowledge. Therefore, the time interval between the tests was limited to assess the test–retest reliability in this study. As shown in Table 2, the Cronbach’s α on the J‐DKT on admission (0.43 < 0.7) estimates a low internal consistency. The percentage of correct answers and the item‐total correlation in all items for the J‐DKT on admission were also lower than those for the J‐DKT before discharge. Low values for the percentage of correct answers or item‐total correlation implied that participants did not answer based on their diabetes‐related knowledge on admission. Because a higher percentage of correct answers and item‐total correlation were observed after participating in hospital‐based diabetes education program, the J‐DKT would be more beneficial as an evaluation tool after receiving diabetes education.

To develop a new test, it should be compared with other tests that have been established. However, there have been no tests to evaluate diabetes‐related knowledge in Japan. In the present study, the J‐DKT could not be compared with other tests.

This study has several limitations. First, the small sample size might have caused selection bias that may have affected our results. However, this study was conducted as a multicenter study in different cities. Second, the individual hospitalization period varied from 6 to 16 days (median, 10 days [8, 11]). However, the duration of the hospitalization was not associated with the J‐DKT scores before discharge, as shown in Table 4. Finally, this study included no controls; comparisons should have been made between in‐patients who received diabetes education with those who did not. However, it was ethically inappropriate to not provide some in‐patients with diabetes‐related knowledge.

The J‐DKT developed in this study aimed to evaluate diabetes‐related knowledge, which is essential for self‐management . In another study, we investigated whether higher J‐DKT scores result in better glycemic controls (through self‐management) a year after hospitalization. In the present study, the J‐DKT scores on admission had no correlation with HbA1c levels and diabetes‐associated complications such as diabetic kidney disease (i.e. eGFR and ACR) as well as any previous history of coronary artery disease. Of 102 patients, 95 were hospitalized for glycemic controls and were receiving diabetes education for the first time. Their J‐DKT scores on admission were lower than those of 12 patients with a history of previous hospitalization for diabetes (5 [3‐6] vs 7 [6‐8], P < 0.01). Most patients in this study might have participated in few diabetes education programs and had insufficient knowledge regarding diabetes. Therefore, J‐DKT scores on admission had no correlation with HbA1c levels and diabetic complications in this study.

J‐DKT should be administered after the patients have participated in diabetes education programs. In the present study, the J‐DKT was considered inadequate to evaluate diabetes‐related knowledge among patients who do not possess much knowledge on diabetes. If the J‐DKT has to be examined among outpatients, they will need to be informed about diabetes and the complications associated with it. We are currently preparing educational videos on diabetes, which are prepared based on J‐DKT questionnaires, to upload on YouTube and will investigate whether the J‐DKT scores improve after watching our YouTube videos.

To our knowledge, this study is the first to establish a Japanese version of the DKT for patients with type 2 diabetes before and after receiving diabetes education in a hospital. The acceptable validity and reliability of the J‐DKT after receiving diabetes education have demonstrated the feasibility of this test for the clinical assessment of diabetes education programmes. Further studies with larger sample sizes are needed to better assess its clinical applicability and usability.

DISCLOSURE

The authors declare no conflict of interest.

Approval of the research protocol: The study was approved by the ethics committee of Saiseikai Yokohama‐Shi Nanbu Hospital and Odawara Municipal Hospital on November 12, 2018.

Informed consent: Written informed consent was obtained from all participants in accordance with the tenets of the Declaration of Helsinki.

Approval date of registry and registration no. of the study/trial: Date: October 1, 2018. No. UMIN ID: 000034329.

Animal studies: N/A.