Validity of self-reported diabetes among middle-aged and older Chinese adults: the China Health and Retirement Longitudinal Study.

BACKGROUND: Self-reported diabetes has been found to be valid to evaluate people's diabetes status in the population of several countries. However, no such study has been conducted to assess the validity of self-reported diabetes in the Chinese population, the largest population with the highest rate of diabetes. The aim of our study is to evaluate the validity of self-reported diabetes among the middle-aged and older Chinese population.
METHODS: Data from 11 601 participants, aged ≥45, of the China Health and Retirement Longitudinal Study (CHARLS) during 2011-2012, were analysed. Prevalent self-reported diabetes was compared with reference definition defined by fasting glucose, glycated haemoglobin and medication use. Sensitivity, specificity, positive predicted value, negative predicted value and κ value were calculated overall, by 5-year age groups, by education levels and by living areas.
RESULTS: The sensitivity of prevalent self-reported diabetes was 41.5%, and the specificity was 98.6%. The sensitivity of self-reported diabetes increased with education levels, and was much higher among urban residents than rural residents (58.2% vs 35.0%). The specificity was above 98% among all age groups, in different education levels, and in rural and urban areas. Self-reported diabetes had substantial agreement with reference definition among participants with above vocational school education or those living in urban areas (κ=0.658 and 0.646, respectively).
CONCLUSIONS: Although the sensitivity of self-reported diabetes was poor among middle-aged and older Chinese adults, the specificity and positive predictive values were fairly good. Furthermore, self-reported diabetes performed well among those with more than vocational school educations or those living in urban areas. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Study population was a representative sample of China's middle-aged and older population, which indicates high generalisability of the result of the current study.

We used fasting blood glucose and glycated haemoglobin levels instead of medical records to define diabetes, which eliminated undiagnosed cases of diabetes.

Stringent quality control and quality assurance measures were implemented in every stage of the China Health and Retirement Longitudinal Study (CHARLS) study, which guaranteed the quality of our study.

We included using traditional Chinese antidiabetic medication as one of the criteria for reference definitions of diabetes, which was different from the criteria that were generally accepted in studies outside China.

Introduction

Self-reported diagnosis of diabetes is the major method for determining diabetes in large-scale epidemiological studies.1 2 Self-reported diabetes has been found to be a valid method to evaluate people's diabetes status in the populations of several countries, such as Britain,3 Spain,4 Brazil,5 Japan6 and the USA.7 8 However, no study has been conducted to examine the validity of self-reported diabetes in China, which has the largest number of people with diabetes in the world,9 particularly among the most vulnerable middle-aged and older Chinese population.

Diabetes is one of the leading causes of mortality and morbidity worldwide, which increases the risk for cardiovascular and kidney diseases.10 In China, approximately 114 million, or 11.6% of adults, had diabetes in 2010.9 However, less than one-third (30.1%) of these patients were aware of their disease condition,9 and this may affect the estimate of diabetes prevalence based on self-reports in large-scale epidemiological studies. The accuracy of self-reported information can be affected by respondents’ age, gender, education, medical knowledge and frequency of contact with a physician.11 The inconsistency between self-reported diabetes and the reference standard can bias the results of studies using self-reported diabetes. Typical reference definitions of diabetes include fasting glucose level, random blood glucose test, haemoglobin A1c (HbA1c), oral glucose tolerance test and/or taking diabetes medications.8 12 Therefore, the purpose of this study was to assess the validity of prevalent self-reported diabetes by using reference definitions defined by fasting or random blood glucose levels, HbA1c and taking diabetes medication, in a representative sample of middle-aged and older Chinese population using baseline data from the China Health and Retirement Longitudinal Study (CHARLS).

Methods

Subjects and design

The CHARLS surveys a representative sample of more than 17 000 Chinese aged ≥45 from China's 28 provinces every 2 years to explore the dynamics of retirement and how retirement interacts with health, health insurance and economic well-being.13 Participants were selected using a four-staged, stratified, cluster sampling method.13 The baseline national survey was carried out from 2011 to 2012 through face-to-face household interviews, which collected detailed information on demographics, biomedical measurements, socioeconomic status and self-reported health status and functioning including diabetes.13 The CHARLS also collected data on fasting blood glucose and HbA1c. The comprehensive and multifaceted dataset provided by the CHARLS national baseline survey allows us to evaluate the validity of self-reported diabetes.

The response rate among eligible households was 80.51%.14 Overall, 17 314 individuals aged ≥45 within 10 257 households were investigated in the baseline survey.14 15 Blood samples were collected from 11 847 (67%) out of the 17 314 study individuals by medically trained staff. A total of 11 601 participants had blood samples and/or information on self-reported diabetes, insulin or diabetes medication, and were included in the current analysis. The current study was secondary analysis of the open-access dataset of the CHARLS.

Self-reported diabetes and reference definitions of diabetes

Participants were defined as having self-reported diabetes if they answered ‘yes’ to the questions: “Have you been diagnosed with diabetes or high blood sugar?”. The reference definitions of diabetes were defined as: fasting blood glucose ≥126 mg/dL, or random blood glucose ≥200 mg/dL, or HbA1c ≥6.5%, or insulin use, or taking antidiabetic medications including traditional Chinese medicine and modern medicine. The cut-off points for diagnosis of diabetes were based on current recommendations from the American Diabetes Association.12

Measurement of blood glucose and HbA1c

Blood samples were collected by medically trained staff from the China Center for Disease Control and Prevention. Participants were asked to fast overnight. However, a small proportion (8%) of the blood samples was also drawn even if the participants did not fast.15 For those who did not fast, standards for random blood glucose level (≥200 mg/dL) and/or glycated haemoglobin (HbA1c) (≥6.5%) were used to define diabetes.

After collection, plasma for glucose assay was separated from blood samples and stored at −20°C,15 and whole blood for HbA1C assay was stored immediately and during shipment at 4°C.15 All the blood samples were transported within 2 weeks to the China Center for Disease Control and Prevention, where samples were placed at −80°C in a deep freezer before assay.15 Blood assays were performed at the Youanmen Center for Clinical Laboratory of Capital Medical University during February 2013 and June 2013.15 The laboratory used quality control samples daily during the testing of the CHARLS study samples, and all test results were within the target range (within two SDs of mean quality control concentrations). Glucose was measured using an enzymatic colorimetric test, and HbA1c was analysed using Boronate affinity chromatography.15

Demographic variables

Demographic variables including age, gender and educational level were collected using standardised questionnaires during the interview. Educational level was categorised as illiterate, less than primary school, primary school, middle school, high school and above vocational school.

Statistical analysis

We assessed the validity of self-reported diabetes by calculating sensitivity (proportion of people with diabetes who correctly self-reported having the condition), specificity (proportion of individuals without diabetes who correctly self-reported not having the condition), positive predictive value (proportion of individuals who self-reported having diabetes were diabetic based on reference definitions), negative predictive value (proportion of individuals who did not report having diabetes were not diabetic based on reference definitions) and κ value compared with the reference definitions of diabetes. The validity was evaluated overall, by 5-year age groups, education levels and by rural and urban areas. Sensitivity analysis using reference definitions excluding traditional Chinese antidiabetic medication was also performed. All analyses were performed in SAS V.9.3 (SAS Institute, Cary, North Carolina, USA).

Results

Characteristics of the participants are shown in table 1. The participants had a mean age of 59.5 years old. Most had less than high school education and 28.6% were illiterate. The majority of the participants (80.2%) lived in rural areas. Overall, the participants had optimal fasting blood glucose, HbA1c and body mass index. A total of 7.8% of the participants reported physician-diagnosed diabetes, and most of them took modern medicine to treat diabetes; however, 22.6% of them did not treat their diabetes. Based on the reference definitions, 15.9% of the participants had diabetes.

Table 1

Baseline characteristics of the participants, the China Health and Retirement Longitudinal Study, 2011–2012

	Participants (n=11 601)
Age, mean (SD)	59.5 (9.6)
Male%	47.3
Education%
Illiterate	28.6
Less than primary school	18.4
Primary school	21.8
Middle school	20.3
High school	7.0
Above vocational school	3.9
Living areas%
Rural	80.2
Urban	19.8
Self-reported diabetes%	7.8
Confirmed diabetes using definition 1*, %	13.7
Confirmed diabetes using definition 2†, %	13.8
Treatment method of diabetes%
Modern medicine	63.3
Traditional Chinese medicine	13.4
Insulin	14.7
None	22.6
Blood glucose. mg/dL, mean (SD)	110.5 (37.6)
HbA1c, %, mean (SD)	5.3 (0.8)
Body mass index, kg/m2, mean (SD)	23.6 (4.0)

*Defined as fasting glucose ≥126 mg/dL or random glucose ≥200 mg/dL or HbA1C ≥6.5% or taking oral diabetes medication (including only modern medicine) or using insulin.

†Defined as fasting glucose ≥126 mg/dL or random glucose ≥200 mg/dL or HbA1C ≥6.5%, taking oral diabetes medication (including traditional Chinese medicine and modern medicine) or using insulin.

HbA1C, glycated hemoglobin.

The validity parameters of self-reported diabetes are presented in table 2. Self-reported diabetes had high specificity (98.6%), positive predicted value (84.7%) and negative predicted value (89.9%). However, the sensitivity of self-reported diabetes was only 41.5%. Overall, self-reported diabetes and the reference definitions had moderate agreement (κ=0.505). In different 5-year age groups, sensitivity varied from 33.2% to 47.1%, specificity was all above 98%, positive predicted values (PPV) ranged from 79.4% to 96.8%, and negative predicted values (NPV) were between 84.5% and 92.2%. In each age group, self-reported diabetes had moderate agreement with the reference definitions, with the 60–64 age group having the highest κ value (0.548). The sensitivity of self-reported diabetes increased with education levels. The specificities were also all above 98% among participants with different education levels. PPV and NPV ranged from 79.6–92.2% and 87.5–91.4%, respectively, in different education levels. Self-reported diabetes among participants with above vocational school education had substantial agreement (κ=0.658) with the reference definition. Furthermore, participants living in rural areas had much lower sensitivity and PPV compared to those living in urban areas. However, the specificity and NPV are similar between the two populations. Finally, urban residents had a much higher κ value than rural residents (0.646 vs 0.441).

Table 2

Validity of self-reported diabetes using reference definitions* in the baseline survey of the China Health and Retirement Longitudinal Study, 2011–2012

			Sensitivity		Specificity		PPV		NPV		κ
	n	Diabetes†, %	Per cent	95% CI	Per cent	95% CI	Per cent	95% CI	Per cent	95% CI	Per cent	95% CI
Overall	11 601	15.9	41.5	39.3 to 43.7	98.6	98.4 to 98.8	84.7	82.4 to 87.1	89.9	89.3 to 90.5	50.5	48.2 to 52.9
Age groups
45–49	2181	11.2	33.2	27.3 to 39.1	98.9	98.5 to 99.4	79.4	71.6 to 87.3	92.2	91.0 to 93.3	43.1	36.5 to 49.7
50–54	1697	14.5	39.4	33.3 to 45.5	98.6	98.0 to 99.2	82.9	76.1 to 89.7	90.6	89.1 to 92.0	48.6	42.2 to 55.1
55–59	2428	16.4	42.4	37.5 to 47.2	98.8	98.4 to 99.3	87.6	82.9 to 92.2	89.7	88.5 to 91.0	52.0	47.0 to 56.9
60–64	2053	17.7	47.1	42.0 to 52.2	98.2	97.5 to 98.8	84.7	79.7 to 89.6	89.6	88.2 to 91.0	54.8	49.7 to 59.9
65–69	1319	17.4	43.5	37.1 to 49.9	98.2	97.4 to 99.0	83.3	76.7 to 90.0	89.2	87.4 to 90.9	51.3	44.7 to 57.9
70–74	964	18.1	43.1	35.8 to 50.5	98.1	97.2 to 99.1	83.3	75.6 to 91.0	88.7	86.6 to 90.8	50.8	43.2 to 58.4
75–79	600	18.2	39.5	30.3 to 48.6	98.8	97.8 to 99.8	87.8	78.6 to 96.9	88.0	85.3 to 90.7	48.6	38.9 to 58.4
≥80	359	22.6	37.0	26.5 to 47.6	99.6	98.9 to 1.00	96.8	90.6 to 1.00	84.5	80.5 to 88.4	46.9	35.6 to 58.3
Education
Illiterate	3317	14.9	35.6	31.3 to 39.8	98.5	98.0 to 98.9	80.4	75.1 to 85.6	89.7	88.6 to 90.8	44.2	39.6 to 48.8
<Primary school	2137	15.8	38.3	33.1 to 43.5	98.2	97.6 to 98.8	79.6	73.4 to 85.8	89.5	88.1 to 90.8	46.2	40.6 to 51.8
Primary school	2523	15.6	43.5	38.6 to 48.4	98.9	98.5 to 99.4	88.1	83.6 to 92.7	90.5	89.3 to 91.7	53.5	48.5 to 58.5
Middle school	2350	16.1	42.6	37.6 to 47.6	98.9	98.4 to 99.4	88.0	83.3 to 92.7	90.0	88.7 to 91.3	52.4	47.3 to 57.5
High school	810	14.8	46.7	37.7 to 55.6	98.4	97.5 to 99.3	83.6	74.7 to 92.5	91.4	89.4 to 93.4	55.1	46.3 to 64.0
>Vocational school	452	26.1	60.2	52.3 to 69.0	98.2	96.8 to 99.6	92.2	86.2 to 98.2	87.5	84.1 to 90.8	65.8	57.5 to 74.0
Living areas
Rural	9295	14.3	35.0	32.4 to 37.5	98.7	98.4 to 98.9	81.4	78.2 to 84.6	90.1	89.5 to 90.7	44.1	41.3 to 47.0
Urban	2292	22.6	58.2	54.0 to 62.5	98.2	97.6 to 98.8	90.4	87.2 to 93.6	89.0	87.6 to 90.4	64.6	60.6 to 68.5

*Defined as fasting glucose ≥126 mg/dL or random glucose ≥200 mg/dL or HbA1C ≥6.5% or taking oral diabetes medication (including traditional Chinese medicine and modern medicine) or using insulin.

†Confirmed using the reference definitions.

HbA1C, glycated hemoglobin; NPV, negative predicted values; PPV, positive predicted values.

The sensitivity analysis using the reference definitions including only modern antidiabetic medications (table 3) showed similar specificity (96.3%) and NPV (90.0%). However, the sensitivity (32.3%) and PPV (58.4%) of self-reported diabetes were much smaller, and the agreement between self-reported diabetes and the reference definitions was only fair (0.353).

Table 3

Validity of self-reported diabetes in the baseline survey of CHARLS using the reference definition not including taking Traditional Chinese medicine*

		Reference Diabetes, %	Sensitivity		Specificity		PPV		NPV		κ
	n		Per cent	95% CI	Per cent	95% CI	Per cent	95% CI	Per cent	95% CI	Per cent	95% CI
Overall	11 576	13.7	32.3	30.0 to 34.7	96.3	96.0 to 96.7	58.4	55.1 to 61.6	90.0	89.4 to 90.5	35.3	32.7 to 37.9
Age groups
45–49	2179	10.1	26.0	20.2 to 31.8	97.8	97.2 to 98.5	57.0	47.3 to 66.7	92.2	91.1 to 93.4	31.4	24.5 to 38.3
50–54	1694	13.2	33.5	27.3 to 39.7	97.4	96.5 to 98.2	65.8	57.1 to 74.5	90.6	89.1 to 92.0	38.9	32.1 to 45.8
55–59	2421	14.1	32.8	27.9 to 37.8	96.4	95.7 to 97.2	60.2	53.2 to 67.3	89.8	88.5 to 91.0	36.2	30.6 to 41.7
60–64	2051	15.1	39.0	33.6 to 44.5	95.5	94.5 to 96.4	60.5	53.7 to 67.3	89.8	88.4 to 91.2	40.4	34.6 to 46.2
65–69	1315	14.8	33.0	26.4 to 39.6	95.4	94.1 to 96.6	55.2	46.1 to 64.2	89.2	87.4 to 90.9	34.0	26.7 to 41.4
70–74	961	14.9	31.5	23.9 to 39.1	94.9	93.4 to 96.4	51.7	41.2 to 62.2	88.8	86.7 to 90.9	31.4	22.9 to 39.9
75–79	599	15.7	29.8	20.5 to 39.0	96.0	94.3 to 97.7	58.3	44.4 to 72.3	88.0	85.3 to 90.7	32.3	21.6 to 42.9
≥80	356	17.4	17.7	8.2 to 27.3	94.2	91.6 to 96.9	39.3	21.2 to 57.4	84.5	80.5 to 88.4	15.3	3.0 to 27.5
Education
Illiterate	3315	13.5	29.4	25.2 to 33.6	97.0	96.4 to 97.7	60.8	54.3 to 67.3	89.8	88.7 to 90.8	33.8	28.9 to 38.7
<Primary school	2132	13.9	30.6	25.4 to 35.9	96.4	95.6 to 97.3	58.0	50.2 to 65.7	89.6	88.2 to 90.9	33.7	27.7 to 39.7
Primary school	2521	13.5	34.8	29.7 to 39.9	96.6	95.9 to 97.4	61.5	54.6 to 68.3	90.5	89.3 to 91.7	38.5	32.9 to 44.0
Middle school	2341	13.3	30.8	25.7 to 35.9	96.2	95.3 to 97.0	55.2	47.8 to 62.6	90.0	88.8 to 91.3	33.1	27.4 to 38.9
High school	809	12.5	36.6	27.2 to 46.0	95.9	94.4 to 97.4	56.1	44.1 to 68.0	91.4	89.4 to 93.4	38.2	28.2 to 48.2
>Vocational school	446	19.3	45.4	34.8 to 55.9	91.1	88.2 to 94.1	54.9	43.4 to 66.5	87.5	84.1 to 90.8	39.1	28.1 to 50.0
Living areas
Rural	9288	13.0	28.6	26.1 to 31.2	97.3	97.0 to 97.7	61.3	57.3 to 65.3	90.1	89.5 to 90.8	33.5	30.5 to 36.5
Urban	2274	16.7	44.2	39.2 to 49.2	92.2	91.0 to 93.5	53.3	47.8 to 58.8	89.2	87.8 to 90.6	39.1	34.0 to 44.2

*Defined as fasting glucose ≥126 mg/dL or random glucose ≥200 mg/dL or HbA1C ≥6.5% or taking oral diabetes medication (including only modern medicine) or using insulin.

CHARLS, China Health and Retirement Longitudinal Study; HbA1C, glycated haemoglobin NPV, negative predicted values; PPV, positive predicted values.

Discussion

To the best of our knowledge, this is the first study evaluating the validity of self-reported diabetes among the middle-aged and older Chinese population. We found that self-reported diabetes was 40.21% sensitive and 98.47% specific. In general, self-reported diabetes did not perform well in the middle-aged and older Chinese population. However, self-reported diabetes among participants with above vocational education or living in urban areas had high sensitivity and substantial agreement with the reference definitions.

In the current study, self-reported diabetes largely underestimated the prevalence of diabetes in the middle-aged and older Chinese population. This is different from studies reported in developed countries such as the USA, where self-reported diabetes was 55–80% sensitive.8 This difference is likely driven by the education level of the participants. The proportion of people who had received formal schooling higher than high school was only around 10% in the current study, which is much lower than that (above 81%) in the study conducted in the USA.8 On the contrary, sensitivity of self-reported diabetes from participants with high school or above vocational school education levels was similar to that in the USA.8 Our study indicates that blood glucose and/or HbA1c tests are necessary to accurately estimate the prevalence of diabetes in large nationwide diabetes studies.

Similar to studies in most countries, self-reported diabetes was highly specific in the current study.3–8 In addition, the PPV are very high in the current study. Hence, for studies with patients with diabetes, self-report is a useful tool to identify and recruit participants.

Self-reported diabetes performed much better among urban residents compared to rural residents in the current study. This may reflect differences in education and access to healthcare services between populations in these areas. Previous studies indicated that higher education and a recent health examination increased the sensitivity of self-reported diabetes.11 In China, urban residents had a higher education level and better access to medical services.16

Our study has several important strengths. First, the CHARLS participants were a representative sample of China's middle-aged and older population. So the findings can be generalised to all middle-aged and older Chinese adults. Second, most of the previous studies on the validity of self-reported diabetes used medical records as the ‘gold standard’, which may miss cases of undiagnosed diabetes, particularly for rural residents whose access to healthcare services is limited.16 The current study used fasting blood glucose and HbA1c levels to confirm diabetes, which eliminated cases of undiagnosed diabetes. Third, stringent quality control and quality assurance measures were implemented in every stage of the CHARLS study. Therefore, the quality of the current study can be guaranteed. Our study also has limitations. First, the response rate of blood samples collection was a bit low (67%), and according to the guideline of CHARLS, younger men were more likely to be missed in blood sample collection, mostly due to time conflict with their work. Since younger people have lower diabetes prevalence, lower response rate of this population will decrease the negative predictive values. Second, we included using traditional Chinese antidiabetic medication in the reference definitions, which was different from studies outside China. However, traditional Chinese medicine is an integral component of the healthcare system in China, where it is practiced side by side with conventional medicine in most hospitals and clinics.17 Many patients seek modern medicine and traditional Chinese medicine to manage diabetes. If not considering traditional Chinese medicine in the reference definitions, many cases would be missed. As in the current study, the prevalence of diabetes using the reference definitions including traditional Chinese medication was very close to that in a recent large scale national survey on diabetes in 2010.9 On the other hand, the diabetes prevalence using reference definitions excluding traditional Chinese medicine was much smaller than that in the national survey in 2010.9 We also performed sensitivity analyses for the validity of self-reported diabetes excluding participants only taking traditional Chinese medicine to manage their diabetes, as shown in the online supplementary table, and the validity of self-reported diabetes slightly decreased.

In conclusion, self-reported diabetes among the middle-aged and older Chinese population is 41.5% sensitive, indicating a low awareness rate of diabetes in this population. Public health efforts to increase the awareness of diabetes in this population are warranted. Self-reported diabetes is highly specific and the positive predictive values are fairly good. So self-reported diabetes can be used to identify and recruit diabetic participants. Furthermore, self-reported diabetes had substantial agreement with the reference definitions among urban residents and among participants with above vocational education. For large scale epidemiological studies among urban residents or participants with high education levels in China, self-reported diabetes is a useful tool to measure diabetes.