Development and Validation of a New Diabetes Risk Score in Guyana.

INTRODUCTION: We present a new diabetes risk score developed and validated in a multi-ethnic population in Guyana, South America. Measurement of in-country diabetes prevalence is a vital epidemiologic tool to combat the pandemic. It is believed that for every person diagnosed with type 2 diabetes there is another undiagnosed. The International Diabetes Federation (IDF) recommends a two-step detection programme using a risk score questionnaire to identify high-risk individuals followed by glycaemic measure.
METHODS: Data on 798 persons from the 2016 STEPwise Approach to Chronic Disease Risk Factor Surveillance (STEPS) were used to correlate responses to 36 questions with glycated haemoglobin (HbA1C) and fasting plasma glucose (FPG) results. Bootstrapping was used to internally validate the derived seven-variable model. This model with the addition of family history questions was tested in a convenience sample of 659 Guyanese adults and externally validated in a cohort of another 528.
RESULTS: An 8-item Guyana Diabetes Risk Score (GDRS) was derived. The final model performed with an area under the curve (AUC) of 0.812
CONCLUSIONS: The validated eight-item Guyana Diabetes Risk Score will be extremely useful in identifying individuals at high risk of having diabetes in Caribbean, Black or East Indian populations.

Key Summary Points

Introduction

Type 2 diabetes (T2D) has a long asymptomatic pre-clinical phase which frequently goes undetected. It is generally recognized that for every person diagnosed with T2D there is another undiagnosed. The Addition-Europe trial concluded that “major benefits are likely to accrue from the early diagnosis and treatment of glycaemia and cardiovascular risk factors in type 2 diabetes” [1]. The International Diabetes Federation (IDF) recommends a two-step detection programme based on identification of high-risk individuals using a risk questionnaire followed by glycaemic measures [2]. It is estimated that three-quarters of people with diabetes live in low- and middle-income countries [3].

Guyana, the second poorest country in South America, is a small English-speaking country of 750,000 people of mostly African and East Indian descent. Risk scores developed in Caucasian populations cannot be applied to other populations of diverse ethnic origin [4]. Other risk scores developed in the region have been for Hispanic populations [5]. The Guyana Ministry of Public Health (MOPH) was interested in using a diabetes risk score in Guyana. Development of a simple Guyana Diabetes Risk Score (GDRS) became a core objective of the Guyana Diabetes Care Project. We used data from a national risk factor survey to estimate prevalence of diabetes, undiagnosed diabetes and gender ratios [6]. We then correlated answers on this survey with diabetes status and developed a model to predict risk of diabetes. In this article we present a new eight-point diabetes risk score, the GDRS, developed and validated in Guyana.

Methods

Risk Score Development: Derivation Cohort

In 2016, the Guyana MOPH carried out a World Health Organization (WHO) STEPwise Approach to Chronic Disease Risk Factor Surveillance (STEPS). This national survey was conducted in all ten regions of Guyana in adults between ages 25 and 70 years. A multistage cluster sampling technique was used and participants selected using the Kish method [7]. Since fasting plasma glucose (FPG) and glycated haemoglobin (HbA1c) were measured in a survey segment, diabetes prevalence could be estimated and is detailed separately [6]. HbA1c was measured at the Georgetown Public Hospital Corporation using a high-performance liquid chromatography assay (D-10®). We used the WHO criteria of either FPG ≥ 126 mg/dl (7 mmol/l) or HbA1c ≥ 6.5% (48 mmol/mol) to diagnose diabetes [2] in this cohort (see Fig. 1).

Fig. 1

Flow diagram of cohorts to derive and validate the Guyana Diabetes Risk Score

We analyzed the relationship between the biochemical diagnosis of diabetes and responses to 36 STEPS questions (see Appendix 1 in ESM). We carried out both unadjusted and multivariable logistic regression to examine the association of pre-specified literature-derived variables [8] [known diabetes; on treatment for diabetes; age; sex; family history of diabetes; ethnicity; blood pressure and body mass index (BMI)] with diabetes status in participants whose glycaemic measure could be linked to responses in STEPS questions. The receiver-operator characteristic (ROC) curve was plotted and the area under the curve (AUC) calculated. Bootstrapping was used to internally validate the model. Results of this analysis were used to select relevant variables for the GDRS questionnaire. Family history questions were inadvertently omitted from the STEPS survey. Since family history is known to be associated with T2D, we added family history questions to the two subsequent cohorts.

Risk Score Development: Test Cohort

The GDRS questionnaire was tested in a convenience sample of Guyanese consenting adults over 25 years of age associated with the Congregations Taking Action against NCDs (CONTACT) Study [9]. The biochemical diagnosis of diabetes was defined as HbA1c ≥ 6.5% (48 mmol/mol) or random capillary blood glucose (RBG) ≥ 200 mg/dl (11.1 mmol/l) as the population was not fasting. The variables of the two cohorts were matched across both data sets. Standardized mean differences (SMD) were used to compare the derivation and test cohorts. A SMD < 0.1 indicates the groups were similar [10]. Multivariable analysis similar to the STEPS cohort was performed. Odds ratios (OR) (95% CI) for unadjusted and adjusted models were calculated. We examined the possibility of non-linear relationships for several continuous variables using restricted cubic splines with three knots. We transformed the final model into a simple point system to improve ease of use [11].

External Validation: Validation Cohort

Using the statistical weights derived from the test sample, the GDRS was externally validated in a convenience sample of non-fasting persons attending 13 MOPH health outreach events (Fig. 1). Participants were defined as having diabetes if they had a blood sugar ≥ 200 mg/dl (11.1 mmol/l) (irrespective of the last food eaten) or an HbA1c ≥ 6.5% (48 mmol/mol).

We measured the performance of the risk score model derived from the test cohort using the validation cohort. We described model discrimination using the c-statistic and calibration using Brier’s score and examined for model over-fitting.

Missing data were minimal (< 5%) and handled by complete case analysis. All analyses were performed in R using the RMS package [12].

Ethical Approval

All phases of this study received approval from Guyana’s Institutional Review Board FWA00014641 as Protocol #278. Persons with elevated blood tests suggesting diabetes were contacted by MOPH. The study was conducted in accordance with the Declaration of Helsinki 1964 and its later amendments. Informed consent was obtained from all participants of this study.

Results

The numbers and exclusions in each cohort are shown in Fig. 1. We eliminated those variables (smoking, exercise, diet) not strongly correlated with diabetes status in the STEPS cohort. Since waist circumference and BMI are highly correlated (ρ = 0.8) and result in multicollinearity, MOPH opted to use BMI only.

There were some demographic differences between the three cohorts (Table 1). Afro-Guyanese made up 39% of the GDRS test cohort, 46.7% of the validation cohort and 29% of the STEPS cohort. Persons of Afro-Guyanese ethnicity are generally considered to constitute 30% of the population [13]. The Indo-Guyanese percentages are 39%, 41% and 23.1%, respectively. The GDRS test cohort had only 1% persons with Amerindian ethnicity, while STEPS had 9% and the validation cohort 4.6%. The mean age and SD also differ: 49.1 (14.1) for the GDRS test cohort, 47.2 (14.1) for the validation cohort and 41.8 (14.5) for the STEPS. Finally, the overall percentage with diabetes in the test cohort was higher at 28%.

Table 1

Comparison GDRS and STEPS cohorts

	GDRS validation DM positive	GDRS test-DM positive	STEPS-DM positive	SMD	GDRS validation DM negative	GDRS test-DM negative	STEPS-DM negative	SMD
Number	213	186	136		309	474	662
Told have diabetes (%)	94 (44.1)	96 (51.6)	64 (47.1)	0.091	19 ( 6.1)	50 (10.5)	32 (4.8)	0.216
Currently treated for T2D (%)	76 (35.7)	71 (38.2)	55 (40.4)	0.046	9 ( 2.9)	25 (5.3)	17 (2.6)	0.140
Age (years) (SD)	53.43 (13.27)	53.67 (12.97)	49.82 (12.33)	0.304	42.85 (13.07)	46.99 (13.96)	40.17 (14.32)	0.482
Sex = male (%)	59 (27.7)	53 (28.5)	40 (29.4)	0.020	81 (26.2)	154 (32.5)	252 (38.1)	0.117
FH				NA				NA
Both parents (%)	10 (4.7)	8 (4.3)	NA		11 (3.6)	15 (3.2)	NA
Both parents and sibling (%)	7 (3.3)	6 (3.2)	NA		1 (0.3)	6 (1.3)	NA
Do not know/other (%)	8 (3.8)	9 (4.8)	NA		22 (7.1)	24 (5.1)	NA
None (%)	95 (44.6)	56 (30.1)	NA		196 (63.4)	248 (52.3)	NA
One parent (%)	59 (27.7)	65 (34.9)	NA		56 (18.1)	133 (28.1)	NA
One parent and sibling (%)	7 (3.3)	21 (11.3)	NA		0 (0)	19 (4.0)	NA
Sibling (%)	27 (12.7)	21 (11.3)	NA		23 (7.4)	29 (6.1)	NA
Ethnicity				0.442				0.410
African/Black (%)	94 (44.1)	80 (43.0)	36 (26.5)		150 (48.5)	181 (38.2)	198 (29.9)
Amerindian (%)	6 (2.8)	3 (1.6)	11 (8.1)		18 (5.8)	5 (1.1)	64 (9.7)
East Indian (%)	66 (31.0)	75 (40.3)	68 (50.0)		54 (17.5)	181 (38.2)	264 (39.9)
Mixed/other (%)	47 (22.1)	28 (15.1)	21 (15.4)		87 (28.2)	107 (22.6)	136 (20.5)
Mean SBP(SD)	136.48 (21.27)	139.16 (22.27)	140.69 (39.31)	0.048	124.37 (18.03)	130.78 (21.74)	126.68 (21.72)	0.188
Mean DBP(SD)	75.27 (16.72)	84.46 (13.19)	86.40 (13.60)	0.145	72.40 (14.78)	79.34 (14.43)	78.73 (13.02)	0.045
BMI [mean] (SD)	31.10 (17.40)	30.34 (5.73)	29.48 (6.08)	0.146	30.17 (23.82)	28.80 (6.34)	26.91 (6.46)	0.296

Told diabetes. Answered yes to question “Have you ever been told you have diabetes or high blood sugar?”

GDRS Guyana Diabetes Risk Score, T2D Type 2 Diabetes, FH family history, SBP systolic blood pressure, DBP diastolic blood pressure, STEPS STEPwise Approach to Chronic Disease Surveillance

Odds ratios (95% CI) for unadjusted and adjusted models for the GDRS test cohort are shown in Table 2. There was a strong correlation between previous diagnosis of diabetes or current treatment and diabetes diagnosis. Age, mean diastolic blood pressure (DBP) and BMI were also strongly correlated with diabetes. Using the STEPS data alone, the ROC curve had an AUC of 0.829 for the entire sample and 0.734 for those not known to have elevated blood sugar. The internal bootstrap validation indicated minor overfitting. The slope shrinkage factor was 0.930 and the Somers’ Dxy had a corrected value of 0.631 with some slight overfitting as indicated from an optimism of 0.027. The internal bootstrap validation indicated minor overfitting. The slope shrinkage factor was 0.905 and the Somers’ Dxy had a corrected value of 0.540 with some slight overfitting as indicated from an optimism of 0.043.

Table 2

GDRS test survey: unadjusted and adjusted models

Parameter	Unadjusted model				Adjusted model
	OR	Lower 0.95	Upper 0.95	Cstat	OR	Lower 0.95	Upper 0.95
Told diabetes yes: no	9.045	5.999	13.638	0.705	3.764	2.013	7.040
Treatment for diabetes	11.088	6.727	18.277	0.664	2.815	1.318	6.013
Age	1.416	1.248	1.608	0.644	1.137	0.969	1.335
Sex—M:F	0.828	0.571	1.201	0.520	0.936	0.599	1.461
FH at least 1 parent vs. none	2.560	1.750	3.745	0.621	1.888	0.787	4.530
FH only sibling vs. none	3.207	1.704	6.034	–	1.850	1.186	2.887
Do not know/other vs. none	1.661	0.732	3.767	–	1.984	0.922	4.272
Ethnicity Amerindian: African/Black	0.737	0.172	3.157	0.544	1.505	0.274	8.279
Ethnicity East Indian: African/Black	0.691	0.161	2.963	–	0.647	0.404	1.038
Ethnicity mixed/other: African/Black	0.436	0.098	1.936	–	0.636	0.363	1.116
Mean SBP/10 point. change	1.180	1.093	1.273	0.621	NA	NA	NA
Mean DBP/10 point. change	1.281	1.136	1.444	0.620	1.202	1.038	1.391
BMI per 1 unit	1.040	1.012	1.068	0.588	1.037	1.003	1.072

Told diabetes. Answered yes to question “Have you ever been told you have diabetes?”

GDRS Guyana Diabetes Risk Score, M male, F female, FH family history, BMI body mass index, SBP systolic blood pressure, DBP diastolic blood pressure

In the final validation cohort the c-statistic was 0.82, close to the original c-statistic from the training model. The Brier’s score was 0.136, which is fairly close to 0, indicating good calibration. Using a likelihood ratio test we found that there was no statistically significant improvement using RCS and the linear fit was adequate. Lastly, the slope value was 0.892, indicating some minor overfitting.

The final questionnaire scoring is shown in Table 3 and the sensitivity and specificity with different scores in Table 4. The risk score (Fig. 2) has eight variables: gender, age, known diabetes, current treatment, ethnicity, family history, diastolic BP and BMI scored based on the degree of association with diabetes. The impact of the addition of family history questions was statistically significant and improved the risk score in those with unknown diabetes status (see Appendix Table 2). We recommend that a risk score ≥ 10, which maximizes sensitivity with a reasonable number of false positives, be the trigger for biochemical verification of diabetes status in any subsequent surveys. Forty-three per cent of the population will be ruled “out” at this score.

Table 3

GDRS scoring and prediction from test cohort

Table 4

Sensitivity and specificity of GDRS

Risk score points	No. with score	Sensitivity %	Specificity %	PPV %	NPV %	Accuracy %
> 5	635	99.5	5.1	29.1	96.0	31.7
> 10	431	88.2	43.7	38.1	90.4	56.2
> 15	179	59.1	85.4	61.5	84.2	78.0
> 20	114	43.5	93.0	71.1	80.8	79.1
> 25	37	14.5	97.9	73.0	74.5	74.4
> 30	1	0.5	100.0	100.0	71.9	72.0

Fig. 2

Guyana Diabetes Risk Score questionnaire

Discussion

Many risk scores have been developed in various countries and their methodology and reporting levels subjected to systematic review [5, 8]. Use of capillary RBG as a diagnostic measure of diabetes is thought to be appropriate for population surveys [14]. Our score uses eight standard, non-invasive and easily measured variables associated with diabetes and included in many other scores: age, gender, family history of diabetes, blood pressure, BMI, ethnicity and whether participants are known to have diabetes or on treatment. Our multivariable model was developed using both statistical and clinical significance and demonstrated that the model works by bootstrapping in the STEPS derivation cohort. Because prediction models tend to perform better on the sample on which they are derived than on new data, we then externally validated the model using two completely different real-life cohorts of individuals from multiple different locations than those used to derive the model. This compensated for any optimism in the internally validated model and allowed us to add family history questions and use measures of RBG, which is better for community screening when people are unlikely to be fasting. Statistical analysis suggests the results are quite generalizable. Another strength of our study is its use of HbA1c as well as direct measures of glycaemia as it has been shown these measures identify different populations of people with dysglycaemia [15].

Some of the weights assigned to the questions seem counter-intuitive but this is a consequence of the relatively small non-random samples used to test and validate the score. There was no upper age limit in the GDRS test and validation cohorts and this may account for the mean age differences. Ethnicity differences between the three cohorts may have occurred because the STEPS questionnaire offered a mixed ethnicity option. The self-selection of persons at higher risk of diabetes in the GDRS cohorts explains the higher rate of diabetes in these cohorts.

We have had a small trial use of the score where we saw 79 people at a Health Expo over 4 days, 40 of whom were deemed high risk and referred for further management.

Our study is limited by differences in the definition of diabetes in the two surveys with FPG used in STEPS and RBG in the GDRS test and validation cohorts as well as the omission of family history in the STEPS survey. Another limitation is the inclusion of only a small number of people of Amerindian descent. Ethnic differences in risk factors for diabetes have been shown in neighbouring Suriname [16] with an absence of an association with BMI in Amerindians. Other studies have shown a lower rate of diabetes in Amerindians in remote villages [17] but higher rates associated with increased biological risk factors in urban settings [18].

Conclusion

Further work is needed to confirm the applicability of the GDRS to Amerindians. The total number surveyed, high diabetes rate, consistency of the results and internal and external validation suggest that the Guyana Diabetes Risk Score will be extremely useful for identifying those at high-risk of having diabetes in the Caribbean as well as countries such as the UK and Mauritius with individuals of African, East Indian or mixed descent.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 16 kb)

Why carry out this study?

A two-step diabetes screening programme (risk score followed by a glycaemic measure) is recommended.

Most risk scores have been developed in Caucasian populations and few have been properly internally and externally validated.

What was learned from the study?

A risk score developed from a nationally representative sample and validated in two convenience samples had an area under the curve (AUC) of 0.812.

The eight-item Guyana Diabetes Risk Score is suitable for screening for diabetes in African and Indian Caribbean lineage populations.