A Pilot Retrospective Study Validating Noncommunicable Disease Risk Assessment Score (AMNRAS).

BACKGROUND: Globally Noncommunicable Diseases is lead causes of mortality. This calls for the need of sensitive and cost appropriate screening tools to identify asymptomatic healthy individuals with higher risk and/or subclinical NCD in the community. The study aims to generate pilot evidence based, validated, good quality, cost appropriate, and sustainable risk assessment score for NCD for developing countries like India.
METHODS: This descriptive retrospective study of diabetic camp data of 84 patients was conducted. A risk score having 10 questions and three measurements for NCDs appropriate for Indian communities was generated. It was compared to IDRS, FINDRISC, FRS, CBAC, and WHO/ISH prediction charts.
RESULTS: The study finally included 36 patients with NCD as case and 44 subjects without NCD as control. The means of weight, waist circumference, blood pressure, and blood sugar were significantly different among the two groups. AMNRAS of more than 14 was highly predictive for an individual to be at risk of NCD or sub clinical case of NCD requiring evaluation. The proposed cut-off of 8 for AMNRAS, the sensitivity and NPV was highest compared to other score, 88.9% and 84.6%, respectively. Score for the Area under curve was significantly higher for AMNRAS [0.83 (0.74 to 0.92)] compared to other scores.
CONCLUSIONS: AMNRAS has higher performance parameters than the other five tested in the present study. Other scoring system performs only modestly in discrimination of NCD cases. The accuracy of AMNRAS for NCD risk will have to be determined in large size cohorts. Copyright:

Introduction

Noncommunicable Diseases (NCDs) are the leading causes of death globally, killing more people each year than all other causes combined. Despite their rapid growth and inequitable distribution, much of the human and social impact caused each year by NCD-related deaths could be averted through well-understood, cost-effective, and feasible interventions. Of 56.9 million global deaths in 2016, 71% were due to noncommunicable diseases (NCDs) namely cardiovascular diseases, cancers, diabetes, and chronic lung diseases. The burden of NCDs is rising more in lower income countries and populations.[1]

Non-communicable diseases (NCDs) contribute to 60% of all deaths in India. The major causes of deaths were Coronary Heart Disease, Stroke, and Hypertension (45%), Chronic respiratory disease (22%), Cancers (12%) and Diabetes (3%), respectively.[2]

NCDs are caused mainly by four behavioral risk factors that are pervasive aspects of economic transition, rapid urbanization, and lifestyle: tobacco use, unhealthy diet, insufficient physical activity, stress, and the harmful use of alcohol.[3] The greatest effects of these risk factors are on poorer people, more so in low and low middle income countries. Among these populations, a vicious circle may ensue: poverty exposes people to behavioral risk factors for NCDs and, in turn, the resulting NCDs morbidity and mortality that leads families towards poverty.[4] Majority of these people appear apparently healthy but they may still have NCDs.[5]

Increasing incidence of NCDs in the developing countries has necessitated adoption of preventive approaches at community level aimed at controlling risk factors to reduce NCD morbidity and mortality.[5] There is need of sensitive and cost appropriate screening tools to identify asymptomatic healthy individuals with higher risk and/or subclinical NCD in the community. Number of risk scoring systems has been developed for this purpose. Majority of them are for individuals diseases.[678] In addition due to marked variations in risk level because of difference in culture and lifestyle, these risk scores cannot be uniformly applied to the entire world.[9] The study aim to generate a pilot evidence based, and validated, risk assessment score for NCD for India.

Methods

This descriptive retrospective study was conducted by Community and Family Medicine, All India Institute of Medical Sciences (AIIMS) Mangalagiri, Andhra Pradesh, which is a tertiary care teaching hospital in South India. Medical records of self-reported patient in a health camp conducted in the year 2018 were retrospectively assessed. Available Data of 84 patients at the time of diagnosis were retrieved, which concerned the following variables at the time of registration: age, sex, socio-demographic factors, behavioral risk factor, medical history, anthropometric indices, blood pressure, and blood sugar. Due to missing information the final data were assessed for 80 patients only. A risk score for noncommunicable diseases appropriate for Indian communities and lifestyle was generated. This was on the basis of data on risk factors and blood sugar. Patient having diabetes, hypertension, or cardiovascular disease were outcome variables. This includes all risk factors and has ten questions and three measurements. This study was undertaken to compare five existing scores with new score and check its validity for risk assessment for identifying individuals at risk for NCDs.

Hypothesis was the screening test will be considered to be of public health utility if it has a sensitivity of 75% and above and a specificity above 65% in detecting people with high risk of a range of major NCDs (Coronary Artery Disease, Diabetes, Chronic Obstructive Pulmonary Disease, Stroke, Cancer). Variable studies were sensitivity and specificity of the six scales, misclassification rates, and their implications for management and Area under Curve (AuC).

By definition risk score validity is the ability of a questionnaire/engine/tool to accurately identify diseased and non-disease individuals. The validity of a risk score is based on its accuracy in identifying NCD persons, and this can only be determined if when compared to some “gold standard” that establishes the true NCD status. All patients were categorized into case or control. Gold standard criteria used to define a case was a patient aged 20 years and above having confirmed diagnosis of one or more of the following five non communicable diseases like Diabetes, Hypertension, Cardiovascular disease, Stroke, and Cancer. The confirmed diagnosis was on previous doctor-based consultation with supporting laboratory evidence and/or treatment history. A control was defined as patient not currently diagnosed to have any of the above five noncommunicable disease.

The five scoring systems used in the study are pilot AIIMS Mangalagiri Non-communicable disease Risk Assessment Score (AMNRAS), Community Based Assessment Checklist (CBAC)[5] for Early Detection of NCDs, Indian diabetes risk score (IDRS),[10] Finnish Diabetes Risk Score (FINDRISC)[7], World Health Organization/International Society of Hypertension (WHO/ISH) risk prediction charts[8] and Framingham Risk Score (FRS).[11] Comparison of the five scales and cut off values used for each scale is stated in Table 1.

Table 1

Comparison of various risk scores

Variable	AMNRAS		CBAC risk score		IDRS		FINDRISC		WHO/ISH		FRS
Gender									Male (M)/ Female (F)	Modelling	Male/ Female	Modelling
Age in years	≥50	3	≥50	2	≥50	30	>64	4	Age		30-74 years
	40-49	2	40-49	1	35-49	20	55-64	3
	30-39	1	30-39	0	<35	0	45-54	2
	<30	0					<45	0
Body Mass Index (BMI)	≥30	3					>30	3			BMI
	25.0-29.9	2					>25-30	1
	18.5-24.9	1					<25	0
	<18.5	0
Waist circumference	≥90 F ≥100 M	2	≥90 F ≥ 100 M	2	≥90 F ≥100 M	20	>88 F;>102 M	4
	80-89 F 90-99 M	1	80-89 F 90-99 M	1	80-89 F 90-99 M	10	80-<88 F, 94-<102 M	3
	<80 F < 90 M	0	<80 F < 90 M	0	<80 F < 90 M	0
Systolic Blood Pressure	≥160	3							SBP
	140-159	2
	120-139	1
	<120	0
Physical Activity > 30 min or Heavy worker	Absent	2	<150 min/wk	1	No Exercise and Sedentary work	30	<4 h/wk	2
	Present	1	Atleast 150 min/wk	0	Exercise OR Strenuous Work	20	≥4 hr/wk	0
					Exercise AND Strenuous Work	0
Self perceived Stress	Present	1
	Absent	0
Any Form of tobacco use	Present	1	Daily	2					Smoker		Current Smoking
	Absent	0	Past/Sometimes	1					Non Smoker
	Never	0
Alcohol use	Present	2	Yes	1
	Absent	0	No	0
Fruits	< 2 days a week	1					Daily consumption of vegetables, fruits, or berries	1
Diabetes Mellitus							History of high blood glucose	5	Present		Present
									Absent
Hypertension							H/O of BP medication present	2			SBP
	Absent	0					H/O of BP medication absent	0
Positive Family History (Hx) Diabetes	Parents OR siblings	2	Positive Family Hx for any of HTN, DM, Heart disease 2	Both parents	20	1st Degree relative	5
	Absent	0			Either parent	10	2nd Degree relative	3
					No	0	No	0
Positive Family Hx Hypertension	Parents OR siblings	2
	Absent	0
Cholesterol									Total		Total HDL
Total Score (Range)	0-24		0-10		0-100		26		Green <10% to Deep Red >=40%		Absolute Risk
Operational Cut offs	8		>4		60		11		>20%		>Twice than optimum AR
Risk Assessment	NCD		NCD		Diabetes		Diabetes		CVD		CVD

Operational definition of NCD case: Score of more than 4 in CBAC was considered to be case. The present study did not included questions related to symptoms for cancer cervix, breast cancer, oral cancers and COPD in CBAC.[5] For IDRS a score of more than 60 was considered to be a case.[12] FINDRISC score was calculated using a online calculator. Patient was categorized case if the patient risk score was more than 11.[13] WHO/ISH risk prediction charts were used to categorize the patients for different range of cardiovascular risk. Risk of more than 20% was considered to be a case.[14] FRS score was calculated using a online calculator that is general CVD risk prediction using BMI. Patient was categorized as case if the patient risk value was more than twice the optimum value.[11]

Data were entered in Microsoft excel spreadsheet and analyzed with SPSS version 20.0 (IBM Corp. SPSS Statistics for Windows Released 2011.). Descriptive analysis was done. Proportions and percentages with 95% CI were given wherever necessary. The scoring systems were compared in terms of sensitivity, specificity, postive likelihood ratios, negative likelihood ratios, and predictive values. The other summative indicator used was the area under ROC curve. Inferential statistics was applied with P value < 0.05 as significant.

All information collected in the present study was a part of routine health camps conducted by the department. An informed verbal consent was obtained from the patients. This was a secondary data analysis; hence, approval was obtained after completion of the study from the authority, AIIMS, Mangalagiri.

Results

The study finally included 36 patients with noncommunicable disease i.e either Diabetes Mellitus, Hypertension, CVD or multiple, as case and 44 subjects without non communicable disease as control. There was a preponderance of males (75%) especially among diseased (77.8%) which reflects the general male preponderance seen among outpatient attendees at community-based health camps.

Of the total 30% of the study participants belongs to 30-49 years of age. Nearly 52.5% of the study participants were elderly. The profile of NCDs among the subjects included wide range of diseases reflecting their prevalence in the population. Hypertension, Diabetes, and Heart Disease was present in (14) 17.7%, 28 (35%) and 4 (5%) patients, respectively. Of total patients had 9 patients had both diabetes and hypertension. Among the hypertensives, blood pressure was not controlled in all but one with mean blood pressure 154/93. No cancer cases were included.

The comparison between subjects with disease and without NCDs on presence of different established risk factors which are included in the risk assessment tools is shown in Table 2. It shows that means weight, mean waist circumference, mean blood pressure, and mean blood sugar was significantly different among the two groups. Thus, risk assessment tools which had included these measurements and had given them higher weight/score are likely to perform better in discrimination between noncommunicable disease patients. However, current tobacco use was found more among people without noncommunicable disease. This could because people with disease were likely to have left tobacco use as seen in slightly more proportion of past users. The distribution of AMNRAS of subjects with and without non communicable disease showed that subjects with noncommunicable disease had higher scores, though there was a degree of overlap. AMNRAS of more than 14 was highly predictive for an individual to be at risk of NCD or case of NCD requiring evaluation.

Table 2

Distribution of the main risk factors for NCDs in the study subjects

Domain	Variable	Indicator	NCD case	Non NCD control	P
Health behaviors by Interview
Demography	Mean Age in years (SD)		59.03 (11.3)	59.6 (15.04)	0.841
	Gender	Male	28 (77.8)	32 (72.7)	0.604
		Female	8 (22.2)	12 (27.3)
	Education	Illiterate	18 (50.0)	22 (50.0)	0.861
		Less than Primary	5 (13.9)	9 (20.5)
		Primary	6 (16.7)	6 (13.6)
		Secondary or more	7 (19.4)	7 (15.9)
	Occupation	Farmer/Labour	27 (75.0)	28 (63.6)	0.275
		Other	9 (25.0)	16 (36.4)
Tobacco	Current Smokers [in last 1 month]		4 (11.1)	9 (20.5)	0.26
	Mean No. of Cigrattes/bidis smoked per week (SD)[n=13]		9.7 (7.1)	26.0 (43.3)	0.251
Alcohol	Current alcohol user [in last 1 month]		2 (5.6)	5 (11.4)	0.36
	Mean No. of times alcohol consumed per week (SD)[n=7]		4.0 (4.2)	2.5 (3.0)	0.925
Nutrition	Proportion consuming fruits		16 (44.4)	18 (40.9)	0.75
	Mean No. of times Fruits consumed by week (SD)[n=34]		0.63 (0.8)	1.6 (3.2)	0.319
	Proportion consuming Vegetables		29 (80.6)	27 (61.4)	0.06
	Mean No. of times Vegetable consumed by week (SD)[n=56]		8.6 (3.5)	9.3 (3.3)	0.58
Physical activity	Active adults		16 (44.4)	18 (40.9)	0.75
Others	Self perceived Stress present		5 (13.9)	4 (9.1)	0.499
	Family History of Hypertension		11 (30.6)	6 (13.6)	0.06
	Family History of Cardiovascular disease		1 (2.8)	0 (0.0)	0.26
	Family History of Diabetes Mellitus		12 (33.3)	7 (15.9)	0.06
	Perception of Risk of Hypertension present [n=66]		13 (36.1)	4 (9.3)	0.004*
	Perception of Risk of Diabetes Mellitus present [n=52]		10 (27.8)	2 (4.5)	0.004*
	On Regular treatment for raised BP [n=14]		12 (85.7)	.	.
	On Regular treatment for raised BS[n=28]		25 (89.3)	.	.
Physical measurement Obesity	Mean Height (SD)		164.4 (9.0)	160.4 (17.9)	0.23
	Mean Weight (SD)		72.1 (14.3)	65.9 (12.5)	0.04*
	Mean Waist circumference (SD)		99.0 (11.9)	93.1 (13.6)	0.04*
Blood Pressure	Mean Systolic BP (SD)		142.8 (22.0)	133.8 (19.3)	0.05*
	Mean Diastolic BP (SD)		89.3 (14.4)	81.6 (15.1)	0.02*
Blood Sample
Diabetes Mellitus	Random Blood Sugar		175.3 (86.1)	121.1 (42.3)	0.001*

*P<0.05 was considered significant

Subsequently, performance of each of the risk score was assessed individually first and then a comparison between them was attempted. Table 3 shows that at the proposed cut-off of 8 for AMNRAS, the sensitivity was at acceptable level of 88.9% (73.9% to 96.9%) but specificity was a bit lower than acceptable at 50.0% (34.5% to 65.4%) (Instead of 65%). This cut-off gave the best trade-off between sensitivity and specificity. IDRS at the suggested cut-off of 60, it gave a good sensitivity (72.2%) and a slight higher specificity (56.8%). On contrary FINDRISC, WHO/ISH, and FRS gave a good specifictity but a poor sensitivity. Again, we note that a achieving high values of both sensitivity and specificity is difficult to get. At the cut-off of seventy, the sensitivity and especially specificity were both below par. A cut-off of sixty gave a good sensitivity bit a very poor specificity. Thus, addition of tobacco to the score did not help much. Postive likelihood ratio of positive test was higher for AMNRAS (1.78) compared to all scores other than FINDRISC. This reveals that a person who is screened positive by the AMNRAS is nearly double chance to have the disease as compared to the person who was screened negative by the same. The Negative predictive value was highest for AMNRAS (84.6%). This shows nearly 84% of people with a negative AMNRAS truly don't have the disease. Area under the curve for AMNRAS compared to other scores is depicted in [Figure 1]. Score for the Area under curve was significantly higher for AMNRAS [0.83 (0.74 to 0.92)] as shown in Table 4.

Table 3

Diagnostic values of the Different Scores Compared for AMNRAS

Parameter	AMNRAS		CBAC risk score		IDRS		FINDRISC		WHO/ISH risk score		FRS
	Value	95%CI	Value	95%CI	Value	95%CI	Value	95%CI	Value	95%CI	Value	95%CI
True Positive	32		14		26		12		13		9
False Negative	4		8		10		24		33		27
True Negative	22		36		25		42		32		16
False Positive	22		22		19		2		12		37
Sensitivity	88.89%	73.94%-96.89%	38.89%	23.14%-56.54%	72.22%	54.81%-85.80%	33.33%	18.56%-50.97%	36.11%	20.82%-53.78%	25.00%	12.12%-42.20%
Specificity	50.00%	34.56%-65.44%	81.82%	67.29%-91.81%	56.82%	41.03%-71.65%	95.45%	84.53%-99.44%	72.73%	57.21%-85.04%	84.09%	69.93%-93.36%
Positive Likelihood Ratio	1.78	1.29-2.44	2.14	1.01-4.52	1.67	1.13-2.48	7.33	1.75-30.67	1.32	0.69-2.53	1.57	0.65-3.80
Negative Likelihood Ratio	0.22	0.08-0.59	0.75	0.56-1.00	0.49	0.27-0.88	0.7	0.55-0.89	0.88	0.65-1.19	0.89	0.71-1.12
Positive Predictive Value	59.26%	51.44%-66.64%	63.64%	45.29%-78.72%	57.78%	47.97%-67.01%	85.71%	58.93%-96.17%	52.00%	36.14%-67.47%	56.25%	34.69%-75.68%
Negative Predictive Value	84.62%	67.58%-93.55%	62.07%	54.91%-68.74%	71.43%	58.17%-81.80%	63.64%	57.93%-68.99%	58.18%	50.63%-65.37%	57.81%	52.17%-63.26%
Accuracy	67.50%	56.11%-77.55%	62.50%	50.96%-73.08%	63.75%	52.24%-74.21%	67.50%	56.11%-77.55%	56.25%	44.70%-67.32%	57.50%	45.94%-68.49%

Figure 1

ROC Curve of the Different Scores Compared for AMNRAS

Table 4

Comparison of Area under the curve for various scores

Test Result Variable (s)	Area Under the Curve
	Area	95% Confidence Interval		Sth. Errora	P
		Lower Bound	Upper Bound
AMNRAS	0.832	0.740	0.923	0.047	0.000
CBAC score	0.642	0.519	0.764	0.062	0.030
IDRS	0.670	0.552	0.788	0.060	0.009
FINDRISC	0.729	0.614	0.844	0.059	0.000
WHO/ISH risk score	0.545	0.417	0.672	0.065	0.495
FRS	0.455	0.326	0.583	0.065	0.486

aUnder the nonparametric assumption

Discussion

A strategic objective in the fight against the NCD epidemic must be to ensure early detection and care using evidence based good quality, cost appropriate, and sustainable health-care interventions. There are four main factors influencing the adoption of a risk prediction method for the screening the Indian population to determine their risk for developing NCD. First, is the applicability of risk score to the local setting[6] for example Finnish diabetes risk score (FINDRISC)[7] include vegetable consumption however this domain is irrelevant to Indian population, which is largely a vegetable consumable country, World Health Organization/International Society of Hypertension risk prediction charts are not available for individual countries.[8] Second lack of calibration of a risk score, primarily due to lack of population based database for example Framingham risk score (FRS)[9] lacks Indian population representation, third various risk scores available for assessment of risk factors are for individual diseases like diabetes, cardiovascular disease, etc., Some examples are the FRS for Cardio vascular diseases, FINDRISC, Danish diabetes score, Indian Diabetes Risk Score (IDRS) and others. And finally, the ability of a risk score to capture all major of risk behaviors for example IDRS lacks domains on Smokeless tobacco, diet, and stress.[10] Government of India under National Program for Prevention of Diabetes, Cardiovascular Diseases and Stroke. (NPDCS), now in National Health Mission, has started Community Based Assessment Checklist (CBAC)[5] for early Detection of NCDs but it also do not include fruit consumption, and stress. It also does not provide weight for strenuous work in lieu of physical activity.

For the prevention of NCD in Indian population it becomes essential to find out risk score that can help to identify high NCD risk individuals neither by underestimating nor by overestimating the risk. A study reported that FRS underestimates the risk of cardiovascular disease morbidity and mortality in Asian Indians.[6] A study cross sectional rural community-based study stated that although WHO/ISH risk charts are easy to use but they may underestimate the CVD risk in the population.[14] The one or other NCD risk factors relevant to the Indian context like obesity, abdominal obesity, family history, tobacco chewing, high salt intake, stress, and NCD treatment status missing in the commonly used risk scores.

When the prevalence is high in the community, the screening level is set at a lower level, which will increase sensitivity.[15] The present study reported AMNRAS to have highest sensitivity among the compared scores. The treatment of NCDs like hypertension, diabetes, COPD is available at primary health care level. Moreover, untreated NCD could end up with dreadful complications to a person. Therefore, a score which maximize true positives is preferable. The specificity and sensitivity for IDRS in present study was 72.2% and 56.8% respectively at cut off of 60 and above. The CURES study in 2005 stated 72.5% sensitivity and 60.1% specificity for IDRS score of 60.[12] Another record-based study documented the sensitivity and specificity of IDRS was 78.9% and 56.1%, respectively in the studied diabetic patients.[16] This shows though the present study is having a low sample size, the results are comparable to the present literature. FINDRISC score showed highest specificity for predicting the 10 years diabetes incidence in Finnish population (Europe). Similar were the findings of a retrospective, record-based study of diabetes detection camp organized by a teaching hospital. They reported FINDRISC sensitivity, 55.2% and specificity, 89.6% respectively.[16] High specificity is important where the cost of test or treatment is high, as may be the situation in European countries. In addition, performance of the FINDRISC, at community level in a country with different cultures, lifestyles and eating habit, is still unknown.[17]

On extensive published literature search CBAC sensitivity and specificity could not be found. CBAC mentions that is cut off point only highlight risk factors but does not rule out the NCD therefore individuals over 30 years should be screened at health facility level for NCD. It is handy and simple tool for health workers. However, it fails to collect important risk like stress, diet, and treatment history of NCD. Best index for measuring the performance of a risk score is total area under ROC curve. An area of less than 0.6 is not significant. The higher the area under the curve the better is overall performance of risk score to accurately pick up diseased and nondiseased individuals. In present study area under the curve for AMNRAS at cut of 8 was 0.832. This difference in area under the curve at cut was statistically significant. CBAC, IDRS and FINDRISC also had area under the curve over 0.6. A study on Screening for diabetes in an urban slum of Pune reported IDRS receiver operator area under the curve at cut of 60 was 0.651.[12]

The results of the present study showed that the AMNRAS could identify maximum number of patients into high risk for NCD events. CBAC, IDRS, FINDRISC have performed intermediately. FRS calculator could identify least number of patients to be in high risk so has performed the worst in our patient considering our cut off for high-risk definition. However, it should be noted that AMNRAS estimates risk for a large combination of NCD outcomes including hypertension, diabetes, cardiovascular event, and chronic respiratory disease. In contrast, the other risk engines/tools estimate risk mainly for diabetes and cardiovascular disease only. Thus, AMNRAS is not directly comparable to other risk engines/tools. Other risk engines/tools may perform the best in Indians for individual disease outcomes.

Conclusions

Simple questionnaire and measurement-based scoring system perform only modestly in discrimination between NCD cases and controls. The scoring system developed by AIIMS Mangalagiri has higher performance parameters than the other five tested in the present study. The accuracy of AMNRAS for NCD risk will have to be determined in large size cohort by following the subjects for years to see whether a NCD will be actually present.

Limitation

The primary limitation of the present study is the sample size. Second, it is retrospective record-based study and variables like stress were self reported. Third, FRS-CVD the present study is not directly comparable to other risk calculators as it estimates risk for all NCD. Controlled blood pressure in hypertension cases can affect our score. Regardless of these limitations, we believe AMNRAS includes all the important risk factors for NCD and its future cohorts with validation for weighted risk factors scores are predicted to provide evidence-based, good quality, cost appropriate, and sustainable NCD risk assessment score.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.