Development and internal validation of the Edmonton Obesity Staging System-2 Risk screening Tool (EOSS-2 Risk Tool) for weight-related health complications: a case-control study in a representative sample of Australian adults with overweight and obesity.

OBJECTIVE: Excess weight and related health complications remain under diagnosed and poorly treated in general practice. We aimed to develop and validate a brief screening tool for determining the presence of unknown clinically significant weight-related health complications for potential application in general practice.
DESIGN: We considered 14 self-reported candidate predictors of clinically significant weight-related health complications according to the Edmonton Obesity Staging System (EOSS score of ≥2) and developed models using multivariate logistic regression across training and test data sets. The final model was chosen based on the area under the receiver operating characteristic curve and the Hosmer-Lemeshow statistic; and validated using sensitivity, specificity and positive predictive value. SETTING AND PARTICIPANTS: We analysed cross-sectional data from the Australian Health Survey 2011-2013 sample aged between 18 and 65 years (n=7518) with at least overweight and obesity.
RESULTS: An EOSS≥2 classification was present in 78% of the sample. Of 14 candidate risk factors, 6 (family history of diabetes, hypertension, high sugar in blood/urine, high cholesterol and self-reported bodily pain and disability) were automatically included based on definitional or obvious correlational criteria. Three variables were retained in the final multivariate model (age, self-assessed health and history of depression/anxiety). The EOSS-2 Risk Tool (index test) classified 89% of those at 'extremely high risk' (≥25 points), 67% of those at 'very high risk' (7-24 points) and 42% of those at 'high risk' (<7 points) of meeting diagnostic criteria for EOSS≥2 (reference).
CONCLUSION: The EOSS-2 Risk Tool is a simple, safe and accurate screening tool for diagnostic criteria for clinically significant weight-related complications for potential application in general practice. Research to determine the feasibility and applicability of the EOSS-2 Risk Tool for improving weight management approaches in general practice is warranted. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The Edmonton Obesity Staging System (EOSS)-2 Risk Tool was developed and validated in an Australian sample of community-based ‘high risk’ individuals for potential application in general practice.

The EOSS-2 Risk Tool efficiently detects the presence of unknown clinically significant weight-related complications according to the widely used Edmonton Obesity Staging System.

Research to determine the feasibility and applicability of the EOSS-2 Risk Tool for improving weight management approaches in specific general practice settings is warranted.

Introduction

Overweight and obesity are a major public health issue challenging global health systems.1 It affects a quarter of all young people (aged 2–17 years) and two-thirds of all adults in Australia.2 We recently estimated that millions of Australians have weight-related health complications (complex and/or chronic conditions) associated with increased avoidable health service use and hospitalisations.3 The most common weight-related health problems include cancer, stroke, heart disease, kidney disease, dementia, diabetes mellitus, back pain and osteoarthritis.4 While evidence‐based guidelines provide recommendations on how to provide effective weight management,5 6 excess weight and related complications remain under diagnosed and poorly treated.7 8

Although evidence suggests that the vast majority of patients with overweight and obesity want their general practitioner (GP) to bring up weight management during appointments, they seldom do.7 International experts agree that obesity stigma is a major barrier to seeking and receiving appropriate treatments for weight management.9 Past experiences of obesity stigma and weight-based discrimination may explain why so few people seek and receive appropriate treatment for obesity. Interestingly, the most important criterion GPs consider for initiating weight management conversations with a patient is if they have, or are at risk of developing, new or additional weight-related health problems.7 This suggests that targeting weight-related health status rather than obesity per se may overcome this barrier to initiating treatments in primary care.

The Edmonton Obesity Staging System (EOSS) is based on weight-related health complications among individuals with overweight and obesity.10 A score of ≥2 on the EOSS indicates the presence of clinically significant weight-related complications requiring medical intervention. A brief diagnostic screening tool for predicting EOSS≥2 in patients with excess weight could provide GPs with a structured framework for further investigations to confirm a timely diagnosis in those who screen positive. It may also help GPs initiate a discussion about the health benefits of weight loss with patients, with or without mentioning obesity, resulting in improvements in their quality of care and health outcomes.11 Thus, we aimed to report the development and internal validation a simple screening tool (‘EOSS-2 Risk Tool’) to estimate the risk of clinically significant weight-related complications according to a diagnostic definition of EOSS≥2.3

Methods

We present this paper according to the Journal’s formatting requirements and Strengthening the Reporting of Observational Studies in Epidemiology guidelines for reporting observational (cross-sectional) studies.12

Study design, setting and participants

We analysed cross-sectional data from the Australian Health Survey (AHS) 2011–2013 and partially corrected for at state, section of state, sex and age group levels in the weighting process. It is the largest survey with biochemical and physical measurements ever conducted in Australia. For this study, we selected a subsample of participants aged 18–65 years who had measured anthropometry (n=7518) with at least overweight (defined as a body mass index of 25 kg/m2 or higher) and/or central obesity (defined as a waist measurement of 102 cm and 88 cm or higher for all men and women, respectively).

Consent to participate

Written informed consent was obtained from participants separately for the in-home and pathology collection centre components.13

Patient and public involvement

No patient involved.

Variables

All survey questions are listed in the AHS user guide.13

Diagnostic outcome

To create the diagnostic definition of EOSS stages, we used information from an extensive range of weight-related health complications including chronic disease biomarkers (eg, diabetes, high cholesterol, high triglycerides, chronic kidney disease and abnormal liver enzymes), measured blood pressure, as well as self-reported long-term conditions, disability, psychological distress, health and bodily pain. Specific criteria and thresholds for these variables were used to classify each participant into one of five EOSS categories based on our previous definition (online supplemental material 1).3 Each category reflects the most severe EOSS stage according to weight-related complications for that individual. Given the absence of internationally consistent criteria for assigning weight-related health impairments into EOSS categories,14 we chose this reference standard which has been validated in an Australian sample of community-based ‘high risk’ individuals.3 Our analyses focused on differentiating the presence and absence of clinically significant weight-related complications (EOSS 0–1 against EOSS 2–4).

Candidate predictor variables

To develop a simple tool that could easily be applied in general practice, like the Australian Type 2 Diabetes Risk Assessment Tool (AUSDRISK),15 we considered self-reported predictor variables only. These included demographic variables (age, gender and country of birth); medical history (history of depression or anxiety, family history of diabetes, hypertension, ‘high sugar in blood or urine’ and high cholesterol); lifestyle behaviours (smoking status, exercise, fruit and vegetable consumption); and functional health (self-rated health, bodily pain and disability).

Bias

The AHS achieved a high response rate of 85% defined by fully/adequate responding households, after sample loss.13 A focus of this study is the predictive accuracy of our screening test, which could be misstated if a large group with particularly poor (or good) predictive accuracy were excluded from the data set. We believe that this would be extremely unlikely. Missing data were not considered since we used specific selection criteria for our training and test data sets.

Sample size considerations

The data set of 7518 eligible individuals from the AHS was split into a training data set and 5 test data sets. We randomly assigned about 40% (n=2885) of data records to the training data set and about 12% (770) for each of the 5 test data sets. In the training data set, we observed about 645 to be ‘lower risk’ (EOSS<2) and 2240 to be ‘higher risk’ (EOSS≥2) of clinically significant weight-related complications. When considering up to 14 predictor variables, there was still a generous 46 lower risk and 160 higher risk individuals per predictor variable in the training data set. Each test data set was expected to contain about 172 lower risk and 598 higher risk individuals.

Statistical analysis

The diagnostic outcome of interest was presence of clinically significant weight-related complications defined using EOSS stages ≥2. A total of 14 candidate predictor variables were considered (table 1). We tested for statistical evidence of each variable distinguishing between the two EOSS groups (EOSS<2 vs EOSS≥2) using Pearson’s χ2 test. We considered that the performance of logistic regression would be affected by the probable size of the coefficients and the correlations between predictor variables as well as the number of events per variable.16 Thus, before model fitting, frequency counts were used to check for small categories in the categorical variables for exclusion.

Table 1

Candidate predictor variables from the Australian Health Survey data 2011–2013 by Edmonton Obesity Staging System (EOSS) groups (<2 and ≥2)

Variables	EOSS<2 (n=1678)	EOSS≥2 (n=5840)	P value
Age			<0.001
18–24 years	178 (10.6%)	343 (5.9%)
25–34 years	502 (29.9%)	940 (16.1%)
35–44 years	482 (28.7%)	1337 (22.9%)
45–54 years	352 (21.0%)	1603 (27.4%)
55–64 years	164 (9.8%)	1617 (27.7%)
Gender			0.011
Males	889 (53.0%)	2889 (49.5%)
Females	789 (47.0%)	2951 (50.5%)
Country of birth			0.202
Australia	1226 (73.1%)	4372 (74.9%)
Main English-speaking countries	193 (11.5%)	665 (11.4%)
Other	259 (15.4%)	803 (13.7%)
Smoking status			<0.001
Non smoker	887 (52.9%)	2558 (43.8%)
Ex-smoker	453 (27.0%)	2008 (34.4%)
Current smoker	338 (20.1%)	1274 (21.8%)
Whether exercise met the recommended guidelines			<0.001
Yes	940 (56.0%)	2920 (50.0%)
No	735 (43.8%)	2913 (49.9%)
Unknown	3 (0.2%)	7 (0.1%)
Whether vegetable and fruit consumption met recommended guidelines			0.766
Yes	84 (5.0%)	303 (5.2%)
No	1594 (95.0%)	5537 (94.8%)
Family history of diabetes			<0.001
No	1276 (76.0%)	3828 (65.5%)
Yes	383 (22.9%)	1884 (32.3%)
Unknown	19 (1.1%)	128 (2.2%)
Family history of high sugar in blood or urine
No	1678 (100.0%)	5455 (93.4%)
Yes	0 (0.0%*)	385 (6.6%)
History of depression or anxiety			<0.001
No	1665 (99.2%)	4574 (78.3%)
Yes	13 (0.8%)	1266 (21.7%)
Family history of hypertension
No	1678 (100.0%)	4490 (76.9%)
Yes	0 (0.0%*)	1350 (23.1%)
Family history of high cholesterol levels
No	1678 (100.0%)	4734 (81.1%)
Yes	0 (0.0%*)	1106 (18.9%)
Self-assessed health
Excellent	492 (29.3%)	757 (13.0%)
Very good	740 (44.1%)	2028 (34.7%)
Good	394 (23.5%)	2008 (34.4%)
Fair	52 (3.1%)	772 (13.2%)
Poor	0 (0.0%*)	275 (4.7%)
Disability status			<0.001
Has no limitation or specific restriction or disability or long-term condition	1432 (85.4%)	3859 (66.0%)
Has mild core/school/employment activity limitation	32 (1.9%)	652 (11.2%)
Has moderate core activity limitation	0 (0.0%)	478 (8.2%)
Has severe core activity limitation	4 (0.2%)	273 (4.7%)
Has profound core activity limitation	210 (12.5%)	578 (9.9%)
Bodily pain in the last 4 weeks
None	779 (46.4%)	1366 (23.4%)
Very mild/mild	897 (53.5%)	2385 (40.8%)
Moderate	1676 (0.0%*)	1462 (25.1%)
Severe	0 (0.0%*)	487 (8.3%)
Very severe	0 (0.0%*)	130 (2.2%)
Unknown	2 (0.1%)	10 (0.2%)

*Structural zeros were either due to the definitional variables (variables or levels of variables that were used to define EOSS≥2) or those variables that had significant correlation to EOSS≥2.

Model development

We used logistic regression analysis to assess diagnostic models from the training data and apply the results in the test data sets. In addition, structural zeros resulting from definitional variables that were used in the diagnostic definition of EOSS≥2 and variables with obvious correlation were pragmatically included in the screening tool, bypassing the logistic diagnostic modelling. The self-reported conditions include one or more levels of variables including poor level of self-assessed health, moderate to severe levels of bodily pain and moderate to profound levels of disability.

For the other candidate variables, we used univariate logistic regression analysis to investigate their ability to discriminate the presence or absence of EOSS≥2 based on diagnostic criteria (reference) versus the EOSS index test (EOSS-2 Risk screening Tool) independently. Statistics used to assess predictive ability included statistical significance, goodness of fit and area under the receiver operating characteristic (AROC) curve. We retained only those variables that were judged to be clinically relevant and displaying at least some indication of predictive ability for the candidate diagnostic modelling.

Next, we built different candidate diagnostic models for predicting EOSS≥2 using a combination of clinical judgement and statistical performance. Each model comprised of five to six variables that were retained for multivariate logistic modelling. For each candidate diagnostic model, we used a multivariate logistic regression analysis where variables that were without obvious clinical importance and had a non-significant effect on the model were sequentially eliminated in a backward stepwise manner. The final step of the candidate diagnostic models consisted of only those variables that were statistically significant (p <0.05).

Model selection and scoring system

We compared different models for predicting EOSS≥2. Of these, we chose the model which was consistently observed to have the best discrimination using AROC and the Hosmer-Lemeshow (HL) χ2 statistic (HL χ2 statistic <20 represents good calibration with a p≥0.01) across the five test data sets.17 Once the list of predictors was finalised, we then fitted the model on the whole combined data set of 7518 participants. To simplify the scoring system for the EOSS-2 Risk Tool, integer scores were obtained by dividing the regression (β) coefficient for each variable in the final model by the lowest β coefficient, then multiplying by two and rounding to the nearest integer.18 We capped the maximum score at 10 which we believed was sufficient to convey the substantially increased risk of weight-related complications for those groups (five times larger than the smallest OR). A ROC curve was fitted to the simplified scoring system and we used the co-ordinates of the curve to determine the cut-off score with the maximum sensitivity, specificity and positive predictive value (PPV).

Model validation

The test data sets were used to evaluate the performance and transportability of the model to individuals who were not involved in the development. It is generally recommended to have multiple test data sets, so as to test the model on varying case mix, with each containing 200 or more events.19 The final model was validated on each of the five test data sets. We assessed performance of the final model on each of the five test data sets using sensitivity, specificity and PPV statistics (table 2). All analyses were conducted using SPSS V.26 (IBM Corp).

Table 2

Performance of the final model* in the training and test datasets

Type of dataset†	AROC (95% CI)	HL χ2 statistic	HL χ2 P value	Sensitivity (%)	Specificity (%)	PPV (%)
Training dataset (n=1408)	0.71 (0.68 to 0.73)	3.87	0.868	64	63	64
Test dataset 1 (n=478)	0.67 (0.62 to 0.71)	6.00	0.647	60	59	60
Test dataset 2 (n=467)	0.70 (0.66 to 0.75)	6.09	0.637	63	67	63
Test dataset 3 (n=474)	0.70 (0.65 to 0.74)	3.56	0.895	62	63	62
Test dataset 4 (n=481)	0.69 (0.64 to 0.73)	3.26	0.917	62	64	62
Test dataset 5 (n=507)	0.66 (0.60 to 0.70)	5.31	0.724	60	58	60

*Final model—age, self-assessed health and history of depression or anxiety.

†The reduced number in each dataset is due to removal of observations that had structural zeros and variables with significant correlation to EOSS≥2 from candidate diagnostic modelling.

AROC, area under the receiver operating characteristic curve; HL, Hosmer–Lemeshow; PPV, positive predictive value.

Results

Of the 7518 participants, 1678 (22%) were classified into the EOSS<2 group and 5840 (78%) were classified into the EOSS≥2 group. Participants in the EOSS≥2 tended to be older, current or ex-smokers, with poorer self-rated health and did not meet the recommended exercise guidelines compared with the EOSS<2 group (table 1). The definitional variables along with variables (family history of diabetes, hypertension, high sugar in blood urine, and high cholesterol) identified as having obvious correlation to EOSS≥2 were automatically included in our final screening tool without having to be considered in the modelling.

In the univariate logistic regression analysis, we identified 9 out of 11 variables to be independent predictors of EOSS≥2 which were subsequently retained for the candidate diagnostic modelling. Of the 9 variables in the multivariate logistic regression analysis, age, self-assessed health and history of depression or anxiety were found to be consistently significant at the final step of the training and test data sets. The AROC for the final model in the training data set was 0.71 (95% CI 0.68 to 0.73). Using a cut-off score of ≥7, the sensitivity, specificity and PPV for identifying EOSS≥2 in the training data set were 64%, 63% and 64%, respectively (table 2). The β coefficients for the final model and the scores allocated to each risk factor category were then computed (table 3). To create the final paper-based version of the EOSS-2 Risk Tool, we included the three definitional variables and variables with significant correlation to EOSS≥2 (assigned a maximum score of 25 each) along with the three risk factors in the final diagnostic model.11

Table 3

Beta coefficients from the multiple logistic regression final model predicting Edmonton Obesity Staging System (EOSS)≥2 and points allocated to each component of the EOSS-2 score

Predictors*	§ coefficient	P value	Points allocated†
Age groups
18–24 years	Reference		0
25–34 years	0.889	0.599	2
35–44 years	1.268	0.291	3
45–54 years	2.034	0.002	5
55–64 years	2.637	<0.001	6
Self-assessed health
Excellent	Reference		0
Very good	1.492	0.006	3
Good	2.144	<0.001	5
Fair	3.731	<0.001	8
History of depression or anxiety
No	Reference		0
Yes	21.727	<0.001	10‡

*In addition to the three predictors in the final model, definitional and correlational variables (family history of diabetes, hypertension, high sugar in blood/urine, high cholesterol, self-reported bodily pain and disability) automatically predicting EOSS≥2 were also added to the EOSS-2 Risk Tool and were assigned a maximum score of 25.

†Scores for EOSS-2 Risk screening Tool was obtained by dividing the regression.

‡For practical reasons, positive history of depression or anxiety was scored 10.

§Coefficient for each variable in the final model by the lowest β coefficient, then multiplying by 2 and rounding to the nearest integer.

On validation of the final model on the five test data sets, we found that the EOSS-2 Risk Tool had similar discriminative ability in predicting EOSS≥2 with an AROC ranging between 0.66 and 0.70 and was well calibrated as compared with the training data set (table 2). In assessing the effectiveness of the EOSS-2 risk tool on the combined data set of 7518 participants, we found that 89% (n=4483) of those classified as ‘extremely high risk’ did indeed have EOSS ≥2. Thus, extremely high risk is an appropriate descriptor for a PPV of 89%, ‘very high risk’ is an appropriate descriptor for a PPV of 67% (n=839) and high risk is appropriate descriptor for a PPV of 42% (n=518) (table 4). Based on these PPVs, we used specific thresholds for the EOSS-2 Risk Tool scores to define high risk (<7 points), very high risk (7–24 points) and extremely high risk (≥25 points) of having a diagnosis of clinically significant weight-related complications according to EOSS≥2.

Table 4

Cross tabulation of Edmonton Obesity Staging System (EOSS) based on diagnostic criteria (reference) versus the EOSS index test (EOSS-2 Risk screening Tool) in the combined dataset (n=7518)

EOSS (reference)	EOSS index test			Total
	High risk(<7)	Very high risk(7–24)	Extremely high risk (≥25)
EOSS≥2	518 (41.5%)	839 (67.0%)	4483 (89.4%)	5840 (77.7%)
EOSS<2	731 (58.5%)	414 (33.0%)	533 (10.6%)	1678 (22.3%)
Total	1249 (100.0%)	1253 (100.0%)	5016 (100.0%)	7518 (100.0%)

Discussion

We have developed a simple, safe and accurate screening tool (EOSS-2 Risk Tool) to predict the presence of unknown clinically significant weight-related complications according to a diagnostic definition of EOSS stages 2–4,[3] based on nine self-reported risk factors11 relevant to the Australian population. A score of 25 or more was assigned to six out of nine risk factors to automatically predict having an extremely high risk of meeting diagnostic criteria for EOSS≥2 with 100% accuracy, as expected (table 4). For EOSS scores less than 25 assigned to the remaining three risk factors, the results of our validation work selected a threshold of 7 points to discriminate between high risk (<7 points) and very high risk (7–24 points) groups for predicting diagnostic criteria for EOSS≥2, with excellent performance characteristics. We recommend that GPs use the EOSS-2 Risk Tool as a screening tool in all patients with suspected overweight and obesity, regardless of their lowest risk score (high risk), to warrant further investigations and confirm the presence and severity of weight-related complications and diagnostic criteria for EOSS staging. This is because all three risk categories reflect increasing degrees of risk for weight-related complications according to our diagnostic criteria for EOSS stages 2–4.

The EOSS-2 Risk Tool may provide GPs with a new screening tool for conducting further investigations in their patients who screen positive to confirm a timely diagnosis of clinically significant weight-related complications indicating medical intervention. Guidelines released by the Royal Australian College of General Practitioners (‘Red Book’) recommend similar screening tools, such as the AUSDRISK15 for assessing risk of diabetes, and the cardiovascular disease (CVD) risk calculator for assessing absolute CVD risk,6 both in high risk patients, typically aged 40 and 45 years or more, respectively. The AUSDRISK risk factors include gender, age, ethnicity/country of birth, family history of diabetes/high blood sugar, medication for high blood pressure, lifestyle behaviours smoking, fruit/vegetables, exercise), and waist circumference. The CVD risk tool, developed by the National Vascular Disease Prevention Alliance, is based on the Framingham Risk Equation.20 The risk factors include gender, age, systolic blood pressure, smoking status, total cholesterol level, high-density lipoprotein-cholesterol level, diabetes status and left ventricular hypertrophy (by electrocardiography).

The EOSS-2 Risk Tool is unique because it considers functional health status and quality of life in screening for risk of meeting diagnostic criteria for EOSS≥2. It may present GPs with an opportunity to set new clinical targets in their patients based on diagnostic criteria for EOSS stages (eg, from EOSS 2 to 1) with appropriate intervention. This would ensure that the focus of weight management is holistic, and complications-based. The AROC of 0.71 for the EOSS-2 Risk Tool is slightly smaller than those reported for the AUSDRISK (AROC of 0.78)15 and Framingham Risk Equation (C-Statistic of 0.74 for men and 0.80 for women).21

The EOSS-2 Risk Tool may also help GPs initiate a discussion about the health benefits of weight loss with their patients opportunistically during appointments. Results of the ACTION International Observation study found that there was very strong agreement among both patients and healthcare professionals about the health benefits of modest weight loss of 5%–10% in patient with excess weight.7 Furthermore, patients reported that their most important weight loss goal was to reduce health risks associated with excess weight. Similarly, GPs reported that a specific personal medical event (eg, CVD) or diagnosis (eg, diabetes, liver disease, sleep apnoea) was the most important motivation to lose weight in patients. This suggests that screening for clinically significant weight-related complications may help GPs activate weight management discussions with, and treatments for, their patients. We recently published the first evidence of a nationwide pilot study supporting the clinical usefulness of the EOSS-2 Risk Tool (including a paper-based version) for activating weight management discussions in general practice, although further research is required to assess its scalability in Australia’s healthcare system.11 Despite their variable application in Australian general practice, both the AUSDRISK and CVD risk tools are considered clinically useful for patient engagement and education, as well as assessment and management of risk followed by appropriate diagnostic tests.22–24 The EOSS-2 Risk Tool could also be used by other healthcare professionals involved in multidisciplinary clinical obesity services in both public hospitals and private settings such as nurses, dietitians, clinical psychologists, exercise physiologists and physiotherapists.25 26

We acknowledge study limitations and potential risks of bias associated with the data source and methods used, additional to those previously reported.3 As there is no universal definition of EOSS criteria, the performance of the tool based on other diagnostic definitions of EOSS≥2 needs to be established. We recently published a rapid review of relevant studies and highlighted the need for developing standardised tools for clinical settings based on a consistent set of criteria with standardised cut-offs for classifying people into EOSS categories.14 As with the AUSDRISK15 and CVD risk20 tools, the EOSS-2 Risk Tool was developed and validated in a population-based sample and may not be relevant to patients in primary care settings. Furthermore and despite financial incentives,22 27 the implementation of these types of screening tools into routine general practice remains challenging.22 28

Conclusions

The new EOSS-2 Risk Tool is a simple, safe and accurate screening tool for detecting the presence of unknown clinically significant weight-related complications, based on our diagnostic definition of EOSS≥2, in a subsample of the Australian population with overweight and obesity. Research to determine the scalability of the EOSS-2 Risk Tool for improving weight management approaches in general practice is warranted.