Comparison of 2 fracture risk estimation processes in Alberta: a cross-sectional chart review.

BACKGROUND: In Canada, decisions regarding osteoporosis pharmacotherapy are based on estimated 10-year risk of osteoporotic fracture. We aimed to determine how frequently 2 common approaches (Canadian Association of Radiologists and Osteoporosis Canada [CAROC] tool and Fracture Risk Assessment Tool [FRAX]) produced different estimates and to seek possible explanations for differences.
METHODS: We conducted a cross-sectional chart review at a tertiary osteoporosis centre (Dr. David Hanley Osteoporosis Centre in Calgary). Included patients were women referred for consideration of osteoporosis pharmacotherapy who attended a consultation between 2016 and 2019 and whose charts contained 10-year osteoporotic fracture risk estimates using both the CAROC tool (based on bone mineral density [BMD] results) and FRAX (based on BMD results and clinically assessed fracture risk factors). Risk estimates provided on BMD reports (calculated with CAROC) and generated through osteoporosis clinic consultation (calculated with FRAX, including BMD) were categorized as low (< 10.0%), moderate (10.0%-19.9%) or high (≥ 20.0%). Estimates were considered discordant when they placed the patient in different risk categories.
RESULTS: Of 190 patients evaluated, 99 (52.1%) had discordant risk estimates. Although a similar proportion were considered high risk by BMD reports using the CAROC tool (17.9%) and clinic charts using FRAX (19.5%), the 2 methods identified different patients as being high risk. Around the crucial high-risk (20.0%) treatment threshold, discordance was present in 37 patients (19.5%, 95% confidence interval [CI] 14.5%-25.7%); discordance around the moderate-risk (10.0%) threshold was present in 69 (36.3%, 95% CI 29.5%-43.2%) patients. Disagreement regarding fracture history between BMD reports and clinic charts was observed in 19.8% of patients.
INTERPRETATION: Fracture risk estimates on BMD reports (using the CAROC tool) and those calculated in the clinical setting (using FRAX) frequently result in different risk classification. Osteoporosis treatment decisions may differ in up to half of patients depending on which estimate is used, highlighting the need for a consistent and accurate assessment process for fracture risk.

Up to 50% of women and 20% of men will sustain an osteoporosis-related fragility fracture after age 50 years.1 Available treatments can reduce the risk of fracture, but identification of individuals who are at the highest risk of sustaining a fracture and therefore most likely to benefit from therapy has proven challenging.2,3 The 2010 Osteoporosis Canada guideline recommends that decision-making regarding pharmacologic therapy be based on estimated 10-year risk of osteoporotic fracture rather than bone mineral density (BMD) results alone.4 The guideline recognizes a 10-year osteoporotic fracture risk of less than 10% as “low risk,” 10%–20% as “moderate risk” and 20% or more as “high risk,” with pharmacologic therapy to be recommended for individuals in the high-risk category and considered for those at moderate risk.4 The national guideline endorses the use of either of the following tools to estimate 10-year fracture risk: the Fracture Risk Assessment Tool (FRAX)5 or the Canadian Association of Radiologists and Osteoporosis Canada (CAROC)6 tool.

In accordance with the national guideline,4 most Canadian BMD reports provide a 10-year fracture risk estimate. These estimates are generated using the CAROC tool, in accordance with the 2013 Canadian Association of Radiologists technical standards for BMD reporting.7 The CAROC tool lends itself well to use in the radiology office, as it incorporates few clinical risk factors (e.g., prior fracture and glucocorticoid use), which can be ascertained through self-administered patient survey.8

Fracture risk estimates provide an entry point into a shared decision-making process with the patient regarding initiation of pharmacologic therapy. Although risk estimates provided on BMD reports can be used for this purpose, physicians may also choose to calculate fracture risk using the more comprehensive FRAX tool. This process usually involves ascertainment of key clinical risk factors via direct patient interviews and review of medical records. This information is then incorporated into the FRAX algorithm, along with the patient’s BMD result, to generate a risk estimate.8,9

Differences in the process of fracture risk assessment may introduce variations to risk group classification between BMD reports and clinic assessments, and such variation could have a major impact on the treatment decision-making process. We sought to determine how frequently the risk estimates provided on BMD reports (calculated using CAROC) place patients in a different fracture risk category than estimates calculated in an osteoporosis clinic (using FRAX with BMD), and to seek possible explanations for these differences.

Methods

Study design

This was a cross-sectional chart review involving women referred to our multidisciplinary osteoporosis centre for postmenopausal osteoporosis who attended an initial consultation between January 2016 and June 2019.

Setting

This study was conducted at the Dr. David Hanley Osteoporosis Centre (DHOC), a multidisciplinary clinic in Calgary, serving a catchment area of more than 2 million people. In our health region, BMD acquisition is conducted at community-based radiology offices. All BMDs are reported by radiologists and include a fracture risk estimate generated using the CAROC tool. At the DHOC, bone health consultations involve adjudication of fracture risk factors by an osteoporosis specialist and fracture risk estimation using FRAX (with BMD, when available). Both the CAROC and FRAX tools incorporate clinical risk factors in addition to BMD results (with BMD input being optional for FRAX but required for CAROC) and provide a 10-year estimate of osteoporotic fracture risk.5,6,10,11 However, although both tools account for patient age, history of fragility fracture and glucocorticoid use, FRAX incorporates several additional key clinical risk factors (Table 1).

Table 1:

Features of FRAX and CAROC fracture risk calculators

Feature	FRAX*	CAROC†
Clinical risk factors	Age (40–90 yr)	Age (> 50 yr)
	Sex	Sex
	Weight
	Height
	Previous fracture in adulthood	Fractures since age 40
	Parent fractured hip
	Current smoking
	Glucocorticoid use‡	Glucocorticoid use§
	Rheumatoid arthritis
	Secondary osteoporosis
	Consumes ≥ 3 units alcohol per day
Bone mineral density	Femoral neck BMD optional	Femoral neck BMD required¶
Other considerations	Incorporates competing risk of mortality	No competing risk of mortality
	Different algorithms can be used for different	No different algorithms for different ethnicities
	ethnicities and countries
	Involves interactions between variables	No interactions between individual variables
Type of fractures predicted	Hip fracture within 10 years
	Major osteoporotic fracture within 10 years	Osteoporotic fracture within 10 years

Note: BMD = bone mineral density, CAROC = Canadian Association of Radiologists and Osteoporosis Canada, FRAX = Fracture Risk Assessment Tool.

FRAX is available at www.sheffield.ac.uk/FRAX.

CAROC is available at https://osteoporosis.ca/health-care-professionals/tools/caroc.

FRAX: Exposed to oral glucocorticoids for more than 3 months at a dose of prednisolone of 5 mg daily or more or current use.

CAROC: History of glucocorticoid use for 3 months or more in the past 1 year at 7.5 mg prednisone equivalent.

If fracture risk is low or undefined based on femoral neck BMD, and T-score is ≤ −2.5 at either lumbar spine or total hip, fracture risk is increased to moderate.

Participants

Included patients were women after menopause referred for consideration of osteoporosis pharmacotherapy who had a community-based BMD measurement within the 24 months before consultation in which the report included femoral neck T-score and a fracture risk statement (i.e., using the CAROC tool). Included patients also had a FRAX-generated fracture risk estimate (which incorporated the femoral neck BMD result) documented in their osteoporosis clinic chart, derived at the time of in-person consultation. Patients with complex metabolic bone disease, treatment failure or adverse events, and questions about stopping therapy or “drug holidays,” and patients without BMD reports or fracture risk estimates were excluded.

Data sources

Clinic charts, containing consultation notes and BMD reports, were reviewed by 1 investigator (A.L.F.). Data were abstracted according to a protocol developed by A.L.F. and E.O.B., and entered into an electronic data collection instrument (using REDCap) that had been developed for the purpose of this study. The following information was extracted from each patient’s osteoporosis clinic consultation note: fracture risk factors relevant to FRAX and 10-year fracture risk estimates (calculated using FRAX Canada with BMD). Each patient’s most recent BMD report was reviewed for the following reported data: BMD T-scores (lumbar spine, femoral neck, total hip), fracture risk factors relevant to the CAROC tool (prior fracture or glucocorticoid use) and 10-year fracture risk estimate (calculated using CAROC). Abstracted data were reviewed by E.O.B. for potential inaccuracies (e.g., outlying data points), and any potential inaccuracies were cross-checked with source data from the chart.

Statistical analysis

Each patient was classified as being “low risk” (< 10.0%), “moderate risk” (10.0%–19.9%) or “high risk” (≥ 20.0%)4 based on the risk estimate provided on their BMD report (CAROC tool) and the estimate from their osteoporosis clinic consultation (FRAX). We considered risk classifications to be concordant when both estimates placed the patient in the same risk category and discordant when estimates placed the patient in different risk categories.

Clinical risk factors recorded on BMD reports were compared with the risk factors listed in clinic consultation letters. For fracture risk factors that are common to both CAROC and FRAX (i.e., prior fracture and glucocorticoid use), the proportion of patients with discrepancies between recorded risk factor status in BMD reports and clinic charts was calculated. We also determined the proportion of patients with FRAX-specific risk factors (Table 1) listed in clinic charts.

The prespecified primary outcome was the proportion of patients with discordant risk classifications. Secondary outcomes were proportion with discordant risk classifications around the high-risk treatment threshold; proportion with discordant risk classifications around the moderate-risk threshold; proportion placed in a low-risk category by one estimate and high-risk category by the other estimate (i.e., severe discordance); and likelihood of having a discordant estimate if there was a discrepancy in risk factor adjudication between the BMD report and osteoporosis clinic chart for clinical risk factors common to both CAROC and FRAX, or if a FRAX-specific clinical risk factor was present.

We estimated proportions of patients with discordant fracture risk estimates using the Wilson calculation to determine 95% confidence intervals (CIs).12 Agreement between the 2 modes of fracture risk assessment was evaluated using the κ statistic for 2 by 2 tables and the weighted κ statistic for 3 by 3 tables. To determine whether one mode of risk assessment tended to result in higher risk estimates, we used the McNemar test for 2 by 2 tables and the Bowker test of symmetry for 3 by 3 tables. The proportion of patients with FRAX-specific risk factors and the proportion with discrepancies in adjudication of common CAROC and FRAX risk factors (prior fracture and glucocorticoid use) between BMD reports and clinic charts was determined. We used a 2-tailed χ2 test to assess comparisons. Quantitative analyses were done with SAS version 9.4 (SAS Institute).

Ethics approval

This study was approved by the Conjoint Health Research Ethics Board at the University of Calgary.

Results

A total of 190 charts of patients assessed at the DHOC between January 2016 and June 2019 met eligibility criteria for this review. Demographic characteristics and fracture risk factors for included patients are displayed in Table 2. Mean age was 63.2 (standard deviation 6.4) years (range 51–85).

Table 2:

Baseline characteristics of patients referred to a multidisciplinary clinic for postmenopausal osteoporosis

Characteristic	Mean ± SD or no. (%)n = 190*
Age, yr	63.2 ± 6.4
Sex, female	190 (100)
Height, cm, n = 189	161.7 ± 6.3
Weight, kg, n = 189	63.6 ± 10.6
Body mass index, n = 189	24.3 ± 3.9
Lumbar spine T-score	−2.34 ± 0.91
Femoral neck T-score	−2.12 ± 0.75
Total hip T-score	−1.78 ± 0.76
CAROC risk factors†
Previous fracture, n = 187	39 (20.9)
Glucocorticoid use, n = 186	1 (0.5)
FRAX risk factors†
Previous fracture	59 (31.0)
Glucocorticoid use, n = 189	3 (1.6)
Parental hip fracture,‡ n = 189	38 (20.1)
Current smoker‡	7 (3.7)
Rheumatoid arthritis‡	6 (3.2)
Consumes ≥ 3 units alcohol/d‡	3 (1.6)
Ten-year FRAX MOF risk, %	13.6 ± 7.7
Ten-year FRAX hip fracture risk, %	3.2 ± 3.6
Fracture risk on BMD report (using CAROC)
High (≥ 20.0%)	34 (17.9)
Moderate (10.0%–19.9%)	130 (68.4)
Low (< 10.0%)	26 (13.7)
Fracture risk on clinic chart (using FRAX)
High (≥ 20.0%)	37 (19.5)
Moderate (10.0%–19.9%)	72 (37.9)
Low (< 10.0%)	81 (42.6)

Note: BMD = bone mineral density, CAROC = Canadian Association of Radiologists and Osteoporosis Canada, FRAX = Fracture Risk Assessment Tool, MOF = major osteoporotic fracture, SD = standard deviation.

n = 190 unless stated otherwise.

CAROC risk factors were obtained from BMD reports; FRAX risk factors were obtained from osteoporosis clinic charts.

FRAX-specific risk factors (incorporated by FRAX algorithm but not CAROC).

Risk categorization between BMD reports and clinical assessments

A total of 99 (52.1%, 95% CI 45.0%–59.1%) had discordant risk group classifications (weighted κ 0.26, 95% CI 0.16–0.36), as shown in Table 3. Risk estimates provided on BMD reports placed significantly more of these patients in a higher risk category than clinic estimates (p < 0.001 for the Bowker test of symmetry). As shown in Table 3, this was primarily the result of differences in classification within the low and moderate risk categories. Although a similar overall percentage of the study cohort were classified as high risk by clinic estimates (19.5%) and BMD estimates (17.9%), these high-risk patients were not the same individuals in each of the 2 risk estimation methods. That is, 37 patients (19.5%, 95% CI 14.5%–25.7%) had discordance around the clinically relevant high-risk (20.0%) threshold (κ 0.36, 95% CI 0.19–0.52).

Table 3:

Comparison of 10-year fracture risk estimates reported in osteoporosis clinic charts (calculated using FRAX) and estimates presented on bone mineral density reports (calculated using CAROC) for 190 patients after menopause

		BMD report, no. (%)*†			Total discordant
		Low risk‡n = 26	Moderate risk‡n = 130	High risk‡n = 34
Osteoporosis clinic chart, no. (%)*†	Low risk,‡ n = 81	19 (10.0)	57 (30.0)	5 (2.6)	62 (32.6)
	Moderate risk,‡ n = 72	5 (2.6)	55 (28.9)	12 (6.3)	17 (8.9)
	High risk,‡ n = 37	2 (1.1)	18 (9.5)	17 (8.9)	20 (10.5)
Total discordant		7 (3.7)	75 (39.4)	17 (8.9)	99 (52.1)

Note: BMD = bone mineral density, CAROC = Canadian Association of Radiologists and Osteoporosis Canada, FRAX = Fracture Risk Assessment Tool.

Percentages represent the proportion of the total cohort (n = 190).

Shaded cells represent patients whose BMD reports and osteoporosis clinic assessments provided discordant fracture risk classifications (i.e., the 2 estimates placed the patient in different risk categories).

Low risk = 10-year osteoporotic fracture risk < 10.0%; moderate risk = 10.0%–19.9%; high risk = ≥ 20.0%.

Considering a lower (10.0%) treatment threshold, treatment would be considered for 164 (86.3%) based on BMD estimates and 109 (57.4%) patients based on clinic estimates (p < 0.001 for difference using the McNemar test), resulting in discordance in 69 (36.3%, 95% CI 29.5%–43.2%) patients (κ 0.19, 95% CI 0.07–0.30). There were 7 (3.7%, 95% CI 1.8%–7.4%) patients with markedly discordant risk classifications (i.e., low risk by one estimate and high risk by the other).

Risk factors incorporated in both BMD reports and clinical assessments

Glucocorticoid use was reported by 3 (1.6%) of 185 patients with available data, and discrepancies between the BMD report and osteoporosis clinic chart were present in 2 patients.

A history of fragility fracture was reported by 67 (35.8%) of 187 patients with available data. As shown in Table 4, discrepancies between the BMD report and the osteoporosis clinic chart were present in 37 (19.8%) patients with available data (κ 0.49, 95% CI 0.35–0.63). Of patients with discrepancies in fracture history between BMD reports and clinic charts, 54.1% had discordant risk estimates, compared with 52.0% of patients whose fracture history was consistent between the 2 sources (p = 0.8 for difference in weighted κ coefficients).

Table 4:

Comparison of prior fragility fractures identified in clinic charts (calculated using FRAX) and on bone mineral density reports (calculated using CAROC) among patients after menopause who attended a clinical assessment at an osteoporosis clinic

		BMD report, no. (%)*†
		Fracture reported	No fracture reported
Osteoporosis clinic chart, no. (%)*†	Fracture reported	30 (16.0)	28 (15.0)
	No fracture reported	9 (4.8)	120 (64.2)

Note: BMD = bone mineral density, CAROC = Canadian Association of Radiologists and Osteoporosis Canada, FRAX = Fracture Risk Assessment Tool.

Data were available for 187 patients.

Shaded cells represent patients whose BMD reports and osteoporosis clinic charts provided disagreeing interpretations of fracture history.

Of the 37 patients with inconsistencies in fracture history between BMD reports and clinic charts, 28 had a history of fracture reported in their clinic charts and 9 had prior fracture indicated on their BMD reports (p = 0.002 for difference using the McNemar test). We considered how the inclusion of these fractures in the CAROC-generated risk estimate provided on the BMD report would affect risk category discordance for each of these patients. For 13 of 28 (46.4%), inclusion of the fracture would result in a change from concordant to discordant, and for 12 (42.9%), inclusion of the fracture would result in a change from discordant to concordant. For the remaining 3 patients, including the fracture in the CAROC-generated risk estimate would not affect the agreement in risk categorization.

Risk factors incorporated in clinical assessments but not BMD reports

The proportion of patients with clinical risk factors that are included in FRAX but not the CAROC tool (i.e., parental hip fracture, smoking, rheumatoid arthritis and alcohol use) are shown in Table 2. In total, 45 individuals had FRAX-specific risk factors, and discordant risk classifications were present in 42.2%, compared with 55.2% of patients without FRAX-specific risk factors (p = 0.1 for difference in weighted κ coefficients), suggesting that discordance in risk classification was not explained by FRAX-specific risk factors.

Interpretation

We observed that more than half of patients evaluated at an osteoporosis clinic were placed in different fracture risk categories by the fracture risk estimates provided on their BMD reports and the risk assessments calculated in the clinic. Discordance around the crucial high-risk (20%) treatment threshold was present in one-fifth of patients, and discordance around the moderate risk (10%) threshold was present in one-third of patients. Bone mineral density reports tended to place patients in higher risk categories than clinic assessments. That is, patients were 1.5 times as likely to be classified as moderate or high risk on BMD reports than on clinic charts and were 3 times as likely to be classified as low risk in clinic charts than on BMD reports. Furthermore, BMD reports and clinic assessments provided differing interpretations of fracture history in one-fifth of patients. Despite providing higher risk estimates in general, BMD reports did not account for almost half of the fragility fractures identified at the time of clinic assessment.

One potential reason for the differences in risk classification between BMD reports and clinic assessments is that the CAROC tool was used to estimate risk for BMD reports and FRAX was used to estimate risk in the clinic. Although the 2010 Osteoporosis Canada guideline4 endorses the use of either the CAROC tool or FRAX for fracture risk estimation, the FRAX software was not ubiquitous in the reporting systems of BMD scanners in Canada at the time of publication. It was therefore suggested that the CAROC tool be used to calculate risk to present on BMD reports (grade D evidence), with the caveat that this situation may change as access to the FRAX software improves; a stronger mandate for the use of the CAROC tool was made in the 2013 Canadian Association of Radiologists technical standards for BMD reporting.7

Prior literature has shown that FRAX and the CAROC tool frequently produce different risk estimates for the same patient, even when fracture risk factors are ascertained and adjudicated in the same manner for both tools. In a prospective evaluation of more than 34 000 individuals from the Manitoba bone densitometry registry, when a radiologist generated FRAX and CAROC estimates and compared them with one another, risk category discordance was present in 15%, rising to 31% when considering individuals with prior fracture or glucocorticoid use.9 In 135 patients from Ontario presenting with a fragility fracture, clinical risk factors were obtained from survey data and risk estimates calculated with both the CAROC tool and FRAX, using the same set of clinical risk factors for each calculation. In this study, risk category discordance was present in 33%.13 Similarly, in 60 patients who presented with a wrist fracture and had fracture risk estimated using both the CAROC tool and FRAX, discordance in risk classification was observed in more than 30%.14

In the present study, we observed a higher rate of risk category discordance than the 15%–33% expected from the use of different fracture risk estimation tools (i.e., CAROC and FRAX) alone. Our study design not only allowed us to compare differences between the CAROC and FRAX tools, as has been done previously,9,13,14 but also permitted assessment of differences in the risk estimation results according to the process of clinical risk factor ascertainment between BMD assessment and personal osteoporosis clinic evaluation.

We hypothesized that discrepancies in ascertainment of risk factors common to both the CAROC tool and FRAX (i.e., fracture history and glucocorticoid use) may explain some of the risk category discordance observed between BMD reports and osteoporosis clinic estimates. Although glucocorticoid use was infrequent in our cohort and resulted in only 2 cases of discrepancy, almost 1 in 5 patients had a different fracture status listed on their BMD report than in their clinic chart. Importantly, BMD reports did not identify nearly half of fragility fractures that were documented in clinic charts. Although data regarding timing of prior fractures were not available for all patients in this study, the discrepancy may relate, in part, to the fact that BMD risk assessments (using the CAROC tool) incorporate fractures sustained after age 40, and clinic assessments (using FRAX) incorporate all fractures sustained in adulthood. Additionally, some discrepancies may have resulted from fractures sustained after the BMD report was done but before the clinical assessment.

When considering the entire study cohort, these isolated discrepancies in fracture history did not change the proportions of patients stratified into discordant risk categories. However, for most individual patients who had a history of fracture documented in their osteoporosis clinic chart but not their BMD report, the inclusion of a prior fracture in the BMD (CAROC) risk estimation algorithm would result in changes in risk categorization agreement (i.e., from concordant to discordant or vice versa). Therefore, discrepancies in fracture history between BMD reports and clinic assessments are clinically relevant to individual patients and may influence treatment recommendations. Specifically, many osteoporosis guidelines — including that of Osteoporosis Canada — recommend pharmacologic therapy for individuals with a prior hip or vertebral fracture,4,15 or a recent (i.e., within the past year) fragility fracture,12 regardless of whether the 10-year estimated fracture risk exceeds 20%, underscoring the importance of accurate ascertainment of fracture status to clinical decision-making.

Our findings are directly relevant to clinical care. According to the current guideline,4 in the clinical setting, physicians may either rely solely on the fracture risk estimates provided on BMD reports or choose to use BMD results and additional clinically assessed risk factors to generate a risk estimate using FRAX. The results of this study indicate that the former strategy is likely to result in a greater proportion of individuals being recommended for pharmacologic treatment, but may fail to identify many patients with a history of fragility fracture who would benefit from therapy. The latter strategy, as used at our osteoporosis specialty clinic, results in the availability of 2 fracture risk estimates. In the roughly half of cases where the 2 estimates place patients in different risk categories, this conflicting information may be a cause of confusion for patients, impairing effective shared decision-making.

This study highlights a need to develop a consistent and accurate process for fracture risk assessment that can be adopted by both radiologists and clinicians. Collaborative discussion between radiologists, primary care physicians and patients will therefore be required to reach a consensus on a process for BMD reporting and fracture risk assessment that is practical without compromising accuracy.

Limitations

Some limitations should be considered when interpreting our results. Our inclusion criteria were limited to patients after menopause with uncomplicated osteoporosis, and it is therefore unknown as to whether risk discordance may be the same or greater with more complex clinical cases. This study was conducted at a single tertiary osteoporosis centre, and although our centre uses standard clinical evaluation and tools for risk assessment that are accessible and similar to any specialist or clinic across Canada, it is not known whether our results can be generalized to primary care or other geographic locations.

Our sample was smaller than those of prior population-based registries and chart reviews were conducted by a single investigator, but our patient-level data review strengthens the validity of the individual risk profile characterizations. Although this study compares risk estimates generated using FRAX and the CAROC tool, it was not intended or powered to evaluate the discriminative accuracy of either tool. Additionally, practices for BMD reporting may vary across the country, so our findings may not be applicable in all centres.

Conclusion

Among patients after menopause, fracture risk estimates provided on BMD reports using the CAROC tool are discordant with estimates generated at clinical assessments using FRAX (with BMD) in about half of cases. Osteoporosis treatment paradigms rely on accurate detection of patients at high risk of fracture, and a consistent process for generation and reporting of fracture risk estimates is required. Our data support the need for a collaborative discussion between radiologists and treating clinicians about the BMD reporting and fracture risk estimation process.