What is the validity of self-reported fractures?

We assessed the validity of self-reported fractures, over a median follow-up period of 6.2 years, in a well characterized population-based cohort of 3560 postmenopausal women, aged 60-85 years, from the Fracture Risk Brussels Epidemiological Enquiry (FRISBEE) study. Incident low-traumatic (falls from a standing height or less) or non-traumatic fractures, including peripheral fractures, were registered during each annual follow-up telephone interview. A self-reported fracture was considered as a true positive if it was validated by written reliable medical reports (radiographs, CT scans or surgical report). False positives fractures were considered to be those for which the radiology report indicated that there was no fracture at the reported site. Among self-reported fractures, false positive rates were 14.4% for all fractures. The rate of false positives of 11.2% (n = 48/429) was not negligible for the four classical major osteoporotic fractures (MOFs: hip, clinical spine, forearm or shoulder fractures). In terms of fracture site, we found the lowest false positive rate (4.4%) at the hip, and the highest (16.8%) at the spine, with the proximal humerus and the wrist in between, at about 10% each. The global rates of false positives were 12.5% (n = 22/176) for other major fractures and 22.3% (n = 49/220) for minor fractures. Younger subjects, individuals with fractures at sites other than the hip, with a lower education level, or with a higher BMI were more likely to report false positive fractures. Our data indicate that the inaccuracy of self-reported fractures is clinically relevant for several major fractures, which could influence any fracture risk prediction model.

Introduction

Osteoporosis is a disease characterized by low bone mass and structural deterioration of bone tissue, with a consequent increase in bone fragility and susceptibility to fracture. Fragility fractures result from mechanical forces that would not ordinarily result in fracture, known as ‘low energy’ trauma (equivalent to a fall from a standing height or less). A number of risk assessment tools are available to predict fracture incidence over a period of time, and these may be used to aid decision-making (). Currently, the most used and validated worldwide is the Fracture Risk Assessment Tool (FRAX) developed by the University of Sheffield, which allows to calculate the 10-year probability of major osteoporotic fractures (MOFs: clinical spine, proximal humerus, wrist, hip) or hip fracture alone, based on BMD and validated clinical risk factors (CRFs) independent of BMD (Kanis et al., 2007).

The predictive value of any model is dependent on the accuracy of the epidemiological data used to derive them (). The quality of epidemiological studies with fracture outcomes includes the method for fracture validation, ideally by verification of medical records. In several large population-based epidemiological studies including the Framingham Study, the Iowa Women's Health Study, the Study of Osteoporotic Fractures (SOF), the Women's Health Initiative (WHI), the Osteoporotic Fractures in Men (MrOS), self-report was the main source of information on fracture incidence (Schousboe et al., 2013). However, self-reporting could be subject to either overreporting (reported fractures that did not occur = “false positives”) or underreporting (fractures that were not reported by study participants = “false negatives”), leading to incorrect classification of fracture status. A posteriori verification by medical records is time consuming and expensive, and often not possible in very large or multicenter studies due to difficulties encountered in accessing this information (Ismail et al., 2000).

Therefore, validation studies in well-characterized populations to determine the precision of self-reported fractures are necessary to better assess the quality of current epidemiological research. In the 90s, a few studies have evaluated the validity of self-reported fractures with a percentage of “false positives” varying from 0 to 12% (Nevitt et al., 1992; Colditz et al., 1986; Bush et al., 1989; Beard et al., 1990; Honkanen et al., 1997), depending on the period of recall and the frequency of contacts with participants. Limited data indicate that accuracy is influenced by fracture site, age, education, previous history of osteoporosis and falls (Chen et al., 2004). However, to the best of our knowledge, similar validation studies have not been performed for over twenty years.

The primary purpose of this study was to assess the validity of self-reported fractures in a well characterized, population-based cohort of 3560 postmenopausal women, aged 60–85 years, from the Fracture RISk Brussels Epidemiological Enquiry (FRISBEE) study. Secondarily, we examined the effect of clinical characteristics on the rate of false positive fractures.

Methods

Participants

Between July 2007 and June 2013, 3560 participants aged 70.1 ± 6.4 (mean ± SD) years were included in the FRISBEE study, an ongoing prospective population-based cohort study that aims at validating and integrating several independent CRFs in order to develop a fracture risk model in a well characterized patient population in Brussels, Belgium. Detailed description of the FRISBEE cohort and study methods can be found in previous publications (Cappelle et al., 2017; Iconaru et al., 2019). Briefly, participants were randomly selected from population lists of six districts of Brussels and recruited by postal letter. Baseline characteristics were collected by trained nurses during a face-to-face interview. DXA was performed on the same day. Follow-up data collections are done by annual phone calls and are planned to continue for at least 10 years.

Informed consent was obtained from each woman by return mail. The protocol was accepted by the Ethics Committee of all participating sites (approval number B07720072493).

Fracture definitions and assessments

Incident low-traumatic (falls from a standing height or less) or non-traumatic fractures, classified as MOFs (Kanis et al., 2007) or other fractures, including peripheral fractures, were registered.

We have used the following classification in order to clearly define the fracture site while collecting and verifying data: “hip fracture” means fractures of the proximal femur, including fractures of the femoral neck, intertrochanteric region, and greater trochanter; “proximal humerus” means a fracture of the following regions: greater tuberosity, lesser tuberosity, humeral head, anatomical and surgical neck or humeral head; “clinical spine” means a vertebral fracture with clinical symptoms and >20% deformity on X-Rays; “wrist” means fractures of the lower end of the radius and/or ulna.

Besides the four classical MOFs, we considered as “other major fractures”: fractures of the pelvic bone, sacrum, elbow, upper arm (humerus shaft), lower arm (radius, ulna shaft), upper leg (femoral shaft), lower leg (tibia/fibula shaft), tibial plateau and ankle (distal tibia/fibula or talar dome). These fracture sites were considered as “major” based on several criteria such as long-term hospitalization, severe pain, and restrictions in function or increased mortality (Nilsson et al., 2007; Schmitz et al., 2018). All other fractures (ribs, clavicle, scapula, carpal, patella, tarsal, sternal, skull, face, fingers, toes, metacarpal, metatarsal) were registered in this article as “minor”.

Definitions

The term “self-report” was used to refer to participant reports but also to reports obtained from a relative or medical staff for institutionalized patients. A self-reported fracture was considered as a true positive if it was validated by written and reliable medical reports (radiographs, CT scans or surgical reports).

"False positive’ fractures were considered to be those for which the report indicated that there was no fracture at the reported site. Radiologic reports using words such as “uncertain” or “suspicious” were not taken into consideration as being true positive fractures. Neither were fractures already identified as being a preexisting fracture.

Reports of fractures that were impossible to confirm because no X-ray was taken or the fracture was sustained while overseas, or when no medical record could be found were classified as ‘unavailable radiology reports’ (Fig. 1).

Fig. 1

Distribution of participants with fractures into confirmed and false positive fractures.

Percentages of "confirmed fractures" and "false positive fractures" are calculated by using as denominator self-reported fractures with an available and reliable radiology report.

Factors influencing the rate of “false positives”

We evaluated if the rate of “false positive” reports was significantly influenced by participants clinical characteristics. Information on ethnicity, current or past treatment for osteoporosis (including calcium and vitamin D), age, body mass index (BMI), prior fragility fracture, parental history of hip fracture, use of oral glucocorticoids during a period of three months or longer, rheumatoid arthritis, smoking, alcohol intake, early non-substituted menopause and any cause of secondary osteoporosis were assessed from baseline and follow-up questionnaires (Cappelle et al., 2017). Fall history (typically defined as a fall from standing height or less) was documented using a frequency indicator (no falls, less than one fall per month, less than one fall per week and more than one fall per week) and time to the last fall (during the previous month, after more than one month but within the previous six months, after more than six months) (Cappelle et al., 2017). Activity level was evaluated according to the 6-level IPAQ adapted WHO scale; a sedentary lifestyle, was defined as the lowest level on this scale (De Bruin et al., 1996).

Statistical analysis

The positive predictive value was calculated by dividing the number of validated fractures by the number of all self-reported fractures for which a surgical or adequate radiological report was available. The false positive rate was calculated by dividing the number of ‘false positive’ fractures by the number of all self-reported fractures verified by a radiological report.

As the data are not independent (many patients have >1 fracture), we used the Generalized Estimating Equations (GEE) in order to take into account the correlation structure within patients for the analysis of the predictors for unconfirmed self-reported fractures. GEE was used in uni- and multivariate analyses.

In the GEE analyses, empirical instead of model-based standards errors were used since they are more robust against misspecification of the correlation structure. Exchangeable covariance matrix was used. Odds ratios are presented with their 95% confidence intervals (Hanley et al., 2003).

All analyses were done with the SAS package 9.4. A 5% level of significance was used.

Results

In our cohort of 3560 postmenopausal women, 721 subjects reported a total of 1016 incident fractures over a median follow-up period of 6.2 years (range 0.4–11.1 years). We had access to the radiological records of 829 (81.6%) participants who self-reported a fracture, and 706 (85.2%) of these fractures were confirmed (Fig. 1).

Four fractures (4/829) were old fractures that had been sustained previously (the x-ray at the time the subject reported the fracture showed an old fracture that had occurred >5 years before participant's report.

The global percentages of validated fractures were 77.3% for MOFs (hip, proximal humerus, clinical spine, wrist), 75.4% for “other major” (pelvic bone, sacrum, elbow, upper leg, lower leg, tibial plateau and ankle) and 53.6% for “minor” fractures (all other fractures) (Fig. 2).

Fig. 2

Validity of self-reported fractures (validated by X-Rays or CT scans).

*MOFs – major osteoporotic fractures: hip, proximal humerus, clinical spine, wrist.

**Other major fractures: pelvic bone, sacrum, elbow, upper leg, lower leg, tibial plateau, ankle.

No fracture was found on the radiological report at the indicated site in 119 cases, which means a rate of false positives (FP) of 14.4% (n = 119/829). The global rates of “false positives” were 11.2% (n = 48/429) for all 4 MOFs, 12.5% (n = 22/176) for “other major” (fractures of the pelvic bone, sacrum, elbow, upper leg, lower leg, tibial plateau/knee and ankle), and 22.3% (n = 49/220) for “minor” fractures (all other fractures) (Fig. 2). The percentages of confirmed fractures varied by fracture site, with the hip having the highest proportion of confirmed fractures (90.3%, n = 65), followed by fractures of the pelvic bone (88.5%, n = 46), wrist and proximal humerus (80.7%, n = 129 and 80.8%, n = 84, respectively). Fractures considered as “minor” had a lower percentage of validation by a radiological report (Fig. 3).

Fig. 3

Rates of validated and false positive fractures according to the fracture site.

Positive predictive values were highest for fractures of the hip (95.6%) and elbow (95.2%), followed by pelvic bone (93.9%), wrist (90.2%) and proximal humerus fractures (89.4%) (Table 1)

Table 1

Accuracy of self-reported fractures (validated by X-Rays or CT scans).

Self-reported fracture site		Radiological diagnosis
		Validated fracturesa (n, %)	Unavailablle radiology report (n, %)	Available radiology reports (n, %)
				All fractures	No fracture = “false positives”b	Previous fractureb	Positive predictive value (%)
All fractures (n = 1016)		706 (69.5)	187 (18.4)	829 (81.6)	119 (14.4)	4 (0.5)	85.2
MOFs	Hip (n = 72)	65 (90.3)	4 (5.6)	68 (94.4)	3 (4.4)	0	95.6
	Wrist (n = 159)	129 (80.7)	16 (10.1)	143 (89.9)	14 (9.8)	0	90,2
	Proximal humerus (n = 104)	84 (80.8)	10 (9.6)	94 (90.4)	10 (10.6)	0	89.4
	Clinical spine (n = 159)	104 (65.4)	32 (20.1)	127 (79.9)	21 (16.5)	2 (1.6)	81.9
Other “major fractures”	Ankle (n = 80)	55 (68.8)	12 (15.0)	68 (85)	13 (19.1)	0	80.9
	Pelvis (n = 52)	46 (88.5)	3 (5.8)	49 (94.2)	3 (6.1)	0	93.9
	Elbow (n = 25)	20 (80.0)	4 (16.0)	21 (84)	1 (4.8)	0	95.2
	Knee (n = 22)	16 (72.7)	2 (9.1)	20 (90.9)	4 (20.0)	0	80.0
	Long bones (n = 24)	16 (66.7)	6 (25)	18 (75)	1 (5.6)	1 (5.6)	88.9
Minor fractures	Ribs/clavicle/scapula/carpal/patella/tarsal/sternal (n = 146)	79 (54.1)	41 (28.1)	105 (71.9)	26 (24.8)	0	75.2
	Skull/face/fingers/toes/metacarpal/metatarsal (n = 173)	92 (53.2)	57 (32.9)	116 (67.1)	23 (19.8)	1 (0.9)	79.3

No fracture – the radiology report indicated no fracture at the reported fracture site.

Previous fracture – the radiology report indicated an old fracture at the time of reported fracture.

Long bones - Upper & Lower Leg, Upper & Lower Arm.

Positive predictive value (%) - calculated by dividing the number of validated fractures by the number of all self-reported fractures without taking into account the fractures that could not be validated.

False positive rate per group (%) - calculated by dividing the number of “false positive” fractures by the number of all self-reported fractures without taking into account the fractures that could not be validated.

Percentages are calculated by using as denominator ‘self-reported fractures with an available radiology report’.

Percentages are calculated by using as denominator ‘fractures with a reliable radiology report’.

The number of self-reported fractures shown to be “false positives” also varied by fracture site: 4.4% (n = 3/68) at the hip, 16.5% at the spine (n = 21/127), 10.6% at the proximal humerus (n = 10/94) and 9.8% at the wrist (n = 14/143) (Fig. 3). Reported information was thus the most accurate at the hip with only 3 invalidated self-reported fractures (2 reported hip fractures were in fact a total hip replacement due to severe hip osteoarthritis and 1 was located at the pelvic bone). Regarding the false positive spine fractures, they were mostly due to pain caused by spinal osteoarthritis, degenerative disc disease, skeletal irregularities such as scoliosis.

Additionally, we analyzed several baseline characteristics of study participants that could have influenced the rate of “false positives” fractures (see Methods section). In a multivariate analysis, younger subjects (OR 2.2; 95% CI, 1.4–3.7; P = 0.002), individuals with fractures on other sites than hip (OR 4.2; 95% CI, 1.2–14.1; P = 0.02), with a lower education level (OR 1.9; 95% CI, 1.3–3.9; P = 0.002) or with a higher BMI (OR 0.5; 95% CI, 0.4–0.8; P = 0.003) were more likely to report “false positive” fractures Table 2.

Table 2

Predictors for false positive self-reported fractures.

	False positive fractures (n = 123) %	P valuea	OR (95%CI)b	P valueb
Age		0.002
60–75	17.7		2.2 (1.4–3.7)	0.002
76–84	9.1		1
BMI		0.003
<25	7.3		0.5 (0.4–0.8)	0.003
≥25	16		1
Education		0.002
<High school	21.1		1.9 (1.3–3.9)	0.002
Post high school or higher	12.1		1
Smoking		0.7
Yes	12.9		1
No	15.3		1.2 (0.6–2.2)	0.7
Alcohol intake		0.2
Yes	8.3		1
No	15.5		2.0 (0.8–5.1)	0.2
Menopausal hormone therapy		0.6
Yes	14.4		0.9 (0.6–1.3)	0.6
No	15.7		1
Sedentary lifestyle		0.6
Yes	17.7		1	0.6
No	14.8		0.8 (0.4–1.7)
History of fracture		0.2
Yes	13.2		1
No	16.3		1.3 (0.9–2.0)	0.2
Family history of hip fracture		0.97
Yes	14.7		1
No	15.2		1.0 (0.6–1.8)	1.0
Falls in last 6 mo		0.7
Yes	15.5		1	0.7
No	14.9		0.9 (0.6–1.4)
Insomnia		0.2
Yes	16.5		1	0.2
No	13.2		0.8 (0.5–1.1)
Use of sleeping pills		0.3
Yes	13		1	0.3
No	16.2		1.2 (0.8–1.9)
Calcium supplements intake		0.8
Yes	14.8		1	0.8
No	15.1		1.1 (0.7–1.6)
Vitamin D supplements intake		0.6
Yes	14.1		1	0.6
No	15.7		1.1 (0.8–1.7)
Osteoporosis treatment (antiresorptives)		0.2
Yes	10.4	0.2	1	0.3
No	15.4		1.6 (0.7–4.0)
Fracture site		0.01
Hip	4.41		1	0.02
Others	15.9		4.2 (1.2–14.1)

n = 119 false positive fractures + 4 previous fractures (self-reported fractures with an available radiology report).

Obtained by chi-square.

Obtained by GEE.

Discussion

Systematic validation of all self-reported fractures is quite difficult in large epidemiological studies. In our sample of 1016 self-reported fractures, which is larger than similar scale prospective trials (Ismail et al., 2000), we had only 18.4% of unconfirmed fractures due to a missing radiology report. Other studies had reported a higher rate of unavailable radiology reports (35%–46%) (Chen et al., 2004; Ivers et al., 2002).

For example, Chen et al. evaluated the validity of self-reporting for fractures in the WHI cohort, a widely citated study to support the use of self-reported fractures. From their results, we could extract a false positive rate for hip fractures of 19.3%, which is much higher than what we found (Chen et al., 2004).

Compared with older studies, our percentage of “false positive” fractures (14.4% for all fractures) was slightly higher. The methods of data collection, the analyzed sites of fractures and especially the frequency of follow up were different between published studies, that by now are relatively old (conducted in the 90s). In the Study of Osteoporotic Fractures (SOF), Nevitt et al. (1992) reported a false-positive rate of 11%. These authors compared self-reports of non-spine fractures with radiology reports and medical records, but participants were contacted every 4 months. Another study, conducted by Ismail et al. (2000), assessed recall of fractures over 12–18 months in women and men enrolled in the European Prospective Osteoporosis Study. The false-positive rate for all fractures was 11% for men and women and 9% for women. Ivers et al. (2002) analyzed the accuracy of self-reported fractures in an Australian cohort of 2326 subjects in which 272 subjects reported 318 fractures. In this study, a high proportion (34.6%) of fractures could not be confirmed, mainly due to unavailability of a medical record. Among self-reported fractures, false positive rates were 10.7% for all fractures. Not surprisingly, different rates can be observed in highly selected populations. For example, there were no “false positives” in a sample of 30 young women (30–55 years) from the Nurses Health study (Colditz et al., 1986). All distal forearm fractures reported by mailed questionnaire, with 12-month period of recall, were validated by medical records. Participant's age and profession may have influenced recall accuracy.

Another finding of our study indicates that the validity of some reported fractures, especially the peripheral ones, is much lower with a “false positive” rate of 22.2%. Such an unacceptably high rate of false positives indicates that such fractures should not be considered without proper validation in epidemiological studies.

Our data also showed an age effect, younger subjects (60–75 yrs) reporting more “false positive” fractures than older ones (76–84 years), an unexpected finding already reported in previous studies (Ismail et al., 2000). One of the explanations for these results could be that older people were already interviewed in previous years, making them more sensitive to a fracture event. Also, elderly subjects are more often hospitalized explaining a better recall of fracture. They are also more often institutionalized and the interview/postal questionnaire is then answered by a medical healthcare professional. Unconfirmed fracture reports were also more common among participants with a higher BMI. There are no readily explanations for this association. Less surprisingly, we found that a lower education level and fractures on other site than hip were significantly associated with poorer validity of self-reports.

This article focuses on false-positive self-reports. The ongoing revision of medical records will allow us to report rates of false negatives fractures and, as a result, to define the global accuracy of self-reported fractures in this prospectively studied cohort. In our study, we did not register if the fracture was reported by the patient or by a proxy (a relative or medical staff for institutionalized patients). Therefore, we cannot analyze which information is more reliable. Another limitation of our study is that we included only women, and thus the observations may not be extended to men.

Our study points to large variations in the validity of fracture reports depending on the fracture site. The inaccuracy of self-reported fractures is far from being negligible for several major fractures, especially wrist, humerus, spine, and also for ankle and knee fractures, considered in our study as ‘other major’ fractures. Complementing self-report with an objective method of fracture validation should be the rule in all studies of small to moderate size. Further studies evaluating the accuracy of self-reported fractures (over- and under-reporting) are needed for a better understanding of the impact of this phenomenon in large population-based studies.

In summary, our data indicate that the inaccuracy of self-reported fractures is clinically relevant for several major fractures, which could influence any fracture risk prediction model.

CRediT authorship contribution statement

F. Baleanu: Conceptualization, Methodology, Writing - original draft, Writing - review & editing. M. Moreau: Formal analysis. V. Kinnard: Data curation. L. Iconaru: Conceptualization, Methodology, Writing - review & editing. R. Karmali: Writing - review & editing. M. Paesmans: Formal analysis. P. Bergmann: Conceptualization, Methodology, Writing - review & editing, Supervision. J.J. Body: Conceptualization, Methodology, Writing - review & editing, Supervision.

Declaration of competing interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.