Accuracy of evidence-based criteria for identifying an incident hip fracture in the absence of the date of injury: a retrospective database study.

OBJECTIVES: Hospital discharge data (HDD) in many health systems do not capture the date of injury (DOI); the absence of this date hinders researchers' ability to distinguish repeat from incident injury admissions. Various approaches using somewhat arbitrary criteria have been explored to increase the accuracy of incident injury identification. However, these approaches have not been validated against a data source which contains DOI. The aim of this study was to determine the accuracy of evidence-based criteria for identifying fall-related incident hip fractures in the absence of DOI using HDD containing DOI as the reference standard.
DESIGN: Retrospective database study.
SETTING: New Zealand. PARTICIPANTS: 8761 patients aged 65+ years admitted for fall-related hip fracture between 1 July 2005 and 30 June 2008, inclusive. OUTCOME MEASURES: We defined person-identifying HDD containing DOI as the reference standard and calculated measures of the accuracy of evidence-based criteria for identifying fall-related incident hip fractures from HDD not containing DOI. The criteria were principal diagnosis of hip fracture, mechanism of injury indicating a fall, admission type emergency, admission source other than a transfer and presence of hip operation code(s). For a subsequent fall-related hip fracture, additional criteria were time between successive hip fractures ≥120 days, and all external cause-of-injury codes being different to those for the previous hip fracture.
RESULTS: The sensitivity and specificity of the criteria for identifying fall-related incident hip fractures from data not containing DOI were 96.7% and 99.3%, respectively, compared with the reference standard. The application of these criteria resulted in a slight underestimation of the percentage of patients with multiple hip fractures.
CONCLUSIONS: Although it is preferable to have DOI; this study demonstrates that evidence-based criteria can be used to reliably identify fall-related incident hip fractures from the person-identifying HDD when DOI is unavailable.

To determine the accuracy of evidence-based criteria for identifying fall-related incident hip fractures from person-identifying hospital discharge data in the absence of the date of injury.

The criteria for identifying fall-related incident hip fractures from person-identifying hospital discharge data in the absence of date of injury were highly accurate when compared with the reference standard which contained the date of injury.

The high accuracy of the criteria raises the question: is it really necessary to capture the date of injury in hip fracture hospital discharge data? The answer depends on a number of considerations; the majority of which relate to the quality and the reliability of hospital discharge data. It would still be ideal, however, to collect the date of injury for hip fracture, and other injury causes, in hospital discharge data as this would facilitate clinical and health services research.

The New Zealand hospital discharge data contain a unique national person identifier which enables the identification of the same patient across multiple admissions. In many health systems, however, hospital discharge data contain no such identifier. Before the criteria can be applied in this situation, data linkage may be used to obtain person-identifying hospital discharge data.

Previous studies investigating the accuracy of arbitrary, commonly used criteria for identifying incident hip fractures from hospital discharge data did not employ a reference standard containing date of injury. Our study is therefore the first to examine the accuracy of evidence-based criteria in comparison with the reference standard for identifying incident hip fractures using date of injury.

Our study was performed using hospital discharge data from only one health system. Future research could further evaluate the criteria using hospital discharge data from other health systems, which capture the date of injury.

Background

Hip fractures are a major injury burden in older people, with the majority of cases being fall-related and involving community dwellers.1 2 High initial hospitalisation cost has been reported in many countries, including the UK, Belgium, the USA and Australia.3–6 Costs of rehabilitation, home nursing, home help and institutional care may be substantial depending on subsequent functional impairment.6 7 Patients with a hip fracture have also been found to have a higher risk of dying in the subsequent 20 years compared with the general population.8 9

Accurate estimates of hip fracture incidence are required in the planning, delivery and prioritisation of service provision, and in the planning, implementation and evaluation of appropriate preventive interventions for at-risk groups.10 Hospital discharge data (HDD) are an appropriate data source for estimating hip fracture incidence because virtually all hip fractures necessitate hospital admission.11 The accurate estimation of hip fracture incidence in many health systems, however, is hampered by the lack of a unique system-wide patient identifier (UPI) and/or the absence of date of injury.12 In health systems where a UPI is lacking, HDD refers to specific episodes of care rather than to cases or persons. Data linkage may be used to identify the same patient across different admissions and create a person-identifying dataset. However, in the absence of date of injury, researchers’ ability to distinguish multiple admissions for the same hip fracture from new admissions associated with a different hip fracture in the same person is limited.12

A common approach for distinguishing an admission due to an incident hip fracture from a readmission attributable to an old hip fracture is the application of a clearance period, commonly assumed to be 28 or 30 days.12–14 However, the clearance period is arbitrary and ignores individual fall and injury risk profiles. While the risk of readmission is high during the first 30 days following discharge, only 12–34% of all relevant readmissions occur during this period.15–17 Thus, more than two-thirds of readmissions would be counted as incident cases resulting in a significant overestimation of true incidence estimates.

In order to reliably differentiate readmissions due to an ‘old’ hip fracture from a new admission due to a subsequent hip fracture, a person-identifying data source, which captures the date of injury, is required. As far as we are aware, very few countries in the world capture this information in their HDD (as opposed to dedicated trauma or specialty registers which may capture this information).18–20 In countries with no capture of this information in their HDD, various approaches for eliminating duplicate records have been explored to increase the accuracy of incident hip-fracture identification.10 12 To the best of our knowledge, these approaches have not been validated against a data source which contains the date of injury. We modified the current approach of using a clearance period of 28 or 30 days by including additional, evidence-based criteria for incident hip fractures21 and estimated the accuracy of our criteria using data containing the date of injury as the reference standard.

Methods

We analysed New Zealand (NZ) person-identifying HDD which contain records of all publicly funded inpatient treatment of injuries in NZ. Almost 100% of all hospital injury treatment was publicly funded in 2006–2007 (Chris Lewis, NZ Ministry of Health, personal communication 2010). The data were obtained from the NZ Health Information Directorate, and maintained and enhanced by the Injury Prevention Research Unit, University of Otago, Dunedin, NZ. Discharges that occurred between 1 July 2005 and 30 June 2008 (inclusive) were used in the analysis—records were selected if the patient's age at admission was 65+ years, the cause of injury was a fall (W00–W19 in the International Classification of Diseases, 10th Revision, Australian Modification (ICD-10-AM)) and the principal diagnosis was a hip fracture (S72.0–S72.2 in ICD–10–AM).22

The selected records included the date of injury and a readmission indicator, and we defined this dataset as the reference standard. This indicator is determined by a computer algorithm which uses information from four variables in HDD, namely the unique national person identifier, date of admission, date of discharge and date of injury. The indicator has recently been validated and found to have excellent accuracy between 87% and 99%.23 We used this indicator to exclude readmissions and estimate the number of fall-related incident hip fractures and the number of patients with multiple fall-related hip fractures between 1 July 2005 and 30 June 2008, inclusive.

We omitted the date of injury in the reference standard dataset to form a comparison dataset on which criteria for incident hip fractures based on our review of relevant hip fracture literature21 were applied (the base case; see table 1). Multiple criteria were developed based on well-documented issues of identifying repeat admissions for the same injury from HDD without the date of injury or a readmission indicator.24 25 These criteria defined an incident hip fracture as an acute hospital admission and not an interhospital transfer. Cases which satisfied this definition but which only had hip revision procedure code(s) listed in their records were excluded.26 However, cases with one or more hip operation procedure codes in addition to one or more hip revision procedure codes were retained because they are likely to represent incident hip fractures subsequent to previous hip fractures.27 The hip operation and revision procedure codes used in this study are available on request.

Table 1

Fall-related hip fracture incident case identification by data source

	Reference standard dataset	Comparison dataset
Inclusion criteria
Date of discharge from 1 July 2005 to 30 June 2008	✓	✓
Principal diagnosis hip fracture (S72.0–S72.2 in ICD-10-AM)	✓	✓
First-listed external cause indicating a fall (W00–W19 in ICD-10-AM)	✓	✓
Age at admission 65+ years	✓	✓
Exclusion criteria
All hip fractures
Readmission indicator positive	✓
Transfer from another facility		✓
Hip revision procedures only		✓
Admission type non-acute		✓
Additional criteria for subsequent hip fractures
Readmission within 120 days of discharge		✓
All first-listed external cause codes being identical to those in the previous hip fracture		✓

ICD-10-AM, International Classification of Diseases, 10th Revision, Australian Modification.

We developed additional selection criteria in order to distinguish between the first and subsequent fall-related hip fracture in the same patients due to the inaccuracy of fracture type classification28 and the lack of ICD-10-AM codes on laterality of fractures (Patricia Saad, National Centre for Classification in Health (Australia), personal communication 2010). We assumed that the minimum clearance period between the discharge date of the index episode of care for the first incident hip fracture and the admission date of the index episode of care for the second incident hip fracture would be 120 days,29 30 and that all first-listed external cause of injury codes (mechanism of injury, place of occurrence and activity being undertaken when injured) would differ between ‘old’ and ‘new’ hip fractures belonging to the same patient. Given that external cause of injury is detailed in ICD-10-AM, it would appear that the probability of two hip fractures in the same patient having three identical first-listed cause codes would be very small. Nevertheless, we tested this assumption in a sensitivity analysis and also investigated the completeness or otherwise of external cause information in the reference standard dataset.

We applied the above criteria sequentially according to the order listed in table 1. We calculated the sensitivity, specificity, positive and negative predictive values of the criteria compared with the reference standard using standard definitions.31 We conducted a one-way sensitivity analysis of the base case by modifying the criteria one at a time. Eight scenarios were considered:

S1 included interhospital transfers

S2 included cases with only hip revision procedures

S3 included non-acute admissions

S4 modified the clearance period to 30 days

S5 modified the clearance period to 60 days

S6 modified the clearance period to 90 days

S7 omitted the criterion regarding external cause codes

S8 omitted the criterion regarding the clearance period

We conducted all analyses in Stata V.11.32 We evaluated equality of proportions using two-sample χ2 tests of proportions or Fisher’s exact test, as appropriate. For skewed continuous variables, we compared medians using non-parametric K-sample tests on the equality of medians.

Results

Fall-related hip fracture in older people in NZ according to the reference standard dataset

The reference standard dataset contained 9637 episodes associated with 8761 patients. A total of 787 episodes of care were classified as readmissions by the readmission indicator and hence excluded, resulting in an estimate of 8850 fall-related incident hip fractures in 8524 patients between 1 July 2005 and 30 June 2008. Among these patients, 8200 (96.2%) had one fall-related hip fracture, 322 (3.8%) had two fall-related hip fractures and 2 (<0.1%) had three fall-related hip fractures. Fewer than 1% of the fall-related incident hip fractures were admitted as non-acute cases and a similar percentage of cases were coded as interhospital transfers. Overall, 6% of patients had at least one readmission (range 1–3).

The median time from hospital discharge after treatment for one fall-related hip fracture to admission for treatment of another fall-related hip fracture was 203 days (interquartile range 71–441).

A total of 1054 fall-related incident hip fractures (11.9% of total cases) belonging to 1039 patients did not receive any surgical interventions. Fourteen cases (in 14 patients) had both hip revision and hip operation procedures. Of the cases without surgical intervention, 3% had at least one hip revision procedure code suggesting the possibility of coding error.

In patients with multiple fall-related hip fractures, 212 (65%) sustained a subsequent fall-related hip fracture at the same place as the previous fall-related hip fracture. Few subsequent fall-related hip fractures (28 or 8.6% of all refracture cases) had all first-listed external cause codes identical to those in the preceding fall-related hip fracture. The presence of an unspecified element in all first-listed external cause codes was rare (1.8% of all cases).

Accuracy of criteria for identifying fall-related incident hip fractures in the absence of date of injury

Our criteria for identifying fall-related incident hip fractures in the absence of date of injury compared favourably with the reference standard. The sensitivity and specificity of the base case criteria were 96.7% and 99.3%, respectively (table 2). These criteria failed to detect 3.3% of cases (false negatives). Of the false negatives, 244 (84.4%) were first fall-related hip fractures and 45 (15.6%) were second fall-related hip fractures. More than one criterion was responsible for the exclusion of 22.1% of the former and 100% of the latter (table 3).

Table 2

Accuracy of evidence-based criteria for identifying fall-related incident hip fractures without the date of injury

Accuracy*	Base case	S1 accepted interhospital transfers	S2 accepted hip revision(s) alone	S3 accepted non-acute admissions	S4 accepted clearance period >30 days
Sensitivity	96.7 (96.3 to 97.1)	97.3 (97.0 to 97.7)	97.1 (96.7 to 97.4)	97.3 (97.0 to 97.6)	97.7 (97.4 to 98.0)
Specificity	99.3 (99.2 to 99.4)	99.2 (99.1 to 99.3)	99.3 (99.2 to 99.4)	99.3 (99.1 to 99.4)	99.4 (99.2 to 99.5)
PPV	98.3 (98.0 to 98.6)	97.9 (97.6 to 98.2)	98.3 (98.0 to 98.6)	98.1 (97.8 to 98.3)	98.3 (98.0 to 98.6)
NPV	98.7 (98.6 to 98.9)	99.0 (98.8 to 99.1)	98.9 (98.7 to 99.0)	99.0 (98.8 to 99.1)	99.1 (99.0 to 99.2)
Accuracy*	Base case	S5 clearance period 60 days	S6 clearance period 90 days	S7 no criterion on external cause codes	S8 no criterion on clearance period
Sensitivity	96.7 (96.3 to 97.1)	97.3 (97.0 to 97.7	97.0 (96.6 to 97.4)	96.9 (96.2 to 97.2)	98.0 (97.7 to 98.3)
Specificity	99.3 (99.2 to 99.4)	99.3 (99.2 to 99.4)	99.3 (99.2 to 99.4)	99.4 (99.2 to 99.5)	98.7 (98.6 to 98.9)
PPV	98.3 (98.0 to 98.6)	98.3 (98.0 to 98.6)	98.3 (98.0 to 98.6)	98.3 (98.0 to 98.6)	96.8 (96.4 to 97.2)
NPV	98.7 (98.6 to 98.9)	99.0 (98.8 to 99.1)	98.8 (98.7 to 99.0)	98.8 (98.6 to 98.9)	99.2 (99.1 to 99.3)

*Values are percentages (95% CIs).

NPV, negative predictive value; PPV, positive predictive value.

Table 3

Reason(s) for the exclusion of false negatives

	Reason, n (%)
Excluded first hip fractures (n=244)
Transfer from another hospital	90 (36.9)
Non-acute admission	73 (29.9)
No hip operation	27 (11.1)
Transfer from another hospital and non-acute admission	54 (22.1)
Excluded second hip fractures (n=54)
Transfer from another hospital and non-acute admission, and time to refracture <120 days and first-listed external cause codes identical to a previously admitted hip fracture	54 (100)

The one-way sensitivity analysis (table 2) suggests that all accuracy measures estimated for the base case are robust. Accepting interhospital transfers (S1), including cases with hip revision(s) alone (S2) and accepting non-acute admissions (S3) reduced false negative cases from 3.3% to 2.7%, 2.9% and 2.7%, respectively. However, the modification in S1, S2 and S3 had little effect on the accuracy measures of each of these scenarios and only minor accuracy differences separated them.

The inclusion of cases irrespective of whether or not they had the same first-listed external cause codes as the preceding fall-related hip fracture (S7) resulted in accuracy estimates being identical or near identical to those in the base case, suggesting that this criterion was not a key driver of accuracy. The modification of the clearance period in S4–S6 revealed that a clearance period greater than 30 days (S4), in conjunction with all other criteria, was optimal based on a combination of measures: the percentage of false negatives (2.3%), the percentage of patients with multiple fall-related hip fractures (3.2%) and accuracy estimates (between 98% and 99%).

Discussion

The criteria for identifying fall-related incident hip fractures from person-identifying HDD in the absence of date of injury were highly accurate when compared with the reference standard. Our findings clearly demonstrate that when the date of injury is not available in person-identifying HDD, which is the situation in many countries, evidence-based criteria can be used to reliably identify fall-related incident hip fractures. This is the first published study to investigate the accuracy of evidence-based criteria in comparison to a gold standard of identifying fall-related incident hip fractures using the date of injury. Two previous studies compared commonly used criteria for identifying incident hip fractures from HDD but neither included this gold standard.10 12

The high accuracy of the criteria for identifying fall-related incident hip fractures raises the question: is it really necessary to capture the date of injury in hip fracture HDD? The answer depends on a number of considerations. First, NZ patients with hip fracture may or may not be representative of all patients with this condition in terms of the clinical and admission characteristics incorporated into these criteria. These characteristics, in turn, may be related to patients’ age, and hip fracture admission policies and treatment practices in a specific health system. Second, the performance of our criteria depends on the reliability of coded HDD. The misclassification error in surgical procedure codes in NZ HDD would appear to be small, since the surgical intervention rate is within the range of rates reported in the literature15 17 33 and only 3% of incident hip fractures were coded with one or more hip revision procedure and no hip operation procedure. In contrast, Dalal and Roy34 reported significant coding errors in hip fracture surgery between 24% and 32.8% in one UK hospital. The completeness and specificity of first-listed external cause of injury codes in HDD may also require consideration. The level of missing, first-listed external causes of injury in the NZ hip fracture HDD used was negligible (≤0.1%), whereas the level of missing external causes of injury in other HDD is reportedly high.35 The percentage of incident cases in the NZ hip fracture HDD with all first-listed external cause codes denoting unspecified elements was small (<2.0%). If this percentage is significantly elevated, the external cause criterion may be rendered less effective in distinguishing multiple hip fractures in the same patients.

Our analysis of the reasons behind the exclusion of false negative cases by the base case suggests that some of these cases may be atypical presentations of fall-related incident hip fractures, such as elective admissions rather than being admitted as an emergency case or hip operation procedure(s) not being performed due to patients’ frailty or a terminal illness. Additionally, some patients might have been transferred from another hospital on the same day without having the first episode of care at the originating hospital being recorded. It should be noted that the study population was comprised of both community-dwelling older people and nursing home residents. The latter group is frail and/or disabled, and is commonly affected by a range of comorbidity36; hence, they may have received palliative care and/or other type of non-acute care rather than surgical intervention(s) for their hip fractures. However, we were unable to disaggregate our results by community-dwelling status to confirm this.

The one-way sensitivity analysis suggests that the base case may be improved by reducing the clearance period from 120 to 30 days, a period commonly used to exclude readmissions but which until now had not been validated by population-level dataset. This finding appears to contradict previous research which suggests that only 12–34% of all relevant readmissions occur during this period.15–17 However, it is important to note that we did not solely use the clearance period to identify incident hip fractures; it was only one of a set of criteria used to differentiate readmissions from incident hip fractures. Finally, the sensitivity analysis found that the inclusion of interhospital transfers (S1) did not alter the accuracy of the base case, suggesting that transfers may be counted as incident cases. This finding should be interpreted with caution because the S1 scenario contains other criteria in addition to that for transfers. Furthermore, the transfers identified by the reference standard as incident cases have been verified by the date of injury. It is well established that the exclusion of transfers would minimise the overestimation of the incidence of hospitalised injury.12 15

Our study was performed using only one national injury hospital discharge dataset. The NZ data contain a unique national person identifier which enables the identification of the same patient across multiple admissions. In many health systems, however, HDD contain no such identifier.12 37 Before our criteria can be applied in this situation, data linkage may be used to obtain person-identifying HDD. Future research could further evaluate the criteria using other person-identifying HDD datasets that contain the date of injury. Hospital admission policies and treatment practices for hip fractures might differ between countries and this may translate into different admission patterns compared with those in NZ. Moreover, almost 100% of all hospital injury treatment is publicly funded in NZ, whereas in many other countries the public as well as private hospital sectors are involved in the provision of hospital treatment for injuries. Accordingly, the reconciliation of episodes of care belonging to the same patients may be easier in the former compared with the latter. Ultimately, admission patterns reflect underlying injury incidence rates and injury severity in older people, and variations in hospital utilisation for those injuries.

Our study demonstrates a valid approach for identifying fall-related incident hip fractures in older people from person-identifying HDD in the absence of date of injury. It would still be ideal, however, to collect the date of injury in HDD as this would enable clinical and health services research questions to be addressed for all injury causes.

Conclusions

Few countries capture the date of injury in their HDD. It has previously been suggested that the lack of this date has hampered the accurate identification of incident hip fractures from HDD. Our study has shown that in the absence of date of injury, evidence-based criteria may be used to accurately identify fall-related incident hip fractures in older people from person-identifying HDD. However, the collection of date of injury for inclusion in hip fracture HDD might still be necessary unless the quality and reliability of coded HDD can be established. Furthermore, the date of injury would facilitate clinical and health services research for all injury causes.