Prospective monitoring and self-report of previous falls among older women at high risk of falls and fractures: a study of comparison and agreement.

BACKGROUND: The identification of the occurrence of falls is an important step for screening and for rehabilitation processes for the elderly. The methods of monitoring these events are susceptible to recording biases, and the choice of the most accurate method remains challenging.
OBJECTIVES: (i) To investigate the agreement between retrospective self-reporting and prospective monitoring of methods of recording falls, and (ii) to compare the retrospective self-reporting of falls and the prospective monitoring of falls and recurrent falls over a 12-month period among older women at high risk of falls and fractures.
METHOD: A total of 118 community-dwelling older women with low bone density were recruited. The incidence of falls was monitored prospectively in 116 older women (2 losses) via monthly phone calls over the course of a year. At the end of this monitoring period, the older women were asked about their recall of falls in the same 12-month period. The agreement between the two methods was analyzed, and the sensitivity and specificity of self-reported previous falls in relation to the prospective monitoring were calculated.
RESULTS: There was moderate agreement between the prospective monitoring and the retrospective self-reporting of falls in classifying fallers (Kappa = 0.595) and recurrent fallers (Kappa = 0.589). The limits of agreement were 0.35 ± 1.66 falls. The self-reporting of prior falls had a 67.2% sensitivity and a 94.2% specificity in classifying fallers among older women and a 50% sensitivity and a 98.9% specificity in classifying recurrent fallers.
CONCLUSION: Self-reporting of falls over a 12-month period underestimated 32.8% of falls and 50% of recurrent falls. The findings recommend caution if one is considering replacing monthly monitoring with annual retrospective questioning.

Introduction

Falls are events with a high prevalence among the elderly population, even among those who are active and healthy, and constitute one of the major preventable geriatric syndromes1. Among the community-dwelling elderly, approximately 30% suffer a fall each year, and half experience recurrent falls2. Elderly women with osteoporosis and having a high risk of fractures exhibit an even higher frequency of falls (51.1%)3. A significant portion of these falls results in injuries (36%)4, fractures (3.4% to 19%)2 , 4 , 5, and the need for medical assistance (8 to 19%)4 , 5 and affects lifestyle choices, creating a high socio-economic burden6. Additionally, experiencing one or more falls in the course of one year significantly increases the chances of the occurrence of new episodes in the following year among the community-dwelling elderly4 , 5 and postmenopausal women1 , 7.

Thus, the surveillance of falls among the elderly represents a priority health issue6, which is why questioning the occurrence of previous falls has been used in clinical/scientific decision making8 - 10. Several methods have been suggested for monitoring the occurrence of falls among the community-dwelling elderly, including questions asking individuals to recall these events at several intervals by means of telephone, face-to-face or mail interviews, information obtained from medical records, and/or prospective records using falls calendars or diaries8 , 9 , 11 - 13. However, the elderly have difficulties in accurately recalling the occurrence of falls in previous periods13 , 14, especially non-injurious falls6 , 14, and, in many cases, they need help completing the information in the calendar12. Thus, a large part of the available data is susceptible to reporting or recording errors, which under- or overestimates the occurrence of falls6 , 8 , 11 , 15 and renders the assessment of these events in the elderly challenging12.

Therefore, the importance of investigating the sensitivity and specificity of retrospective self-reporting of falls16 with respect to different periods8 , 12 has been stressed, aiming to assess the accuracy of this method. In this context, the objectives of the present study were to (i) investigate the agreement between retrospective self-reporting and prospective monitoring of falls and (ii) compare retrospective self-reporting and prospective monitoring of falls and recurrent falls over 12 months among community-dwelling elderly women at high risk of falls and fractures and within subgroups of elderly women with and without post-fall injuries.

Method

Study design and ethical issues

This work is an observational and longitudinal study that has been approved by the Research Ethics Committee of the Universidade Federal de Minas Gerais(UFMG), Belo Horizonte, MG, Brazil (CAAE 0370.0.203.013-11). All participants signed the informed consent form. This survey is part of a main study with the objective to evaluate risk factors of falls among elderly women with low bone mineral density (BMD).

Sample

Community-dwelling elderly women presenting with low bone mass density (BMD) (T-Score < -1.0 DP) in the L1-L4 spinal segments, the femoral neck, or both in a dual-energy X-ray absorptiometry (DEXA) assessment17 were recruited. In this study, individuals aged 60 years or above were considered as elderly, according to the definition proposed by the World Health Organization for developing countries, including Brazil18. Participants were recruited from programs for health care of the elderly in Ceilândia, Federal District, by convenience sampling. Bedridden or wheelchair-bound women, those with severe visual impairment, lower limb amputations or prostheses, sequelae of a cerebrovascular accident, Parkinson's disease, rheumatoid arthritis, peripheral arterial occlusive disease, a history of recent fractures in the lower limbs, vestibulopathy and/or recent reports of vertigo (i.e. within the past month) or musculoskeletal pain at the beginning of the study and those with poor cognitive performance on the Mini-Mental State Examination (MMSE) were excluded from the study. Poor cognitive performance was identified based on the cutoff scores adopted by the multicenter project - Frailty in Elderly Brazilians (projeto multicêntrico de Fragilidade de Idosos Brasileiros). Specifically, due to discrepancies among the different cutoff scores19, the mean values and standard deviations for each educational level reported in Brucki et al.20 were used - with one standard deviation subtracted from the mean21 - 23: a total of 17 points for illiterates, 22 for one to four years of schooling, 24 for five to eight years of schooling, and 26 for nine or more years of schooling. Analyses were performed using prospective and retrospective information on falls during the 12 months of the study (12.36 ± 1.02 months) obtained from 116 of the 118 recruited elderly women because there were two losses due to death (1.7%).

Variables

Descriptive variables

The studied sociodemographic and clinical variables included age, educational level, bone metabolic diagnosis, cognitive performance in the MMSE, physical activity level, and medication under continuous use. Cognitive performance was categorized into three levels according to Hannan et al.12: poor (17-23 points), moderate (24-28 points) or high (29-30 points). The level of physical activity was determined with the adjusted activity score (AAS) of the Human Activity Profile (HAP)24, classifying participants as inactive (AAS<53), moderately active (AAS=53-74) or active (AAS>74).

Falls

A variable fall was defined as a non-intentional event which resulted in the individual changing position to a lower level in relation to his/her initial position2. High-trauma falls, coming to rest against a wall or other structure, and falling as a consequence of sustaining a violent impact, loss of consciousness or sudden onset of paralysis were not included as falls in this study3 , 4.

Prospective monitoring of falls

Prospective data regarding falls among the studied sample were collected on a monthly basis (to reduce the memory bias) over the 12 months of the study by means of phone calls (10.97±1.20 calls per participant). Participants were asked "Have you fallen in the past month? If so, how many times?", and falls were added up at the end of the study. Those individuals who replied positively were asked further about the (i) location (at home or outside of the home), (ii) conditions causing the fall (no apparent circumstances, slipping, stumbling, half-stepping, dizziness, several causes, or possible effect of medication), and (iii) consequences16 (no consequences, bruises, excoriations, lacerations, fractures, pain and/or edema). The prospective data on falls were considered as the reference standard.

Retrospective self-report of falls

Retrospective data on falls over the same 12 months referred to the same period of time as the falls recorded for the respective prospective data. For retrospective data collection, participants received a single phone call one week after to the last follow-up phone call of the study and were asked "Have you fallen in the past 12 months? If so, how many times?"

General procedures

Data on the monthly assessment of falls over the 12-month period of the study (prospective data) and on the number of falls in the 12 months prior to the end of the study (retrospective data) were obtained by means of phone calls made by a single researcher. During each call, participants received instructions with respect to the definition that was adopted for the event fall.

According to prospective data on occurring falls and retrospective data on previous falls, the sample was categorized as (i) non-fallers (0 falls) or fallers (≥1 fall) and then reclassified as (ii) non-recurrent fallers (≤1 fall) or recurrent fallers (≥2 falls) for different analyses.

Statistical analysis

Data are expressed as the means and standard deviations for continuous variables, and as percentages and frequencies for categorical variables. Normal data distribution was assessed by means of the Kolmogorov-Smirnov test. Agreements between retrospective and prospective data were evaluated with Cohen's Kappa and Bland-Altman limits of agreement. Kappa values above 80% were considered excellent agreement, between 60% and 80% as substantial agreement, between 40% and 60% as moderate agreement, and below 40% as poor agreement25. Global agreement was assessed by calculating the ratio of total agreement. Sensitivity and specificity were calculated to compare the retrospective report of falls in the past 12 months and the prospective monthly report on the occurrence of falls during the course of the same period (reference standard).

Sensitivity was defined as the percentage of participants who correctly recalled having fallen at least once during the past 12 months among those who reported falls in the prospective monitoring13 , 16. Specificity was defined as the percentage of participants who correctly recalled not having fallen during the past 12 months among those who reported no falls during prospective monitoring13 , 16. The same analyses were performed for recurrent falls. The level of significance was set at 5%. Analyses were performed with SPSS 16.0 statistics software (IBM (c) , SPSS Inc., Chicago, IL, USA).

Results

Sample characteristics

Most of the elderly women were in their sixties or seventies, active or moderately active, had a low educational level and moderate or poor cognitive performance12 (Table 1).

Table 1.

Sample characteristics (n=116).

Variable	% (n)	Mean ± SD
Age (years) 60 - 69 years 70 - 79 years 80 or older	- 46.6% (54) 44.0% (51) 9.5% (11)	70.40 ± 6.187 - - -
Bone metabolic diagnosis Osteopenia Osteoporosis	53.4% (62) 46.6% (54)	- -
Schooling (years)	-	4.38 ± 2.73
MMSE total score by schooling group Illiterate 1 – 4 years 5 – 8 years ≥ 9 years	19.8 (23) 51.7 (60) 21.5 (25) 7.0 (8)	24.67 ± 1.53 23.67 ± 2.54 26.16 ± 2.1326.62 ± 1.68
MMSE cognitive performance classification12 17-23 points 24-28 points 29-30 points	43.2% (50) 51.7% (60) 5.2% (6)	- - -
Physical activity level (HAP - AAE) Inactive Moderately active Active	- 2.6% (3) 39.7% (46) 57.8% (67)	75.71 ± 10.10 - - -
Drugs in continuous use	-	4.53 ± 2.48

MMSE = Mini-Mental State Examination; HAP = Human Activity Profile questionnaire; AAS = Adjusted Activity Score.

Occurrence of falls in the present sample and their characteristics

In the monthly phone call-based monitoring over the 12 months of the study, 52 (44.8%) participants reported no falls, 40 (34.5%) reported one fall, and 24 (20.7%) reported two to six falls, with a mean of 1.91±1.34 falls among fallers. Most of the falls occurred outside of the home (50.8%) due to slipping (40.7%) or stumbling (32.2%) and resulted in some type of injury (69.8%) (Table 2).

Table 2.

Characteristics of fall incidents (n=116).

Variable	Percentage (n)
Incidents of falls in total sample No fall Single fall Recurrent falls	44.8 (52) 34.5 (40) 20.7 (24)
Fall location At home Outside home At home and outside home	33.9% (20) 50.8% (30) 15.3% (9)
Fall conditions No apparent circumstance SlipStumble Many causes Half-steppedDizziness Drug effect	3.4% (2) 40.7% (24) 32.2% (19) 13.6% (8) 5.1% (3) 3.4% (2) 1.7% (1)
Consequences of falls No consequences BruisingAbrasions Lacerations Fractures Pain and edema Pain	30.2% (19) 11.1% (7) 41.3% (26) 3.2% (2) 3.2% (2) 7.9% (5) 3.2% (2)

Retrospective self-reporting of falls

As for the retrospective self-report of falls during the past 12 months at the end of the study, 70 (60.3%) participants denied having fallen, 33 (28.5%) mentioned one fall, and 13 (11.2%) reported two to five falls, with a mean of 1.51±0.93 previous falls among fallers.

Agreement and comparison between retrospective self-reporting and prospective monitoring of falls for total sample

Data on the agreement between prospective monitoring and retrospective self-reporting of falls and recurrent falls over a period of 12 months are shown in Table 3. The percentages of global agreement between both methods were 79.31% for falls and 88.79% for recurrent falls. However, chance-corrected agreement produced Kappa indices of 0.595 for falls and 0.589 for recurrent falls, indicating moderate agreement25 between the above methods.

Table 3.

Agreement between prospective monitoring and retrospective self-reporting of falls and recurrent falls over 12 months (n=116).

Retrospective Self-report	Prospective Monitoring		Total
	Non-faller (0 fall)	Faller (≥1 fall)
Non-faller (0 fall)	49	21	70
Faller (≥ 1 fall)	3	43	46
Total	52	64	116
	Non-recurrent faller (≤1 fall)	Recurrent faller (≥2 falls)
Non-recurrent faller (≤1 fall)	91	12	103
Recurrent faller (≥2 falls)	1	12	13
Total	92	24	116

Retrospective self-report of falls: Sensitivity = 43/64 = 67.2% (CI 95% 55.0 - 77.4). Specificity = 49/52 = 94.2% (CI 95% 84.4 - 98.0). Percentage agreement = (49 + 43)/116 = 79.31%. Kappa = 0.595, p=0.001 (CI 95% 0.458 - 0.732). Retrospective self-reporting of recurrent falls: Sensitivity = 12/24 = 50% (CI 95% 31.4 - 68.6). Specificity = 91/92 = 98.9% (CI 95% 94.1 - 99.8). Percentage agreement = (91 + 12)/116 = 88.79%. Kappa = 0.589, p=0.001 (CI 95% 0.395-0.783).

The difference between the number of falls obtained with prospective monitoring and retrospective self-reporting versus the mean of both measurements is shown in a Bland-Altman plot in Figure 1. In this analysis, the mean of the difference between the above methods was 0.35, with a standard deviation of 0.83. The Bland-Altman limits of agreement were -1.30 to 2.01 (95% limits of agreement).

Figure 1.

Bland-Altman diagram: comparison of prospective monthly monitoring over 12 months and annual retrospective self-reporting (previous 12 months) methods. UL = upper limit; LL = Lower Limit; Mean difference = 0.35. SD = 0.83. Limit of agreement = 0.35±1.66.

The evaluations of the sensitivity and specificity of the retrospective self-reporting of falls and recurrent falls are listed in Table 3. The sensitivity and specificity of the retrospective self-reporting of at least one fall in the previous 12 months were 67.2% and 94.2%, respectively. The retrospective self-reporting of two or more falls exhibited 50% sensitivity and 98.9% specificity.

For self-reported falls in the previous 12 months, under-reporting falls and recurrent falls was more common than over-reporting. Of the 64 elderly women who reported falls during the monthly monitoring, 32.8% denied having fallen in the self-reporting of previous falls. Of the 52 participants who reported no falls during the monthly monitoring, 5.76% reported falls in the self-reporting of previous falls. Similarly, of the 24 elderly women who reported recurrent falls during prospective monitoring, 50% denied recurrent falls in the retrospective self-report. Additionally, of the 92 participants who denied having fallen twice or more during prospective monitoring, 1.1% reported recurrent falls in the self-report of the 12 previous months.

However, the ratio of elderly women who incorrectly reported not having fallen in the previous 12 months decreased with the increase in the number of falls during the year of retrospective follow-up: 40% under-reporting among participants who reported one fall, 30.76% among those who reported two falls, 16.6% among those who reported three falls, and no under-reporting among those who had fallen four to six times during the study.

Comparison between retrospective self-reporting and prospective monitoring of falls for sample subgroups

The analyses of the sensitivity and specificity of retrospective self-reporting of falls and recurrent falls with respect to prospective monitoring regarding subgroups of elderly women who may or may not have suffered injurious falls are shown in Table 4. A better recall capacity of falls in the previous 12 months was observed among participants who suffered injurious falls.

Table 4.

Sensitivity and specificity of retrospective self-reporting of falls and of recurrent falls for sample subgroups.

Subgroups	Falls (≥1 fall)		Recurrent Falls (≥2 falls)
	Sensitivity (CI 95%)	Specificity (CI 95%)	Sensitivity (CI 95%)	Specificity (CI 95%)
No injuries after falls (n=19)	63.1% (41.0 – 80.8)	0%	60% (23.1 – 88.2)	92.8% (68.5 – 98.7)
Injuries after falls (n=44)	68.2% (53.4 – 80.0)	0%	47.4% (27.3 – 68.3)	100% (86.7 – 100.0)

Discussion

This study analyzed the agreement and comparison between retrospective self-reporting, by means of a single phone call at the end of data collection, and the one year prospective monitoring of falls, by means of monthly telephone calls, in a population of elderly women with low BMD who were active, had a low educational level and exhibited no cognitive impairment. The findings included a 32.8% frequency of under-reporting, moderate agreement, and limited accuracy of fall monitoring by means of retrospective self-reports relative to the past 12 months.

The compared methods exhibited moderate agreement and considerable limits of agreement, indicating a possible divergence of up to 1.66 falls and a range difference within the limits of agreement of approximately three falls. The analysis of recurrent falls revealed moderate agreement (k=0.589); however, the wide confidence interval with a bottom limit of 0.395 was indicative of poor agreement. These results suggest that the clinical substitution of prospective monitoring of falls with a retrospective single telephone method recalling falls over the past 12 months might not be reliable. Previous international studies have also investigated the agreement between retrospective and prospective monitoring of falls; however, better agreement indices were achieved. The study by Peel6 investigated the agreement between retrospective questioning on the occurrence of falls in the previous year and prospective recording with monthly fall calendars among elderly subjects in their sixties (79% women) with high incidence of falls (52%) and revealed a better level of chance-corrected agreement (K=0.70) than the present study. Hannan et al.12 evaluated the agreement between monthly prospective follow-up using a fall calendar over three months and retrospective self-reports among elderly individuals in their seventies (63% women) who were able to walk without assistance and had no cognitive impairment (56.3% exhibited moderate and 35% high cognitive performance) and also found a good level of agreement (k=0.74). Kunkel et al.8 monitored falls among the elderly in their seventies (67% men) with a history of stroke by means of retrospective self-reports and a fall diary over a period of 12 months and found substantial agreement for falls (K=0.65) and moderate agreement for recurrent falls (K=0.51). However, the authors observed a possible difference of up to five falls between the compared methods.

In this study, the recall of falls and recurrent falls in the previous year was highly specific (i.e. a few false positives), although poorly sensitive with respect to intensive monitoring by monthly phone calls. These findings confirm the important problem of under-reporting due to the use of retrospective self-reporting when monitoring falls among the elderly16. International studies on the accuracy of the one-year retrospective self-report6 , 14 , 15 also support the above statement, although with lower indices of under-reporting6 , 14 , 15. Hale et al.14 compared a retrospective self-report with a weekly prospective record sent by mail for falls that occurred over a period of 12 months among community-dwelling elderly subjects (80% women) in their seventies with no cognitive impairment. The authors observed excellent sensitivity (89%) and specificity (95%) and an under-reporting frequency of only 11%. In the analysis of the recall of falls in the past year relative to the self-reporting of falls using a monthly calendar over the same period of time, Peel6 found 79.5% sensitivity and 91.4% specificity and an under-reporting percentage of 20.5%. Sanders et al.15 compared the self-reporting of previous falls with the prospective monitoring of falls using a monthly calendar over a period of 12 months among community-dwelling elderly women at high risk of falls and fractures and high incidence of falls (42.8%), observing 77.1% sensitivity, 94.2% specificity, and 22.9% under-reporting. Additionally, similar to the present study, Mackenzie et al.16 observed that the ratio of under-reporting previous falls was reduced with the increased number of falls suffered by the elderly (60.5% under-reporting one fall and 26.3% under-reporting recurrent falls).

In this context, the following reasons have been suggested for the failure of self-reports of previous falls14 , 16: (i) forgetting events that occurred in the past, (ii) denial of falls that occurred during the period of monitoring, or (iii) the telescoping effect16, in which individuals project memorable events to a given period of time that actually occurred outside this period. However, several methods have aimed to attenuate this bias in clinical-scientific settings. Specifically, the individual's participation in the studies themselves delimits the recall period more accurately. Monthly contacts and records might further contribute to better memorization and, thus, to a more accurate recall of falls over one year13 , 14. The use of self-reports relative to larger time intervals also reduces the chances of possible recall errors13 , 14 because it allows for better memory-capture than remembrances that are restricted to a shorter period of time16. Additionally, denying falls to conceal possible signs of frailty might contribute to under-reporting15. However, this phenomenon seems to be more accentuated when the fall is relatively recent14. Thus, despite the monthly contact by phone, the use of a long reference period, and a presumably more cautious sample due to a higher fear of falls and fractures7 , 26 , 27, the results of the present study showed a lower index of agreement between the compared methods, reduced accuracy of retrospective self-reporting with respect to prospective monitoring, and significant under-reporting of falls compared to international studies6 , 8 , 12. These findings might be due to a poorer cognitive performance (there was a low cognitive performance12 in 43.2% of the sample)13and to the possibility that some participants of the present sample did not count the falls as legitimate due to their tendency to attribute them to external factors15. Additionally, the use of diaries and calendars for prospective monitoring of falls might have contributed to better memorization in the international studies6 , 8 , 12 , 15 , 16.

Some authors6 , 13 , 14 have found that the recollection of previous falls was more accurate when the episodes were accompanied by injuries, with 87%6 and 100%14sensitivity for subgroups with injurious falls, as opposed to 62%6 and 78%14 sensitivity in subgroups with non-injurious falls. The present study also detected higher sensitivity for self-reported falls in the previous 12 months within the subgroup that reported injurious falls (68.2%) compared to the subgroup that denied the occurrence of injuries (63.1%). The small difference might be because the consequences of the reported injuries were minimal.

In this study, the characterization of the cognitive profiles and educational levels of all participants, the low rate of sample loss, the implementation of both studied self-reporting methods within the same sample, and the 100% availability of prospective and retrospective data reinforce the obtained results.

However, the restricted inclusion of women with osteopenia or osteoporosis, the exclusion of individuals with cognitive impairment, and the active profile of the sample limit the generalization of the obtained results. A further significant limitation of this study was the use of phone calls to monitor the occurrence of falls, thus not allowing for the generalization of the findings to other methods. When using this method, it is important to note that even monthly prospective monitoring of falls by telephone, used as the reference standard, exhibited limitations. As for any form of self-reporting of falls, the monthly telephone call is susceptible to forgetfulness, denial or the telescoping effect16. Furthermore, the discussion of this study's findings in comparison to previous studies was limited by the heterogeneity of the definition of the event fall. While information on the economic cost of telephone calls and human resources might be useful for the future comparison of potential costs between different methods of monitoring the occurrence of falls, it was not recorded in the present study.

According to the findings on comparisons and agreement between retrospective self-reporting of falls and prospective monitoring, the replacement of monthly monitoring with retrospective questioning for the 12 previous months would only be reasonable in a population where the main interest lies in the reduction of false negatives. Thus, the authors of this study considered prospective monthly data collection methods as the ideal for longitudinal surveys on elderly individuals with the occurance of falls as the main outcome of the study8 , 12. In the clinical environment, this combination of daily records by the patient and the monthly monitoring by the staff might be very expensive8, thus justifying the careful use of retrospective interviews. In Brazil, the low educational level and reduced digital inclusion among the elderly hampered the use of either diaries or electronic media for the surveillance of falls. The authors of this study suggest increasing the awareness of this problem14, training elder-healthcare professionals to ensure proper records of the events in the Elderly Health Handbook (Caderneta de Saúde da Pessoa Idosa) and to investigate the accuracy of this tool in future studies. The authors of this study further recommend a more simple definition of falls to allow for a broader understanding11allowing more attention to be paid to the correct understanding of the definition by patients. Thus, the proper recognition of the patient's history of falls would allow clinicians and researchers to develop strategies to reduce the incidence of falls and injuries and to preserve mobility among elderly patients14, especially among those at high risk of falls and fractures.

Conclusion

The method of retrospective self-reporting of falls in the previous 12 months exhibited moderate agreement and limited accuracy with respect to the method of prospective monitoring of falls among elderly women at high risk of falls and fractures. The retrospective self-reporting of falls was more sensitive with respect to prospective monitoring among women with injurious falls than among those with no post-fall injuries. These findings call for caution when substituting monthly monitoring with retrospective questioning and indicate the importance of including associated clinical-functional information for decision-making in clinical-scientific settings.