Accuracy of recall of musculoskeletal injuries in elite military personnel: a cross-sectional study.

BACKGROUND: Self-reported data are often used in research studies among military populations.
OBJECTIVE: The accuracy of self-reported musculoskeletal injury data among elite military personnel was assessed for issues with recall.
DESIGN: Cross-sectional study.
SETTING: Applied research laboratory at a military installation. PARTICIPANTS: A total of 101 subjects participated (age 28.5±5.6 years). Study participants were active duty military personnel, with no conditions that precluded them from full duty. PRIMARY AND SECONDARY OUTCOME MEASURES: Self-reported and medical record reviewed injuries that occurred during a 1-year period were matched by anatomic location, injury side (for extremity injuries), and injury year and type. The accuracy of recall was estimated as the per cent of medical record reviewed injuries correctly recalled in the self-report. The effect of injury anatomic location, injury type and severity and time since injury, on recall, was also assessed. Injuries were classified as recent (≤4 years since injury) or old injuries (>4 years since injury). Recall proportions were compared using Fisher's exact tests.
RESULTS: A total of 374 injuries were extracted from the subjects' medical records. Recall was generally low (12.0%) and was not different between recent and old injuries (P=0.206). Injury location did not affect recall (P=0.418). Recall was higher for traumatic fractures as compared with less severe non-fracture injuries (P values 0.001 to <0.001). Recall for non-fracture injuries was higher for recent as compared with old injuries (P=0.033). This effect of time since injury on recall was not observed for fractures (P=0.522).
CONCLUSIONS: The results of this study highlight the importance of weighing the advantages and disadvantages of self-reported injury data before their use in research studies in military populations and the need for future research to identify modifiable factors that influence recall. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

To the best of our knowledge, this is the first study to assess recall of comprehensive musculoskeletal injury data in an elite military population.

The strength of this study is that a comprehensive list of self-reported and medical record reviewed musculoskeletal injuries was included in the analysis.

The study included an analysis of the effect of injury type and severity, injury location and time since injury on recall of musculoskeletal injury data.

This study was conducted among a sample of elite military personnel, and the results of this study may not be generalisable to the regular military population and to non-military general populations.

Introduction

Musculoskeletal injuries are common in physically active individuals, including military personnel.1–7 Musculoskeletal injuries in the military are a common cause of pain, morbidity, disability,8 loss of duty time,1 9 increased medical costs9 and medical evacuation from theatre.10 Accurate injury data are necessary to measure injury incidence11 for risk factor identification and for evaluation of the efficacy of injury prevention programmes.12

Injury epidemiology studies often use self-reported data. There are many advantages of self-reported data, such as time efficiency, availability and cost-effectiveness. Self-reported injury history can be expected to include information about all injuries that have occurred in the past, even if medical care was not sought or if care was sought from a healthcare professional outside the system from which medical records were obtained.13 A limitation of self-report is that humans are inherently limited in their ability to recall all information.14 The subjects may not have the information being requested (leading to low concurrent validity)14 or there may be issues with reduced recall as time since injury increases.15 16 Medical records have shortcomings too. Data about injuries will be contained in medical records only if medical care was sought. Studies among military personnel suggest that they may not seek medical care for injuries, which can give rise to an underestimation of the injury frequency estimated using medical chart reviewed injury data.17 18

Previous studies in non-military populations have assessed the concurrent validity and recall of self-reported sports injury-related data,13 19 20 occupational injury-related data21 22 and motor vehicle traffic crash-related data.23 24 Studies among military personnel have assessed self-reported data such as the Army Physical Fitness Test (APFT) scores,25 26 body mass index,26 deployment data,27 data on exposure to hazards,28 29 measures of back pain30 and recent mental health diagnosis.31 The validity and recall of self-reported data has varied, with different implications for the use of self-reported data in epidemiology studies. The study of the validity of self-reported measures of back pain among US Army Reserve soldiers did not assess injuries other than back pain.30 Previous studies in non-military populations by Landen and Hendricks and Warner et al have shown that recall of injuries is affected by injury severity and time since injury, with a reduction in recall of less severe injuries, as time since injury increased. This reduction of recall with increasing time since injury was not observed for severe injuries.21 32 Previous studies have also shown that recall of injuries is often affected by anatomic location of the injury, with low recall for back pain30 and upper extremity injuries.33 To the best or our knowledge, no studies have been performed to assess self-reported recall of comprehensive musculoskeletal injury data in military personnel.

Previous studies assessing self-reported recall have either examined injuries among non-military personnel,13 19–24 or assessed non-injury data25–29 31 or only one musculoskeletal condition among military personnel.30 The current study aimed to examine the self-reported recall of comprehensive data about musculoskeletal injuries and related musculoskeletal conditions among a group of elite military personnel. Assessing the accuracy of recall included measuring the per cent of musculoskeletal injuries from medical records that were correctly self-reported by subjects and testing for the effect of time since injury on recall. It was hypothesised that recall would be influenced by injury type and severity, injury anatomic location and time since injury.

Methods

Study design and study participants

All study subjects were active duty military personnel with no conditions that precluded them from full duty. Study subjects were participants in a larger comprehensive performance optimisation and injury prevention research study, among Special Operations Forces (SOF) personnel. Only those SOF personnel who were assigned to a team/unit were included in the study to ensure that the study sample was homogeneous with similar occupational characteristics. Also, participants whose medical record review and self-reported injury history were conducted during the same year were included so that injuries that occurred during the same time frame could be matched between the two sources of data. At the time of data collection, only men were eligible to qualify in the military group that was studied. Data collection for the study was conducted in a research laboratory located at the military installation. Human subjects’ approval was obtained from the university and the Office of Naval Research. Written informed consent was obtained from study participants. Of the 132 subjects included in this cross-sectional study, 101 subjects had at least one injury in their medical records, and these subjects were included in this analysis.

Operational definition of injury

A musculoskeletal injury was defined as an injury to the musculoskeletal system (bones, ligaments, muscles, tendons, among others) and related musculoskeletal conditions (tendonitis, tenosynovitis, bursitis, plantar fasciitis, musculoskeletal pain, among others). Both are henceforth referred to as injury in this manuscript.

Medical record review

The medical records of the subjects were reviewed by certified athletic trainers to extract data about musculoskeletal injuries. Medical record reviews were performed in 2009 and 2010.

Self-reported injury history

Subjects were interviewed about their history of musculoskeletal injuries by a certified athletic trainer. To minimise bias in reporting injuries, subjects were assured that their injury data would be de-identified before entry into a database. Injury self-reports were obtained in 2009 and 2010.

Injury matching

De-identified medical record reviewed and self-reported injury data were entered into a customised database. Within subject, self-reported injuries were compared with medical record reviewed injuries to assess the accuracy of recall of self-report. Within each subject, injuries were identified and matched first by anatomic location (head/face, spine, torso, upper extremity and lower extremity), injury side (only for upper and lower extremities) and year of injury. For upper or lower extremity injuries, bilateral injuries were matched to bilateral, left or right-sided injuries. A second level of matching was then performed, matching on injury type, in addition to the criteria for matching used in the first level. The medical record review and self-report were completed within 5 days of each other for each subject.

Statistical analyses

Medical record reviewed injuries were not used as the criterion measure, but accuracy of recall was expressed as the per cent of medical record reviewed injuries correctly recalled in the self-report (figure 1). Recall was calculated as the per cent of injuries in the medical charts that were accurately remembered by the subject in his injury self-report. A higher recall indicated that the subject remembered a greater proportion of the injuries that were recorded in his medical chart.

Figure 1

Formula used to calculate recall (as a per cent).

An analysis was conducted to understand the effect of time since injury (duration in years since onset of injury) on the accuracy of recall. The time since injury was calculated by subtracting the year of onset of injury from the year of survey. If the injury occurred during the same year as the year of survey, the time since injury was calculated as 0 years.

Fisher’s exact tests were performed to assess the influence of injury anatomic location, injury type and time since injury (number of years since onset of injury) on recall. Statistical analyses were performed using IBM SPSS Statistics, V.23 (IBM Corporation).

Results

Medical records were reviewed and injury self-report was obtained from 132 subjects (age=27.6 ± 5.4 years, mean ±SD). Of these, the medical records of 101 subjects (age=28.5 ± 5.6 years, mean ±SD) had at least one injury recorded. Data in the injury self-reports of these 101 subjects were analysed further to measure recall.

There were a total of 374 medical record reviewed and 294 self-reported injuries, among the 101 subjects included in the analysis. Table 1 contains a description of anatomic location and common injury types of these injuries. The most common anatomic location for injury was the lower extremity. Common injury types recorded in the medical charts were strain, sprain and pain. The injury type pain included injuries identified as pain, spasm, ache, tightness or soreness.

Table 1

Description of medical record reviewed and self-reported injuries

	Medical record reviewed injuries (374 injuries) (%)	Self-reported injuries (294 injuries) (%)
Anatomic location
Lower extremity	204 (54.5)	118 (40.1)
Upper extremity	97 (25.9)	115 (39.1)
Spine	58 (15.5)	32 (10.9)
Torso	12 (3.2)	9 (3.1)
Head/face	2 (0.5)	20 (6.8)
Unknown	1 (0.3)	0 (0.0)
Common injury types
Strain	62 (16.6)	25 (8.5)
Sprain	50 (13.4)	33 (11.2)
Pain	39 (10.4)	6 (2.0)
Tendonitis/tenosynovitis/  tendinopathy	35 (9.4)	13 (4.4)
Contusion	26 (7.0)	9 (3.1)
Traumatic fracture	26 (7.0)	80 (27.2)
Stress fracture	12 (3.2)	7 (2.4)

On matching self-reported injuries with medical record reviewed injuries, recall was generally low and varied by various injury attributes. This is illustrated in tables 2-4 and figure 2. Table 2 includes a description of recall for all injuries and for extremity injuries.

Figure 2

Influence of time since injury on recall (Fisher’s exact test).

Table 2

Self-reported recall of medical record reviewed injuries

Injury attribute	Recalled injuries (%)
All medical record reviewed injuries (374 injuries)
Type of matching—location and year	77 (20.6)
Type of matching—location, year and type	45 (12.0)
Medical record reviewed injuries affecting extremities (301 injuries)
Type of matching—location, year and side	63 (20.9)
Type of matching—location, year, side and type	34 (11.3)

Recall did not differ significantly by anatomic location of the injury (table 3). The highest recall was observed for head/face injuries, but proportions of injuries recalled were not significantly different between anatomic locations.

Table 3

Influence of injury anatomic location on self-reported recall

Injury anatomic location in medical record	Recalled injuries (%)—matched on location and year	Recalled injuries (%)—matched on location, year and type
Lower extremity (204 injuries)	36 (17.6)	23 (11.3)
Upper extremity (97 injuries)	27 (27.8)	11 (11.3)
Spine (58 injuries)	11 (19.0)	8 (13.8)
Torso (12 injuries)	2 (16.7)	2 (16.7)
Head/face (two injuries)	1 (50.0)	1 (50.0)
Fisher’s exact test P value	P=0.202	P=0.418

Tests were repeated without including head/face injuries, since there were only two head/face injuries with an unusually high recall. Even after deleting head/face injuries, the statistical test was not significant on matching by anatomic location and year (Fisher’s exact test P=0.234) or on matching by anatomic location, year and type (Fisher’s exact test P=0.825).

The recall was influenced by injury severity (table 4). Injury types included in the analyses were traumatic fractures, stress fractures, sprain, tendonitis/tenosynovitis/tendinopathy, contusion, strain and pain. Traumatic fractures had the highest recall. Traumatic fractures were the most severe injury type included in our study, and it is expected that subjects would be more likely to remember a traumatic fracture, than a less severe injury. The second highest recall was for stress fractures. Stress fractures are a severe injury but have an insidious onset unlike traumatic fractures. This may make them less likely to be recalled, as compared with traumatic fractures. The recall for traumatic and stress fractures was not significantly different from each other for both types of matching.

Table 4

Influence of injury type and severity on recall

Injury type in medical record	Recalled injuries (%)—matched on location and year	Recalled injuries (%)—matched on location, year and type
Traumatic fractures (26 injuries)	15 (57.7) (comparison group)	13 (50.0) (comparison group)
Stress fractures (12 injuries)	4 (33.3) P=0.295	2 (16.7) P=0.077
Sprain (50 injuries)	13 (26.0) P=0.011*	7 (14.0) P=0.001*
Tendonitis/tenosynovitis/ tendinopathy (35 injuries)	5 (14.3) P=0.001*	2 (5.7) P<0.001*
Contusion (26 injuries)	3 (11.5) P=0.001*	1 (3.8) P<0.001*
Strain (62 injuries)	7 (11.3) P<0.001*	2 (3.2) P<0.001*
Pain (39 injuries)	1 (2.6) P<0.001*	1 (2.6) P<0.001*

*Fisher’s exact test P values <0.05; recall significantly different from recall of traumatic fractures.

The recall for the remaining injury types was also compared with traumatic fractures. For all other injury types, recall was significantly lower as compared with the recall for traumatic fractures (Fisher’s exact test P<0.05). This shows that, as expected, injury severity was related to recall, and subjects were more likely to remember traumatic fractures, which are severe injuries, as compared with various less severe injuries.

An analysis was conducted to understand the effect of time since injury (duration in years since onset of injury) on the accuracy of recall. The year of onset of injury was missing for 5 of the 374 medical record reviewed injuries (5/374=1.3%). For injuries with known year of onset (n=369), the time since injury in years ranged from 0 to 20 (median=4 years, IQR=2 to 7 years). A separate analysis was conducted to assess recall for each year since the onset of injury. Probably due to the small sample sizes in each individual year, there were fluctuations in recall over the range of time since injury. But overall, there was a tendency towards reduction in recall per cent with increase in time since injury. Out of 31 injuries with time since injury greater than 12 years, only one injury could be matched on location and year, and no injuries could be matched on location, year and type.

Data about time since injury was available for 369 injuries (369/374=98.7% of injuries). The median time since injury was 4 years. To further analyse the effect of time since injury on recall, injuries with known time since injury were divided into two groups—recent injuries (time since injury ≤4 years, 196 injuries) and old injuries (time since injury >4 years, 173 injuries). Recall was higher for recent injuries as compared with old injuries (figure 2), though this effect was statistically significant only when matched on anatomic location and year.

To analyse the interaction between injury type/severity and time since injury and their effect on recall, injuries were divided into two strata—severe injuries (fractures only—traumatic or stress fractures) and less severe injuries (all other injuries in the dataset). The analysis of the influence of time since injury on recall was repeated within each of the two strata of injury severity (figure 2). The recall for fractures was not influenced by time since injury. In contrast, for less severe injuries, recall was influenced by time since injury. Recall was higher for recent as compared with old injuries. This was observed when injuries were matched on anatomic location and year and when injuries were matched on anatomic location, year and injury type. These results show that injury severity influenced the effect of time since injury on recall.

Discussion

Injury epidemiology studies, including those among military populations, often rely on self-reported injury data. There have been concerns that self-reported data may have issues with recall, especially as time since injury increases. The results of this study among a sample of elite military personnel demonstrated that self-reported recall was generally low and was influenced by injury severity, with recall of traumatic fractures being significantly higher than the recall of non-fracture injuries included in the analysis. There was an interaction between time since injury and injury severity on recall. Recall of less severe non-fracture injuries was influenced by time since injury, with recall of recent injuries being higher than the recall of old injuries. The recall for fractures was not influenced by time since injury.

To the best of our knowledge, no previous studies have assessed the accuracy of recall of comprehensive musculoskeletal injury data in elite military populations. The strength of this study is that a comprehensive list of self-reported and medical record reviewed musculoskeletal injuries was included in the analysis, which allowed an assessment of the effect of injury type and location and interaction with time since injury on injury recall. One of the limitations of medical records is that data about injuries will be available only if medical care was sought and only if the medical care was sought from an institution from which medical records were extracted. This problem may be compounded in the military due to frequent movements among military installations and deployment. Also military personnel may not seek medical care, giving rise to under-reporting of injuries.17 18 Considering this, it was decided to not use medical records as a criterion measure but instead assess the per cent of injuries which were accurately recalled in self-report (recall as a per cent). This study made possible to assess false negative responses as lack of recall (100—recall as a per cent). It was not possible to measure the false positive responses (injuries recalled in self-report but absent in medical records). It was not the objective of this study to measure false positive responses.

Previous studies have assessed the accuracy of recall of injuries among athletes, with relatively short recall times. The validity of self-reported recall has varied according to the time since injury, injury type and severity, level of detail of data and process of validation (comparison with criterion measure). In a study by Valuri et al assessing the validity of a 4-week recall of injuries, self-reports were compared with external data obtained from multiple sources.13 Agreement was high for the body part injured (kappa=0.78, P<0.001) and the level of injury treatment sought (kappa=0.76, P<0.001). Despite the short duration of recall, agreement was low for injury severity, especially for severe injuries (positive predictive value=0.13). A study by Gabbe et al used a longer recall time (12 months) to compare self-reported injury history with only one source of prospectively collected injury surveillance data, which was used as the criterion measure.19 In the study by Gabbe et al, recall was influenced by the level of information sought (100% for injury occurrence (yes or no), 78.6% for number of injuries, 78.6% for injured body region and 61.4% for number, region and diagnosis considered together). Unlike the studies by Gabbe et al or Valuri et al, which looked at relatively recent injuries, this study assessed the recall of all medical record reviewed injuries, making the effect of time since injury important.

No previous studies among military personnel have assessed the validity of a comprehensive injury self-report, but validity of other self-reported data over relatively short recall times has been measured.25 27 29–31 The validity has varied, and this may have been influenced by the nature of information sought in the self-report. Carragee and Cohen studied the validity of self-reports of no previous back pain by comparing these data to the results of monthly surveillance reports.30 Most soldiers with high numerical rating scale back pain scores and disability during the monthly surveillance described themselves as being asymptomatic for back pain problems. Smith et al used data from subjects in the Millennium Cohort Study, comparing self-reported deployment data with electronic deployment data.27 There was substantial agreement for deployment status, frequency and number of deployments (k=0.81, 0.71 and 0.61, respectively). Another study assessing the validity of self-reported APFT scores, by comparison to the soldiers’ actual scores,25 demonstrated that soldiers tended to slightly over-report performance, but the correlations between self-reported and actual scores were relatively high (Pearson correlation coefficient range 0.71–0.83, P<0.01 for all). Unlike the high validity reported for APFT scores or for deployment data, the results of a study by Nevin assessing the validity of a self-report of recent mental health disorder diagnosis among US service members demonstrated low validity.31

A high occurrence of musculoskeletal injuries has also been described among military populations outside the USA.34–36 Boroujeni et al assessed the effect of an 8-week training programme on lower extremity and lower back clinical findings among a sample of Iranian male recruits. Self-reported data and the results of examination by a physician were analysed. Although the authors did not describe comparison of injuries between the two sources of data, they state that the results of examination by the physician mirror the pattern of self-reported data from the recruits.37

Injury epidemiology studies among other non-military subjects have demonstrated the effect of time since injury and the resultant bias in injury risk estimates.21 32 Warner et al assessed the effects of recall reference period on the number of reported injuries and poisonings in the National Health Interview Survey.32 As the recall reference period increased, there was a reduction in the number of episodes of less severe incidents reported. This trend of reduced recall was not observed for severe injuries, such as those involving fractures and hospitalisation. The findings are like this study, since recall was adversely affected by time since injury and the most severe injury type (fractures) had the highest recall, which was not influenced by time since injury. Harel et al 38 and Cummings et al 39 demonstrated a similar effect of time since injury and injury type/severity on recall. Landen and Hendricks assessed the effect of recall on reporting of at-work injuries from the Occupational Health Supplement to the National Health Interview Survey.21 The incidence rate for all at-work injuries adjusted for recall was 32% higher than the unadjusted rate. Recall estimated in the current study was low compared with non-military studies. This may be specific to the military, since they have a high frequency of injuries, making it difficult to remember all of them. Due to this reason, results of this study may not be generalisable to non-military populations.

A limitation of this study was that since it was conducted among a small sample of elite military personnel who suffer a very high frequency of musculoskeletal injuries, these results may not be generalisable to non-military general populations or even to regular military personnel. Of the 132 participants included in the study, 101 had at least one injury recorded in their medical charts. A total of 374 medical record injuries were considered in the analysis. Future studies would need to analyse recall of injuries in military populations, including regular military forces, using larger samples. Larger samples would allow analysis of recall per cent over smaller increments of time.

To avoid issues associated with self-reported data, injury data may be obtained from medical records, but medical record data also have shortcomings. There may be issues with concurrent validity due to physician, coding and keypunch errors and incomplete data.40 41 Medical records include documentation of all injuries for which medical care was sought, irrespective of injury severity. Many of these injuries may not cause disability or negatively impact function. These injuries with less impact are less likely to be recalled by an individual and may be a cause of the low recall seen in the current study. In the current study, as expected, more severe injuries had better recall.

In any population, medical records will contain data only if medical help was sought. Care from health professionals is sought mostly for self-perceived serious injuries because of which medical records can have incomplete data especially for minor injuries.13 42 Data about injuries can also be obtained from military electronic healthcare databases, such as the Total Army Injury and Health Outcomes Database (TAIHOD), which is maintained by the US Army Research Institute of Environmental Medicine, the Military Health System Data Repository and the Military Health System Mart. An advantage of using such databases is that all injury data where TRICARE (managed service healthcare programme for military members and their dependents) was the payer will be captured. Access to injury data in from these databases was not available for the current study. Future studies can use data in central databases, such as TAIHOD as criterion measures, to assess the criterion validity (sensitivity as well as specificity) of self-reported injury data. Comparing self-reported injury data to electronic medical records and International Classification of Diseases codes, instead of written medical records, could improve data collection and sample size in future studies.

On matching self-reported and medical record reviewed injuries, recall was low (20.6% when matched by injury anatomic location and year, and only 12.0% when matched by anatomic location, year and injury type). For a large proportion of self-reported injuries, data about the exact date of injury were missing and only the injury year was available. Due to missing injury dates, matching could not be conducted by exact dates. Also, matching was conducted within anatomic location, so injuries affecting different sublocations within the same anatomic location could be matched to each other. This may have overestimated the recall per cent, and the actual recall per cent was likely lower.

Given the known limitations of medical record data, the results of this study highlight the importance of weighing the advantages and disadvantages of injury self-report and medical record reviewed injury data. Injury self-reports likely contain data on mild injuries for which medical care was not sought. If self-reported injury data are used, it may be best to limit this to recent injuries. The current study did not assess recall at monthly increments, but a study by Jenkins et al 15 suggested that recall periods greater than 2 months can negatively impact recall of injuries. Warner et al have recommended an even shorter recall period of 5 weeks.32 Limiting injuries to only recent injuries would also need to be balanced with the concern of being able to describe only a small number of injuries. Further investigation of modifiable factors affecting recall of injury data and methods to improve recall are needed, especially in military populations, in whom injuries are frequent.

Conclusions

The accuracy of self-reported recall of musculoskeletal injury in elite military personnel studied was influenced by injury severity and time since injury, with recall being lower for old injuries and less severe injuries. This study highlights the importance of choosing the source of injury data based on the type and severity of injury under study, study design, nature of the study population and the availability of medical records or a centralised database. Future research is needed to identify modifiable factors that influence recall in military populations and create strategies to improve recall.