Validation and reliability of Falls Risk for Hospitalized Older People (FRHOP): Taiwan version.

A comprehensive fall risk assessment can provide information for effective prevention and intervention measures and reduce falls among hospitalized elderly people. The purpose of this study was to develop a Chinese version of an inpatient fall risk assessment tool and evaluate its validity and reliability.This study employed the Falls Risk for Hospitalised Older People (FRHOP) assessment to construct a FRHOP-Taiwan Version (Tw-FRHOP) through forward, synthesized, and backward translation. A face validation was conducted by 5 clinical nurses and a content validation was conducted by 5 specialists using the content validity index (CVI) to validate the proposed model. Thirty hospitalized older adults in an internal care unit were selected for an interrater reliability assessment, conducted separately by specialists in 4 disciplines (i.e., nurses, physicians, occupational therapists, and physiotherapists) by using Cohen kappa statistic and intraclass correlation coefficients (ICCs). Specifically, the assessment rating developed in the Tw-FRHOP was compared with the Morse Fall Scale (MFS), St. Thomas Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY), and the Hendrich II Fall Risk Model (HIIFRM) for criterion validation.According to the analysis results, the CVI was 0.94, and the indexes of criterion-related validity for the FRHOP-Taiwan Version, MFS, STRATIFY, and HIIFRM were 0.49, 0.63, and 0.54 (all P < .001), respectively. In addition, after interrater reliability testing was conducted, the results indicated that the index of response consistency in each discipline was 86.7% to 100%, and the values of Cohen kappa were 0.651 to 1.000. The ICCs of the discipline-related subscale were 0.97 to 1.00.The Tw-FRHOP is a multidisciplinary comprehensive fall risk assessment that can serve as a satisfactorily valid and reliable reference tool for medical personnel with full professional training, as well as inpatient fall prevention interventions for multidisciplinary teams in hospitals.

Introduction

In western countries, the incidence of inpatient falls is approximately 1.3 to 8.9 falls per 1000 occupied bed days[; 30% to 51% of inpatients who fall sustain injuries.[ In Taiwan, inpatient falls are the second most common patient safety event (26.6% of all events) after medication events (30.9%). Approximately 53% of patients in these cases are injured.[ Since October 2008, Medicare and Medicaid in the United States no longer pay for some hospital-acquired conditions, including severe injuries caused by inpatient falls.

Strategies to reduce the incidence of inpatient falls include surveillance and intervention of patients’ risk factors. Comprehensive fall risk assessments can provide information for effective prevention and intervention. Fall risk scales are wildly used for screening or the assessment of risk factors.[ Individualized interventions derived from the results of fall risk assessments can reduce in-hospital falls.[ Many assessment tools have been developed to identify hospitalized patients with high fall risks. Some of these tools are accompanied by standardized prevention strategies. However, the validity of these tools varies, with 43% to 95% sensitivity and 27% to 78% specificity.[ Overestimating fall risks might limit the independence and mobility of hospitalized patients.[

According to the clinical guidelines provided by the National Institute for Health and Care Excellence (NICE), multifactorial assessments and interventions implemented by an interdisciplinary team can more effectively prevent the risk of in-hospital falls than grading through a fall risk assessment alone.[ Adopted as the fall risk assessment tool in this study, FRHOP (Falls Risk For Hospitalized Older People) was developed by the National Ageing Research Institute of Australia by multidisciplinary core team consisting of a geriatrician, a physiotherapist, a registered nurse, and an occupational therapist to provide intervention strategies for patient customization based on the assessment results. The assessment includes 19 items for a total of 45 points and covers the following 11 aspects: recent falls, medication, medical condition, sensory loss and communication, cognitive status, continence, nutritional condition, functional behavior, feet and footwear and clothing, balance, and transfers and mobility. A score of 0 to 5 indicates a low risk of falling, a score of 6 to 20 a medium risk, and a score 21 to 45 a high risk. The intraclass correlation coefficient (ICC) of interrater reliability is 0.85 [95% confidence interval (95% CI): 0.55–0.95], that of test–retest reliability is 0.95 (95% CI: 0.84–0.99). The sensitivity is 0.57 and the specificity is 0.68.[

The purpose of this study was to develop a tool that could be used by a multidisciplinary team to assess the risk factors of falling and to suggest customized prevention and intervention strategies for older inpatients with high fall risks; these strategies would be aimed at reducing their fall rates and the harm incurred. The FRHOP was adopted and translated and tested through a standardized process and empirical examination to verify the validity and reliability of the translated version.

Methods

Questionnaire design

A forward translation procedure was first conducted by 2 qualified bilingual translators to render the content of FRHOP into a Chinese version; 1 translator was employed as a specialist with a background in medicine and nursing, considering the necessity of understanding the assessment framework and the clinical application of this questionnaire, and the other translator was a nonspecialist.

Second, a backward translation was created by translating the previously retrieved Chinese version into a corresponding English version. The derived English backward translation was compared and contrasted with the original version of FRHOP to modify the corresponding Chinese sections with less than 70% semantic consistency in this English version in accordance with the official FRHOP.

Subsequently, face validation was conducted by 5 nursing personnel regarding the linguistic comprehensibility and fluency of the Chinese version to suggest further modifications.

Content validation

A content validity assessment was conducted by 1 geriatrician, 2 physiotherapists, and 2 senior nursing supervisors using a content validity index (CVI) for measurement on a scale of 1 to 5 points: highly inadequate (1 point), inadequate (2 points), adequate (3 points), appropriate (4 points), and highly appropriate (5 points). Individual items with lower consistency were subsequently discussed and amended by a group of specialists to attain a final Chinese version of the FRHOP.

Participants

Thirty inpatients from an internal medicine acute care unit located in a medical center in Taipei were selected for evaluation within 3 days of their hospitalization. The inclusion criteria for the research participants of this study were patients who were aged over 50 years; able to communicate with an interviewer effectively; and fully informed and had provided consent. In addition, the exclusion criteria were patients who were unconscious, bed-ridden, and disabled; and unable to complete the assessment due to death, discharge, transfer from the research ward, or loss of consciousness within 3 days of their hospitalization.

Criterion validation

The standard of the criterion validation in this study was based on the More Fall Scales (MFS),[ St. Thomas Risks Assessment Tool in Falling Elderly Inpatients (STRATIFY),[ and Hendrich II Fall Risk Model (HIIFRM).[ Concurrent validity was adopted for validity testing.

Interrater reliability

The raters of this study were 4 groups of specialists: nursing staff, physicians, occupational therapists, and physiotherapists. Two personnel were assigned to each group to conduct an independent evaluation for patients within 3 days of their hospitalization. Specifically, the percentage of agreement and Cohen kappa coefficient were employed to measure the consistency of individual items. Furthermore, the ICC (2,1) was employed to measure the interrater reliability of the discipline-related subscale. Before the formal assessment was conducted, raters within the same disciplinary group were provided with 2 rounds of training and subsequently convened to attain a consensus on the assessment standards.

Cognitive status

Although the Abbreviated Mental Test score was officially adopted in the FRHOP, to account for cultural differences, the Short Portable Mental Status Questionnaire[ was adopted as the assessment tool for patient cognitive status in the Tw-FRHOP after consultation with the FRHOP designers.

Ethics approval

This study was approved by the Institutional Review Board of the Tri-Service General Hospital National Defense Medical Center, Taipei, Taiwan (TSGHIRB Approval Number: 2-101-05-144). All interviewees were informed of the nature of their participation in this study and their rights, and all signed informed consent forms.

Results

A total of 30 participants, 17 men and 13 women, were recruited from an internal medicine acute care unit in a hospital. Their ages ranged between 52 and 86 years, with an average of 68.1 ± 11.5 years. Ten participants were reported to have fallen within the recent 12 months, and 2 among them had experienced 2 and more falls; specifically, 9 patients had been injured by falls, 6 had been diagnosed with minor injuries requiring medical attention, and 1 was identified as having sustained severe injuries (e.g., fractures). In addition, 26 of the participants had been prescribed more than 4 types of medicine; some of the most common high-risk medications they received were as follows: analgesics (22 persons), antihypertensives (20 persons), sedatives (16 persons), and vasodilators or cardiac medications (16 persons). Among the patients prescribed these medications, 16 were observed to have more than one chronic medical condition affecting their balance and mobility; the most commonly identified conditions among them were diabetes (10 persons), cardiac conditions (10 persons), and strokes or transient ischemic attacks (4 persons).

Content validity

The content validity assessment was designed with 39 items. The CVI collected from the 5 specialists were between 0.77 and 1.00 and had an average CVI value of 0.94. Among the total of 195 responses assessed to determine content validity, 93.8% scored above 4 in appropriateness. Only 4 items were rated as inadequate by 1 or 2 specialists. The main reason was because of controversy resulting from the translation. In subsequent meetings, all specialists reached a consensus.

Criterion validity

A moderate correlation was observed in the total scoring of the following comparison groups of assessment tools, as summarized in Table 1: FRHOP and MFS (Pearson correlation r = 0.489, P = .001); FRHOP and STRATIF (Pearson correlation r = 0.633, P < .001); and FRHOP and HIIFRM (Pearson correlation r = 0.541, P < .001).

Table 1

Pearson correlation table for the FRHOP-Taiwan Version, MFS, STRATIFY, and HIIFRM.

Interrater validity

Physician

The first to tenth items had 96.7% to 100.0% agreement and the κ coefficient was 0.930 to 1.000. Furthermore, the ICC (2,1) was 0.996 (95% CI: 0.995–0.997), showing excellent interrater reliability among the physicians (Table 2).

Table 2

Reliability analysis of the FRHOP-Taiwan Version (n = 30).

Nurse

The 11th to 15th items had 100% agreement, κ = 1.000 and the ICC (2,1) was 1.000, indicating excellent consistency between the 2 clinically experienced nursing personnel.

Occupational therapist

According to the 16th to 19th items, for occupational therapists, agreement was in the range of 86.7% to 96.7% and the κ coefficient was in the range of 0.651 to 0.932. Specifically, except for the fair to moderate consistency that was manifested in the item “does the patient's clothing fit well?” (κ = 0.651) and “ADL & Mobility” (κ = 0.716), the response consistency among the remaining items was excellent. The ICC (2,1) for the specialty-related subscale was 0.979 (95% CI: 0.970–0.985), indicating excellent reliability.

Physiotherapist

The 20th item, “the Timed Up and Go test and the Functional Reach test” and the 21th item, “independence in transferring and in their gait” displayed excellent consistency, with 86.7% (κ = 0.782) and 93.4% agreement (κ = 0.874), respectively. The ICC (2,1) was 0.965 (95% CI: 0.942–0.979).

Table 3 summarizes all of the interrater reliability assessment results and correlations. The correlation values among raters within the same disciplinary group were 0.945 to 1.000 and were all observed to have P values smaller than .001.

Table 3

Distribution and Pearson correlation of subscale scores between raters.

Discussion

According to the results of the multiple validation assessments in this study, the Tw-FRHOP and the internationally adopted assessment tools of MFS, STRATIFY, and HIIFRM were moderately correlated, with a correlation coefficient in the range of 0.489 to 0.633, demonstrating that the Tw-FRHOP proposed in this study was a valid means of identifying fall risks among older inpatients and distinct from previously developed fall risk assessments. Multidisciplinary teams may proceed to provide customized prevention and intervention procedures for individual patients after implementing the Tw-FRHOP.

Among the raters from different disciplines, the consistency of scoring between the physicians and nurses was revealed to be extremely significant. Considering how the evaluation items used by physicians (Question 1–10) and nurses (Question 11–15) were both designed to determine the medical histories of patients in the FRHOP, inconsistent participant responses might not be retrieved when the assessments are conducted concurrently by these 2 raters. To address this, the study assigned 2 assessments conducted by 2 separate raters to each patient at half-day and 3-day intervals; the response consistency among the participants was extremely low. The analysis results revealed that changes in conditions occurred more rapidly among patients at the initial stage of hospitalization, suggesting that the results might be affected by changing illness conditions or levels of consciousness. To manage this concern, a concurrent assessment was employed in the interrater reliability test in this study. In the interrater reliability test, the kappa statistic of Question 19 was 0.651; this item—“does the patient's clothing fit well (not too long or loose fitting)?”—was evaluated by the occupational therapists with 0 or 1 point, and only 1 dissimilar scoring was identified between the 2 raters. Furthermore, the scoring consistencies of other items besides Question 19 among raters and among the physiotherapists were all demonstrably excellent.

A previous study on inpatient falls in Taiwan revealed that the average age among patients who experienced in-hospital falls was 50.04 ± 27.6 years, 39.8% of these patients were over 65 years old, and 51.1% were consequently injured.[ Accordingly, the Tw-FRHOP adopted the official guideline of the Australian FRHOP that inpatients older than 50 years should be duly considered for fall risk assessments.

Recent studies have mostly used MFS, STRATIFY, and HIIFRM for the assessment of inpatient falls. In a review article, Lee et al[ investigated the efficacy of fall risk prevention among MFS, STRATIFY, HIIFRM, and other assessment measures and discovered that although the predictive value of an assessment tool may be satisfactory under a given condition, the evidence for the tool's efficacy remains insufficient overall. A meta-analysis conducted by Aranda-Gallardo et al[ revealed that MFS, STRATIFY, and HIIFRM possess adequate prediction efficacy for patients receiving acute hospital care, among which the diagnostic validity of STRATIFY was the highest. In addition, Walsh et al[ developed a Western health fall risk assessment (WHeFRA) by modifying the FRHOP for a validity and reliability assessment of acute inpatients, accordingly demonstrating that the efficacy of WHeFRA was comparable to that of STRATIFY.

A systematic, broad review of the literature led to consistent findings that multifactorial assessments and interventions can effectively reduce the risk of in-hospital falls by 20% to 30%.[ Despite disparities in assessment and intervention approaches, common emphases among these assessments were mobility, confusion, continence, and the need for toileting assistance, medication, and postural hypotension and syncope. According to the NICE guidelines, a comprehensive inpatient fall risk assessment should consider factors such as cognitive impairment, continence problems, fall history, footwear, health problems, medication, postural instability, mobility and balance problems, syncope syndrome, and visual impairment.[ In addition, because these risk factors were considered in the Tw-FRHOP, the model proposed in this study is thus demonstrated to be a feasible comprehensive fall risk assessment.

A multidisciplinary intervention program facilitated by unit-based staff may be more successful than the one facilitated by visiting specialists.[ Although fall prevention programs may reduce the medical costs resulting from inpatient falls, patients with higher fall risks should be carefully identified to further reduce costs.[ In addition to risk grading, the FRHOP enables the customization of intervention measures for hospitalized patients with medium and high fall risks and the application of a standard procedure for patients with lower fall risks.

Some limitations encountered in the course of this study must be acknowledged. First, we did not examine the predictive accuracy of Tw-FRHOP. In the hospital, inpatient fall prevention programs are universally implemented in every unit. The fall rate of a given unit reflects the effectiveness of the inpatient fall program only. For ethical reasons, we could not measure any falls that could not be prevented in the hospital. Therefore, we used 3 common fall risk assessment tools as references to examine the criterion-related validity. Second, the test–retest reliability was not assessed in this study, mainly because of the short length of stay and rapid change in the condition of acutely ill patients. Despite of the high interrater reliability, when applying the Tw-FRHOP, raters are suggested to first receive adequate training to achieve higher consistency levels. Third, because the purpose of this proposed measure was not risk assessment alone, the Tw-FRHOP was implemented in a single-unit ward only, with criterion-related validation conducted within 3 days of hospitalization; this was not accompanied by wide application in all hospital units or the recording of subsequent hospital stays and fall conditions among patients.

Conclusion

The Tw-FRHOP demonstrated satisfactory validity and reliability for multifactorial fall risk assessment in older inpatients. In future studies, the Tw-FRHOP can be employed by multidisciplinary teams to identify individual fall risk factors and develop targeted intervention plans to verify the effectiveness of team-based, customized fall prevention for patients at risks.