Using the Delphi Method to Identify Risk Factors Contributing to Adverse Events in Residential Aged Care Facilities.

PURPOSE: This study aimed to identify risk factors associated with adverse events in residential aged care facilities in China. PATIENTS AND METHODS: After compiling a list of risk factors for adverse events generated from in-depth interviews with managers of residential aged care facilities, a three-round Delphi method was used to reach consensus. The synthesized risk factors were presented on a Likert scale to the expert panelists three times to validate their responses.
RESULTS: The list identified 67 items as risk factors for adverse events, attached to four first-level indexes (ie, environmental facility, nursing staff, older adults' characteristics, and management factors). The experts' authority coefficient was 0.87. The positive coefficients were 82.76%, 91.67%, and 100%, and the coordination coefficients were 0.154, 0.297, and 0.313 in the first, second, and third rounds, respectively.
CONCLUSION: Using a Delphi method, this study established a consensus on risk factors contributing to adverse events and developed a risk assessment grade for use in future aged care practice and research. The resulting list is useful in prioritizing risk-reduction activities and assessing intervention or education strategies for preventing adverse events in residential aged care facilities. IMPACT: This study fills the gap in risk identification in the Chinese residential aged care system to ensure provision of best-practice care to this vulnerable population. Nursing staff and management factors at the top of the list are not only the most common causes of adverse events but also the core elements in creating a secure and error-free environment. This list was intended to support predictive and prevention-oriented decision-making by managers and nursing supervisors to reduce preventable adverse events.

Introduction

As in all countries with the highest proportions of older adults, China is facing a gradual weakening of home-based care, the traditional bedrock of its aged care system. Alongside this decline, residential aged care facilities (RACFs) are growing and playing a more important role in aged care systems. According to the data released by the Ministry of Civil Affairs in China, the number of RACFs and beds had reached 168,000 and over 7.27 million, respectively, by the end of 2018.1 RACFs—also called nursing homes, long-term or skilled-care facilities in China–vary greatly in size and scope of services, such as medical, social, behavioral, and spiritual care service, catering to residents with different kinds of chronic diseases and complex care needs.2–6 In recent years, abuse and neglect of and serious injuries in older adults as well as infectious disease outbreaks in RACF settings were exposed by the media; consequently, the care quality in RACFs has attracted much attention from the government and the nursing community.7–9 To guarantee the quality of care and safety of older adults in RACFs, the Chinese government enacted laws, rules, and national standards, such as the Law on Protection of the Rights and Interests of the Elderly,10 Rules for the Administration of Elderly Care Institution,11 and National Standard for Basic Specification of Service Quality for Senior Care Organization.12 The national standard sets minimum requirements for aged care service, mainly on management, environment and facilities, and staff requirements. These regulations play a positive role in monitoring the care process. However, a considerable implementation gap emerges as these regulations are written in broad terms, do not provide strict guidelines, and represent only the minimum rather than the optimal standards of quality in residential care.

Background

Care service in RACFs is a complex, dynamic process that requires greater regulatory oversight. It is generally known that the institutionalized elderly tend to be older, frailer, and more disabled with extensive needs for nursing care. The majority may also be on multiple medications and have functional disabilities, cognitive deficits, or visual impairment,13–16 which make them more vulnerable to various unfavorable events during their RACF stays. In addition to the health problems of the residents, at an organizational culture level, there are issues related to inadequate staffing, low levels of staff knowledge and competence, poor managerial leadership, inadequate equipment, and poorly coordinated, low-tech reporting and oversight systems.17–20 Moreover, the caregivers in RACFs have heavy workloads, are underpaid, under time pressure, and experience social stigmatization.21 The conditions in RACFs can lead to mistakes and negligence in care, putting residents at increased risk for adverse events. The Centers for Medicare & Medicaid Services (CMS) have defined adverse events as an untoward, undesirable, and usually unanticipated event that causes death or serious injury, or the risk thereof, and include medication errors, infectious diseases, falls, pressure ulcers, and other injuries to older adults in the caring process.22 Adverse events are a significant issue in RACF settings as they can lead to substantial morbidity and mortality, high rates of hospitalization, and additional health-care costs.23 One study reported that approximately 31.7% of residents in RACFs experienced one or more falls in a year in Xiamen, China; however, the data do not cover the many cases left unreported because of staffs’ fear of punishment.24 Additionally, in a large, national, retrospective study in the United States conducted by the Office of the Inspector General (OIG), they found that approximately 22% of medicare beneficiaries experienced adverse events while staying in skilled nursing homes, 59% of which were deemed preventable.25 In fact, a majority of adverse events are “preventable” and “ameliorable.” The international consensus that “to err is human” emphasizes the prevention of errors as a key safety measure in a health-care system.26 Each facility needs to adopt some measures to prevent adverse events based on its circumstances. Many studies have indicated that one of the most effective methods to reduce adverse events is identifying and reporting risk factors and establishing an alert system.27–30

The calls for identifying risk factors of unexpected incidents and developing preventive measures are compelling. The Agency for Healthcare Research and Quality (AHRQ) said that designing intelligent systems, protocols, or processes to decrease adverse events is not feasible if one does not first determine where and why errors are occurring.31 The causes of adverse events of older adults in a health-care setting have been explored in different studies, which have helped, to a certain extent, clarify the dimensions of risk management. For instance, based on the stakeholder theory, Zhang and Liu32 stated that the operational risks of RACFs mainly involve internal and external factors, such as whether there is enough staffing, whether managers have formulated comprehensive management standards and operating procedures, whether the facilities and equipment are repaired regularly and maintained, and older adults’ physical and psychological characteristics. A study in Sweden reported that there are a total of 693 possible contributing factors to adverse events in nursing homes. The most common are lack of competence, incomplete or scant documentation, teamwork failure, and inadequate communication.33 In addition, the quality system and measurement tools in RACFs are beneficial for exploring risk factors and offer reference to risk management. In the study by Courtney et al,34 quality assessment of residential aged care contains four care domains: resident health, personal care, resident lifestyle, and care environment. Guo et al35 developed service quality indicators for long-term care facilities through a two-round expert consultation. These were 3 first-level (ie, management system and facilities and equipment, basic service, elderly health management), 13 second-level, and 46 third-level items.

Understanding risk factors that contribute to adverse events is critical, as it can lead to interventions in care processes that improve the safety of the elderly. In China, occurrences of adverse events have been reported in some studies; however, most of these studies focused on nursing errors and adverse events in the hospitals. So far, few studies have explored the contributory factors within the context of RACFs. Furthermore, China has not established a structured and comprehensive risk identification system in RACFs. It is urgent to develop a list of potential risk factors of adverse events and create a validation tool to help nursing staff recognize harmful events. This study aimed to fill the gap in risk management in China’s residential care system for the elderly, and further, to ensure the provision of best-practice care to this vulnerable population.

Theoretical Framework

Risk management began to be studied after World War II. Since then, risk management theories and standards have been formulated gradually in developed countries. Risk management is defined in the ISO 31000:2009 as “coordinated activities to direct and control an organization with regard to risk”;36 risk management includes goal setting, risk identification, risk analysis, risk treatment, and risk monitoring. The Chinese Standard GB/T 24353-2009 “Risk Management Principles and Implementation Guidelines,” which adapted ISO 31000:2009 in accordance with China’s national conditions (Figure 1), aims to provide a framework and guiding principles for any organization regardless of size, activity, or sector.37 With the rise in the number of adverse events and accidents, risk management has become an indispensable part of operation and management in RACFs. At present, setting up an effective risk management program to avoid, reduce, and transfer risk and uncertainty has become an urgent issue within any scope and context of RACFs. This study focused on the first step of risk management (ie, risk identification) and used a Delphi method for the expert consultation. Future research should examine risk analysis, treatment, and monitoring.

Figure 1

Adaptation of a theoretical framework on risk management for Chinese organizations.

The Study

Aims

We aimed to 1) explore the risk factors that contribute to adverse events among older adults in RACFs, 2) confirm the weight of each item in the index system and develop a risk assessment grade, and 3) provide recommendations on how these factors could be managed to prevent future adverse events.

Design

A Delphi method is used to obtain the most reliable consensus among a panel of experts using sequential questionnaires and interpolation of controlled feedback. This method is used widely in exploring and establishing various index systems in nursing and health care.38,39 There is no general consensus on the number of representative samples or on sample size.40 Thangaratinam and Redman41 believed that sample size depends on research scope, available resources, and research design. The number of experts (n = 29) recruited in this study was within the scope of the recommended sample size (n = 15–30).42,43 Each expert panel member was required to have managerial experience and provide valuable insights into the occurrence of adverse events during care delivery. Furthermore, these participants were identified as individuals with specialized knowledge and a great interest in the studied topic and resided in different provinces of China to obtain a greater diversity in the outcome. A risk index system relating to adverse events in RACFs was established based on the three rounds of Delphi survey.

Methods

A three-round Delphi expert consultation was used to gather information on multiple risk factors associated with the presence of adverse events. This method was expected to offer a way to understand the occurrence of events rather than criticize the process or assign blame. Risk factors leading to adverse events in older adults were gathered using semi-structured interviews. The interviews were undertaken during the initial phase of the study. In this phase, 13 RACF managers who had at least 5 years of professional experience in supervision and management participated in the face-to-face interviews. Each interview was audio-taped using a recording pen and a mobile phone after obtaining the permission of the participants. The data collected in the interviews were transcribed verbatim and analyzed using Colaizzi’s seven steps. Two experienced RACF managers, who did not participate in the interviews, took responsibility for checking the data and confirming the findings. In addition, 3 overarching themes were identified—“nursing staff,” “older adults’ characteristics,” and “management factors”—under which 8 sub-themes were determined. Subsequently, the synthesized themes were presented in a 5-point Likert scale format to the expert panel. The initial questionnaire comprised 3 first-level, 8 second-level, and 55 third-level items. The panel members were asked to confirm or respond to items listed in the questionnaire. In Round 1 of the Delphi survey, 29 experts used a Likert scale to rate how probable it is that each identified item will cause adverse events in RACFs. In Round 2, a revised questionnaire and feedback document were sent to the same experts who had returned the form in Round 1; 24 of them were asked to re-evaluate each factor while considering their peers’ responses in the feedback document. In Round 3, the updated questionnaires were revised based on the feedback in Round 2 and e-mailed to the same experts who had responded in Round 2; 22 of them assessed the representativeness and clarity of these items and then compared the importance weight of each item to the corresponding theme. In the Delphi survey, the filter criteria of all items were set at a mean score of <4 or a coefficient of variation (CV) of >20%.

Participants

Experts selected for the Delphi survey were either quality assurance administrators or risk managers. Experts should have managerial experience and systematic perceptions of the adverse events occurring in the RACFs. In Kuusi’s44 opinion, panelists should represent the research issue from different perspectives that can develop relevant arguments and expose underlying reasons on a specific subject. Thus, we also chose participants who were either nurses delivering care to older adults or professors conducting research on older adults’ care quality and safety. In determining the panel of experts, a purposive sample of 29 participants, comprising 16 quality assurance administrators or risk managers in RACFs, 6 conveniently selected nurses in geriatric wards, and 7 conveniently selected professors in universities, were initially surveyed.

The 29 experts selected for the Delphi survey had sound theoretical knowledge and excellent practical skills in aged care safety, quality assurance, and risk management. They worked in different areas: Beijing (3), Chongqing (4), Changsha (14), Nanning (5), and Taiwan (3). These experts a) had an intermediate technical title or higher, b) had a bachelor’s degree or higher, c) had over 10 years of experience in the profession, and d) were familiar with the Delphi method and volunteered for this survey. These criteria for selecting experts ensured that all participants had substantial experience and comprehensive insight, thereby rendering them suitable for making accurate judgments. The experts worked anonymously and independently. There were no exclusions to participation.

Data Collection

All the questionnaires were sent to the experts via email. The first round took place from December 15, 2015, to January 17, 2016, and the two subsequent rounds were in February and March 2016, respectively. To maximize the return rate, a phone message was sent to all participants two days after mailing, and a follow-up e-mail was sent a week later.

In the first e-Delphi round, all experts were asked to answer questions on general characteristics, their level of familiarity with the field (Cs), and specific familiarity criteria (Ca). Each expert received an explanation of the objectives of the study and the specific instructions for the Delphi method (confidentiality and anonymity were guaranteed). Simultaneously, each expert received the list of factors identified in the semi-structured interview. Using a 5-point Likert scale ranging from 1 (“not important at all”) to 5 (“extremely important”), experts were asked to rate the weight of each item. These results were coded, and items were assessed using mean, standard deviation, and CV to determine which of the items met the threshold for retention. After rating each item, experts could propose recommendations or suggestions for each item in a provided column. Further, experts were asked to add any additional items that may contribute to the occurrence of adverse events in older adults in RACFs that were not to be listed in the tables.

In the second round, experts were presented with the variables identified in the previous round. All items with a mean score of <4 or CV of >20% were removed. By integrating expert responses, a systematic literature review, and group discussion, we modified and added some items. The Chinese version of the questionnaire was resubmitted to the same experts as in the previous round. These experts used the same rating method as described for the initial consultation but considering the scores and comments given by their anonymous peers in the first round. Each expert received a) the revised questionnaire and b) the feedback document, including the comments of the panelists and results of the statistical analysis (mean, minimum, maximum, standard deviation, and CV) in the previous round while preserving the anonymity of experts’ responses. This procedure is a crucial part of the Delphi technology specifically in determining consensus and stability in the answers.

The third round involved an evaluation of all the information provided by the experts and the updated information, which were revised in the second round. Experts were asked to reassess each item just as in the previous rounds. However, in addition to rating the importance of each item, they were asked to clarify any redundancy and the syntax of each statement or issues regarding comprehension. They were also required to give a score for each comparison using Saaty’s scale to reflect the importance of the weight of the first-level items. The weights of second- and third-level items were determined by the mean importance score of each item. A pairwise comparison matrix was developed to calculate weights, which involved the relative significance among the criteria in the hierarchy. The normalized weights and synthesis of the results of all items were calculated using yaahp v10.5 after obtaining a series of judgment matrices for the items.

Ethical Considerations

The study was part of a larger project, which was approved by the Ethics Review Committee of Affiliated Hospital of Xiangnan University in Chenzhou city, Hunan Province. Participation was voluntary, anonymous, and confidential. Participants were made aware of their right to withdraw from the research at any time for any reason. Full disclosure of the study’s purpose and procedures was given to participants before they responded to the questionnaire. All experts received a copy of the consent form by e-mail and were asked to respond with a return e-mail indicating their willingness to participate in the study. All questionnaires and consent forms were retained in a secure location for at least three years after the completion of the study. Moreover, a financial compensation of RMB 400 was paid to each expert at the end of the third round of the Delphi survey.

Data Analysis

Data analysis of the Delphi survey involved statistical methods and content analysis. The latter was applied to establish a preliminary listing of essential factors for the index system. IBM SPSS Version 25 was used for all quantitative analysis. Descriptive statistics were used to describe each item, including mean, standard deviation, and CV; the reliability and validity of the Delphi method were tested by expert opinion consensus and calculating the positive, authority, and coordination coefficients. The importance weights of these items were calculated using Analytic Hierarchy Process (AHP) method, which was developed by Thomas L. Saaty in the 1970s.45 This method has been studied extensively and used commonly to determine index weight. Saaty’s46 fundamental 9-point scale was adopted to determine the importance of each item at all levels. Consistency Rate (CR) was used to verify whether the matrix is consistent. To enhance rigor, the following strategies were adopted in this study:47 (a) preparing the draft through semi-structured interviews and a systematic review of the literature, (b) selecting the representative and professional groups based on their theoretical knowledge and practical skills, (c) providing a clear explanation of the goals of the project and special instructions for experts, (d) redistributing the feedback document to the panelists in the iterative process, and (e) conducting quantitative analysis to determine the reliability and validity of the Delphi technique.

Results

Participants’ General Characteristics

From the 29 experts who were selected for consultation, 24 questionnaires were collected in the first round. They were from 12 RACFs and 6 nursing universities in 5 provinces. They all had considerable attainment and experience in the field of aged care and risk management. The youngest was 32 years old and the oldest was 60 years old. The average age (± standard deviation) was 43.88 ± 8.46 years. The shortest and longest working experience was 10 years and 35 years, respectively. The average working experience was 22.04 ± 8.72 years. Table 1 displays participants’ general characteristics.

Table 1

Participants’ General Characteristics (N = 24)

Variables	n (f)	Percentage (%)
Age (years)
30–39	9	37.50
40–49	7	29.17
≥50	8	33.33
Gender
Male	4	16.67
Female	20	83.33
Years of professional experience
10–19	11	45.83
20–29	5	20.83
≥30	8	33.33
Educational attainment
Undergraduate	14	58.33
Master’s	6	25.00
Doctorate	4	16.67
Profession
Nursing	18	75.00
Other profession (social work, human resources, medicine)	6	25.00
Title
Intermediate	11	45.83
Deputy senior staff	8	33.33
Senior staff	5	20.83
Graduate tutor
Yes	8	33.33
No	16	66.67

Reliability of the Delphi Method

Positive Coefficients

The positive coefficient is an important foundation of expert consultation that reflects the enthusiasm and cooperation of experts in the research. It usually refers to the recovery rate of the questionnaire, which is calculated as the ratio of experts participating in the consultation to the total number of experts. A response rate of 70% or above indicates high positivity among experts.48 The recovery rates for the three rounds of consultation were 82.76%, 91.67%, and 100%, respectively, with an effective recovery rate of 82.76%, 91.67%, and 95.45% from each respective round (see Table 2). In the first round, 17 experts (70.83%) provided 76 comments. In the second round, 10 experts (45.45%) wrote 29 suggestions. In the third round, 8 experts (36.36%) contributed 17 amendments.

Table 2

Three Rounds of Expert Positive Coefficients

Round	Questionnaires Issued	Questionnaires Retrieved	Return Ratio (%)	Number of Effective Questionnaire	Effective Return Ratio (%)
First	29	24	82.76	24	82.76
Second	24	22	91.67	22	91.67
Third	22	22	100	21	95.45

Expert Authority Coefficient

The expert authority coefficient is a key factor used to judge the validity of consultation results. An authority coefficient (Cr) is defined as the average of the sum of an expert’s level of familiarity with the field (Cs) and a familiarity criterion (Ca) or Cr = (Cs + Ca)/2. Familiarity was divided into five degrees, ranging from 0.2 to 1.0, indicating the lowest and highest familiarity with this field. Criteria, including theoretical analysis (0.3, 0.2, 0.1), practical experience (0.45, 0.35, 0.2), literature at home and abroad (0.2, 0.15, 0.1), and subjective judgment (0.05, 0.05, 0.05), were categorized into more, medium, and less. In this study, the authority coefficient was 0.87 with 0.88 familiarity and 0.86 criterion.

Coordination Level of Experts’ Opinions

A coordination coefficient (Kendall’s Concordance Coefficient, ω) is generally used to reflect the coordination level of experts’ opinions, ranging from 0 to 1. A higher ω value indicates better coordination among expert opinions.49 The χ2 test is used to evaluate the significance of the coordination coefficient. A p-value of <0.05 indicates that expert opinion has good coordination and the result is reliable. The overall coordination coefficients of the first, second, and third rounds were 0.154, 0.297, and 0.313, respectively (Table 3).

Table 3

Coordination Level of Experts’ Opinions

Rounds	Kendall’s (ω)	χ2 value	df	p-value
First	0.154	240.289	65	0.000
Second	0.297	534.708	106	0.000
Third	0.313	605.004	92	0.000

Index Screening Results

Round 1

In this round, 4 third-level items met the predefined cut off value (a mean score of <4 or CV >20%), and the remaining items were identified as important. Seven experts proposed to add “environmental facility” as a first-level item and 3 participants put forward suggestions on the naming and connotation of the first-level items; for example, they believed nursing staff should be defined as a person trained to provide a wide range of health and personal services for the elderly in RACFs. As for the second-level items, “social support” and “spiritual and cultural factors” were among the 5 items suggested to be added to the index system by 9 participants. Two experts suggested that some items’ names and connotations be modified. Moreover, experts recommended that “communication and coordination ability” be downgraded to third-level items. In fact, the third-level items obtained several suggestions regarding naming, connotation, and the addition of items from 12 experts. Based on discussions with the research team and after considering the experts’ comments, the following consensus was reached: add 1 first-level item, 5 second-level items, and 46 third-level items; delete 1 second-level item and 10 third-level items (including 4 items meeting the cut off value, 2 items upgrading to the second-level, and 4 items under the deleted second-level items of “communication and coordination ability”); and adjust the connotations of some items. Finally, systematic analyses resulted in the identification of 4 first-level items, 12 second-level items, and 91 third-level items. These items composed the Round 2 questionnaire. All adjustments in this round were provided to participants.

Round 2

Based on the established criteria, 1 second-level item and 11 third-level items were removed from the subsequent round. The panelists reached a consensus on the first-level items, and only 1 participant suggested that “medical care” and “health guidance” be taken as first-level items. This opinion was not adopted as it overlapped with the retained items. Five experts put forward suggestions on the connotation of “environment,” “facilities and equipment,” and “organization and implementation” and proposed to increase “prevention plans” as a second-level item. In terms of the naming, connotation, and deletion and addition of third-level items, 5 participants submitted their opinions. The advice of the experts was synthesized and discussed after the statistical analysis. Consequently, 1 second-level item and 16 third-level items (including 11 items meeting the predefined criteria, 4 items merging with other items, and 1 item upgrading to a second-level item) were removed; 1 second-level item and 6 third-level items were inserted; and the connotation of the corresponding items was modified. Ultimately, 4 first-level, 12 second-level, and 81 third-level items composed the Round 3 questionnaire sent to participants for expert consultation.

Round 3

The statistical analysis in this round showed that 2 second-level items and 10 third-level items did not reach the threshold of consensus, and the other items met an appropriate level for importance. The first-level and second-level items did not get any opinions from the participants. However, some experts provided professional opinions on the third-level items, the risk assessment standards and grade. Based on the insights gained from the questionnaires and group discussion, 14 third-level items (10 third-level items reaching the cut off value, 2 items merging with other items, and 2 items under the deleted second-level of “social support”) were removed from the final index system. Eventually, the final list of risk factors contributing to adverse events in RACFs contained 4 first-level, 10 second-level, and 67 third-level items. According to the AHP, the index system on risk factors has 4 layers: total risk level, first-level item, second-level item, and third-level item. In our study, we formed 15 judgment matrixes. All consistency test results of the judgment matrix were <0.10, indicating that the degrees of inconsistency of the matrix were within the acceptable range. Table 4 provides the final list of the index system on risk factors and the weight of each item.

Table 4

Weight Assignment of the Index System on Risk Factors

First-Level Indicators (Weight)	Second-Level Indicators (Weight)	Third-Level Indicators (Weight)
1. Environmental facility (0.1610)	1.1 Environment (0.2500)	1.1.1 Architectural planning (0.2748)
		1.1.2 Physical conditions (0.1981)
		1.1.3 Layout design (0.3873)
		1.1.4 Identification system (0.1397)
	1.2 Facilities and equipment (0.7500)	1.2.1 Catering (0.2644)
		1.2.2 Furniture (0.0922)
		1.2.3 Auxiliary equipment (0.1781)
		1.2.4 Sports and fitness (0.0686)
		1.2.5 Medical and nursing (0.1322)
		1.2.6 Safety facilities (0.2644)
2. Nursing staff (0.3950)	2.1 Basic quality (0.2500)	2.1.1 Professional ethics (0.1623)
		2.1.2 General knowledge of regulations (0.0787)
		2.1.3 Safety consciousness (0.2309)
		2.1.4 Code of conduct (0.1145)
		2.1.5 Emergency capacity (0.1900)
		2.1.6 Communication and coordination ability (0.0787)
		2.1.7 Learning ability (0.0304)
		2.1.8 Physical quality (0.0672)
		2.1.9 Psychological quality (0.0473)
	2.2 Professional nursing skills (0.7500)	2.2.1 Basic knowledge of geriatric nursing (0.1139)
		2.2.2 Occupational safety and health knowledge (0.0599)
		2.2.3 Basic nursing skills (0.1971)
		2.2.4 First aid skills (0.1441)
		2.2.5 Security guard (0.1971)
		2.2.6 Disease nursing (0.0868)
		2.2.7 Psychological nursing (0.0452)
		2.2.8 Rehabilitation nursing (0.0346)
		2.2.9 Health education (0.0346)
		2.2.10 Nursing records (0.0868)
3. Older adults’ characteristics (0.1947)	3.1 Physiological factors (0.7500)	3.1.1 Age (0.0371)
		3.1.2 Disease (0.0860)
		3.1.3 Medication (0.1520)
		3.1.4 Ability in daily activities (0.2240)
		3.1.5 Language expression (0.0371)
		3.1.6 Sensory ability (0.0519)
		3.1.7 Balance ability (0.1092)
		3.1.8 Cognitive function (0.1092)
		3.1.9 Swallowing function (0.0762)
		3.1.10 Nutrition (0.0655)
		3.1.11 Skin (0.0519)
	3.2 Psychosocial factors (0.2500)	3.2.1 Character (0.1951)
		3.2.2 Emotion (0.4915)
		3.2.3 Social support (0.1951)
		3.2.4 Income (0.1184)
4. Management factors (0.2493)	4.1 Rules and regulations (0.3092)	4.1.1 Responsibility and authority (0.0810)
		4.1.2 Workflow (0.1343)
		4.1.3 Technical specifications (0.2601)
		4.1.4 Security check (0.2024)
		4.1.5 Training and assessment (0.1343)
		4.1.6 Accident reporting and handling system (0.1343)
		4.1.7 Management system of nursing documentation (0.0538)
	4.2 Organization and implementation (0.4351)	4.2.1 Staffing (0.1171)
		4.2.2 Certificates and Licenses for Nursing Staff (0.1171)
		4.2.3 Division of labor (0.0693)
		4.2.4 Pre-admission assessments (0.2341)
		4.2.5 Risk notification (0.1590)
		4.2.6 Functional partitions (0.0693)
		4.2.7 Quality control (0.2341)
	4.3 Emergency plan management (0.1501)	4.3.1 Emergency plan preparation (0.1817)
		4.3.2 Emergency plan review (0.0622)
		4.3.3 Emergency plan implementation (0.1164)
		4.3.4 Publicity and education (0.0794)
		4.3.5 Emergency drills (0.3337)
		4.3.6 Monitoring and warning (0.2266)
	4.4 Support system (0.1056)	4.4.1 Medical support (0.5485)
		4.4.2 Logistical support (0.2106)
		4.4.3 Accident insurance (0.2409)

Risk Assessment Grade

Assessing the risk level inducing adverse events can provide powerful evidence for RACF managers to carry out risk management. RACF managers will become aware of the focal points of risk management and, thus, put forward rectification measures. Based on the quantitative index system on risk factors and guidance from experts’ opinions, the research group developed the assessment standards of risk level associated with adverse events using a 100-point scale, setting a minimum score of 25 and maximum score of 100. According to the scoring criteria, each three-level item is divided into four grades endowed with 25, 50, 75, and 100 points. The calculation applied was the scores of third-level items of environmental facilities and management factors multiplied by their respective weights plus the average scores of the third-level items of nursing staff and older adults characteristics (the mean score of their third-level items multiplied by their respective weights). Experts pointed out that creating a risk-free culture may be impossible as there are many frail older adults living in RACFs. Therefore, the risk-free grading was removed, and the risk level in RACFs was divided into four grades: extremely high risk, high risk, medium risk, and low risk (Table 5).

Table 5

Risk Assessment Grade

Risk Grading	Risk Assessment Score
Extremely high risk	<40
High risk	40–59
Moderate risk	60–79
Low risk	80 and above

Discussion

Adverse events occur frequently in China’s RACFs. This study aimed to identify risk factors associated with adverse events in residential aged care facilities and create a risk assessment grade. The items in this study were initially derived from a qualitative analysis of semi-structured interviews and finalized using the Delphi method through a three-round expert consultation. Delphi survey results have proven and satisfactory authority and reliability. First, the study chose appropriate panelists based on certain characteristics (eg, age, area, and profession) and who had full insight on the care-related adverse events in older adults. Second, the average authority coefficient (0.87) and questionnaire recovery rate (82.76%, 91.67%, and 100%) were within the acceptable boundaries (more than or equal to 0.70).50,51 Third, the overall Kendall coordination coefficient (from 0.154 to 0.313) confirmed that the expert opinions gradually reached a consensus. In addition, the weight distribution of the index system revealed that nursing staff and management factors were leading elements contributing to the occurrence of adverse events in RACFs, followed by older adults’ characteristics, and environmental factors within the facility.

Many studies have indicated that nursing staff play a decisive role in the care of older adults and have a great responsibility for ensuring each resident in RACFs receives safe and quality care.52,53 In this study, professional nursing ability had the highest combination weight among all 10 second-level items, comprising 10 third-level items. This is in accordance with previous studies that highlight the importance of staff competence as a key determinant of quality care and residents’ safety.54,55 As shown by a systematic review study by Spilsbury et al,56 a poor skills mix has been associated with poor quality care, such as care deficiencies, and urinary tract infections. Unfortunately, in China, many research results indicate that current competencies of nursing staff do not correspond with the tasks required within RACFs.57,58 In one survey, 88.48% of nursing professionals were nursing assistants, most were untrained middle-aged females with no formal qualifications from rural areas.59 Other research has suggested that nursing students, who are regarded as the main source of future aged care providers, are not willing to work in RACFs due to the inequality between workload and salary.60 Zhang61 also found that the staff in RACFs lacked sufficient competence to ensure safe resident care, especially in respect of first-aid care, medication care, and chronic disease care. Besides, the basic quality of nursing staff (eg, professional ethics, psychological condition, safety consciousness) can heavily influence residents’ mental state and the outcomes of health care. A survey study by Andela et al62 suggested that the personal characteristics of staff, for instance, emotional demands, burnout, and poor relationships with colleagues, may be predisposing factors for elder abuse in nursing homes. Hence, care facilities should have sufficient nursing staff with appropriate competencies and skills to provide care and related services to maintain the highest practicable well-being of each resident.

RACFs need a positive organizational culture to create the right conditions for safe, high-quality nursing care. In this study, we found that the management factor is an important element to improve the safety of resident care; this is similar to the study results of Lee and Chung63 and Lefosse et al.64 Management systems and programs can provide an effective framework for safety. Research on resident safety in RACFs highlight latent factors for adverse events in RACFs and seem to be related to the structure of the organization and procedures,65,66 that in turn predispose residents to the risk of harm and injury, especially in the case of frailer residents. In China, the aged care industry is booming in an under-regulated market as a result of an explosive growth in the number of people in the 65-plus group. At present, there are some defects in the management systems within RACFs.67,68 First, some institutions do not develop staff qualifications and service standards for aged care. Therefore, the employees fail to provide desirable individual care without detailed directions and effective regulation, increasing the occurrence of adverse events caused by inappropriate intervention or inadequate nursing care. Second, most RACFs place a one-sided emphasis on physical care and neglect the humanistic care (eg the spiritual and mental care); moreover, some facilities have a limited ability to provide emergency medical care, which directly affects residents’ safety and the outcomes for health care. Third, health information technology (HIT) has not been fully utilized in RACFs and few HIT-related reporting systems on adverse events are designed to collect information on adverse events (possible type, severity, and preventability) that occur during residents’ stay; hence, organizations undertake the risk of adverse events with limited ability to improve the quality of care. In recent years, the Quality Assurance & Performance Improvement (QAPI) program, developed by the Centers for Medicare and Medicaid Services (CMS)69 in the United States, has attracted attention in China. It emphasizes compiling resources and utilizing evidence-based tools and strategies to reduce adverse events by, for example, identifying, tracking, and investigating adverse events. This is of critical importance in improving the quality of care.

Results of this study suggested that older adults’ characteristics, which include physiological factors (weighting of 75%) and psychosocial factors (weighting of 25%), are the common dimensions that should be considered in the risk management. This viewpoint is supported by many studies, which have proved that the poor physical states and abnormal mental conditions of elderly were risk factors that predisposed older adults to adverse events (eg infections, suicide, abuse, and pressure ulcers). For example, RACF residents with functional impairment, multiple comorbidities, and invasive devices (eg feeding tubes, urinary catheters) are more susceptible to infections, as well as other mechanical complications.70 Furthermore, in a systematic review on residents in long-term care facilities, suicide was found to be associated with social isolation, depression, and functional impairment.71 Similarly, abuse in the RACF population has been linked to medical illness, mental health issues, and personality disorders.72 RACF residents with advanced dementia, urinary catheter usage, low body mass index, and anemia are also more prone to pressure ulcers.73 Although it is well known that older persons within RACFs have a high prevalence of physiological and psychosocial problems (eg, disability, polypharmacy, comorbidity, and cognitive impairment),74 there appears to be a lack of awareness of prevention of health-related risk in the RACF population. In round-the-clock care units, nursing staff find it difficult to provide adequate care to the elderly, which in turn leads to a crisis in the care of the aged. A cohort study in the Netherlands indicated that many patients at risk do not receive adequate preventive care.75 Resident participation and trust in the care is an important factor for ensuring the safety, but involvement opportunities are limited in the long-term care system and resident participation in medical care is generally very low.76 Hence, considerable and deserved attention should be focused on this vulnerable population, and greater involvement of older people themselves in adverse events prevention programs is expected to be encouraged in the process of risk management.

It is well recognized that the physical environment is important for the well-being of people. The current study’s results indicate that environmental facilities play a vital role in reducing adverse events in RACFs, although it has the lowest weight ranking in the first-level indexes. The panelists reached a consensus that RACFs should have a suitable, barrier-free, and safe environment that ensures individual protection and facilitates communication among older adults. These concerns could optimize their functional abilities and improve outcomes in the quality of care, such as less falls and better nutrition. The importance of the physical environment in maintaining residents’ independence, social interactions, and well-being has been well documented. A review study by Chaudhury et al77 found that an appropriate and responsive physical environment is beneficial for the development of cognitive abilities and functioning in the elderly. Furthermore, some studies claim that a high-quality care environment can support residents to maximize their full potential and attain a better quality of life, especially in people with dementia.78,79 The study of Woodbridge et al also mentioned that a homelike environment can avoid unnecessary harm by reducing agitation or diffusing behavior.80 It is worth noting that in the initial consultation, the environmental facility was not listed in the questionnaire as we believed that adverse events were mainly caused by human factors. However, in the first round, 7 experts proposed that environment and facilities should be involved in the first-level index as they serve as a foundation in RACFs’ system. Moreover, the British scholar Elwyn Edwards81 proposed that the cause of mistakes should be analyzed from four aspects: the center of the system (ie, human and other components of the system), software, hardware, and environment. Thus, the environmental facility index that includes layout, furnishings, objects, design, etc., is in the final list. This study emphasizes the importance of creating an optimal physical environment and fixing or eliminating environmental hazards, such as clutter, defective or dangerous equipment (eg, nursing home beds, mobility equipment, and furniture).

For RACFs with a high incidence of adverse events and providing poor quality of care, it is important to focus on strategies to prevent the occurrence of adverse events. Table 4 shows the major factors that need to be addressed to gain a better care outcome in actions to prevent adverse events. The index system is a draft checklist that is available to RACFs providers and can be used as a risk management tool and to assist researchers in studying aged care issues. These items not only explain the causes of adverse events or errors in the care delivery but also draw attention to RACF practices around high-risk and problem-prone areas (eg, residents with dementia, inadequate staffing, and incompetent staff). Using this index system is an important step to creating a safe environment. However, the development of this index system requires further research for practical application. Future studies could focus on developing a handbook for the prevention of risk factors that could contribute to the adverse events; by following the step-by-step instructions RACF caregivers could prevent adverse events. In practice, support strategies, such as providing implementation guidance or consulting mechanism, should be established for those RACFs that fail to meet the safety requirements.

Limitations

There are several limitations in this study. First, because of time and financial restrictions, a large sample was not used to verify the quality (reliability, validity, and differentiation) of the index system. The index system should be formulated to the tool and used to guide the practice in RACFs. Second, the index system contained many items, which will lead to much time to assess the risk level. It should be simplified and adjusted to be a borderless utility for subsequent research. Third, the Delphi method has inherent limitations. For example, because of the lack of thought communication, experts’ meanings may be misrepresented by other panelists. It is also easy to ignore minority opinions in the process of expert consultation. New ideas outside the mainstream are likely to be lost because of the need for confirmation from other panelists. Finally, the subjective judgment of researchers may affect the evaluation of questions and the classification of answers. However, the group discussion provides a reliable method to reduce this potential problem.

Conclusion

Experts from different areas of the country identified 67 risk factors in four categories: environmental facilities, nursing staff, older adults’ characteristics, and management factors. The index system emphasizes the priorities of risk management and high standards of the care delivery. The results of this study can improve the understanding of risk factors contributing to adverse events and serve as a reference and guideline for RACF caregivers. The managers and nursing supervisors can use this list to monitor and improve quality of care in RACF settings. For risk management theory, this study is crucial to the development of risk identification that calls for practical application and progressive research proceeding on the platform the Delphi survey created in this study.