Workplace Predictors of Quality and Safe Patient Care Delivery Among Nurses Using Machine Learning Techniques.

BACKGROUND: Working in unhealthy environments is associated with negative nurse and patient outcomes. Previous body of evidence in this area is limited as it investigated only a few factors within nurses' workplaces.
PURPOSE: The purpose of this study was to identify the most important workplace factors predicting nurses' provision of quality and safe patient care using a 13-factor measure of workplace conditions.
METHODS: A cross-sectional correlational survey study involving 4029 direct care nurses in British Columbia was conducted using random forest data analytics methods.
RESULTS: Nurses' reports of healthier workplaces, particularly workload management, psychological protection, physical safety and engagement, were associated with higher ratings of quality and safe patient care.
CONCLUSION: These workplace conditions are perceived to impact patient care through influencing nurses' mental health. To ensure a high standard of patient care, data-driven policies and interventions promoting overall nurse mental health and well-being are urgently required.

Health care workers including nurses are exposed to many workplace risk factors such as human suffering and death, workplace violence, heavy workloads, and inadequate staffing. Existing research has found that unhealthy workplace conditions not only deteriorate nurse mental health but also compromise nurses' ability to deliver effective care to patients and their families.1 This body of evidence, however, was limited as it investigated only a few factors within nurses' workplaces. The purpose of this study was to identify the most important workplace conditions predicting nurses' provision of high-quality and safe patient care using a theoretically sound and comprehensive measure of workplace conditions that was developed on the basis of the National Standard for Psychological Health and Safety in the Workplace (ie, the Standard) by the Mental Health Commission of Canada.2 This study is both timely and relevant in light of the rising mental health problems among nursing providers amidst the battle against a global pandemic fought by nurses and other health care providers.3

LITERATURE REVIEW

Previous research has linked unhealthy workplace conditions to poor-quality and unsafe patient care.1,4,5 In nursing, the majority of workplace studies have focused on 3 distinct bodies of evidence: structural empowerment,4 Magnet hospitals,1 and the area of worklife.5 Structural empowerment describes healthy workplaces as work environments that promote workers' access to 6 empowering structures: information, resources, opportunities, supports, and formal and informal channels of power.4 Magnet hospitals are health care organizations that do not experience nurse recruitment and retention problems due to 5 workplace attributes: presence of effective leadership, collegial nurse-physician relations, opportunities for nurse participation, adequacy of staffing and resources, and a nursing (rather than a biomedical) model of care.1 The areas of worklife model denotes a mismatch between individual nurses and 6 workplace conditions that result in nurse burnout, which ultimately has been linked to patient adverse events; these workplace conditions include workload, control, reward, community, fairness, and values.5 Despite different ways of conceptualizing key workplace conditions, all 3 bodies of literature found that healthier workplaces are associated with better patient care and positive patient outcomes.1,4,5

As opposed to highlighting only a few workplace conditions, the Standard identifies 13 overarching workplace conditions that promote workers' mental health as well as prevent mental health injury in the workplace.6,7 Founded upon empirical and theoretical evidence, these conditions were systematically identified using a grounded theory approach, which involved a comprehensive literature review and consultation with subject matter experts for the intent of optimizing mental health in the workplace. A 13-factor measure, the Guarding Minds at Work (GMW) survey, was developed to measure the Standard's workplace conditions critical to ensuring employee mental health.2,8 The GMW survey was recently validated among 3077 direct care nurses working in acute care settings in British Columbia (BC). The authors found that prepandemic, more than half of the nurse respondents were concerned about 9 of the 13 workplace factors assessed. The most concerning were psychological protection and the workload management conditions of nurses' work environments.2 Furthermore, another recent study of the Standard has found that among the 13 workplace factors, workload management, worklife balance, and psychological protection were the most important determinants of nurse adverse mental health outcomes including depression, anxiety, and posttraumatic stress disorder (PTSD).9

In recognition of the rising mental injury among nurses and other health care providers, BC health authorities were mandated in 2016 to implement the Standard to protect the mental health of their nursing workers and other health care providers in the workplace.2 The implementation of the Standard begins with a comprehensive, baseline assessment of workplace conditions most important to employee mental health using the GMW survey.2 The current study examined a key research question: Which of the Standard's 13 workplace conditions most strongly predict nurses' ability to deliver high-quality and safe patient care?

METHODS

A cross-sectional correlational survey study was conducted in collaboration between university nursing researchers and the British Columbia Nurses' Union (BCNU) representing nearly 48000 nurses in the province. In December 2019, an email invite with the survey link was distributed by the BCNU to its nurse members asking them to participate in the survey. Nurses were informed that participation was completely voluntary and that submitting the survey would imply informed consent. A series of strategies were used to promote participation: a 2-month data collection period, weekly email reminders, study advertisement on social media and other platforms, as well as a raffle draw for 2 Apple Watches. A total of 5512 surveys were returned, yielding an estimated 12% response rate. After excluding nonpracticing nurses (eg, on maternity or disability leave) and nurses in non–direct care roles (eg, leaders), a final sample size of 4029 participants was included in this study. Ethics approval was obtained from the University Behavioral Research Ethics Board (approval no. H18-02724).

Measures

Outcomes

The 3 quality and safety outcomes in this study came from a 12-country survey study of nurses' work environments, the RN4CAST.10 Quality of nursing care was measured by obtaining the mean of 2 questions that asked participants to rate the general and last shift quality of nursing care delivered in their primary workplace, with response options ranging from “poor” (1) to “excellent” (4). Patient safety was measured using a single question that asked participants to give their workplace a grade on patient safety ranging from “failing” (1) to “excellent” (5). Finally, as a general indicator of quality and safety, participants were also asked to identify the likelihood that they would recommend their workplace to (a) family and friends for care and (b) colleagues for work with response options ranging from “definitely no” (1) to “definitely yes” (4). The mean scores of the 2 recommendation questions were included in our analyses.

Predictors

Workplace conditions were measured using the 13-factor GMW survey,2 which comprised 65 items. Each factor consists of 5 statements about a specific workplace condition, and participants were asked to indicate their level of agreement or disagreement with each statement on a 4-point scale ranging from “strongly disagree” (1) to “strongly agree” (4). The internal structure of the measure was previously evaluated among BC nurses and yielded a 13-factor structure.2 In this study, because we were interested in the predictive power of each work environment factor, composite factor scores were obtained using confirmatory factor analysis,11,12 with higher factor scores indicating healthier workplace conditions.

Controls

Demographic variables included age, gender (female, male), years of nursing experience, health care sector (acute, care, community care, long-term care), and geographical region (urban, suburban, rural). These demographic variables were included as control variables in this study. The health care sector and geographic region were dummy-coded.

Data analysis

Composite predictors were evaluated for internal consistency. A coefficient Ω > 0.8 and a coefficient 0.8 > Ω > 0.7 indicated good and acceptable internal consistency, respectively.13 The key method of data analysis was random forest (RF) analysis, which is a machine learning algorithm14 that nonlinearly regressed each of the 3 quality and safety outcomes on all of the 13 GMW factors after taking into account the impact of control variables (ie, age, experience, health care sector, and geographical region). RF analysis is a more appropriate method than conventional regression analysis for evaluating the relative importance of large numbers of predictors, as is the case in this study, due to yielding more stable results.15,16

Consistent with other RF studies,17 a 10-fold cross-validation with a 7:3 data set split for training and testing, respectively, was applied. Subsequently, the training and testing sets were used to evaluate the model performance through the root mean square error (RMSE). A higher RMSE for the testing set compared with the training set would indicate lack of overfit and the appropriate use of RF analysis for prediction purposes. The average level of the decline in prediction accuracy after a specific predictor was excluded was obtained to determine predictors' level of importance or their importance score. A smaller decline in prediction accuracy would indicate that the excluded predictor is low ranked in terms of importance.17 In addition, the proportion of the variance in each outcome variable explained by the model predictors was obtained using the R2. Finally, partial correlations were used to determine the direction of the relationship between GMW factors and quality and safety outcomes while taking into account the impact of control variables. The R package “caret” was used for data analysis.18

RESULTS

Most participants were female (n = 3676; 91%) and worked in the acute care sector (76%), compared with community (16%) and long-term care (7%) sectors, and urban settings (68%), compared with suburban (18%) and rural (19%) areas. On average, participants were 40 years old (SD = 12 years) and had 12 years of nursing experience (SD = 7 years).

The Table in Supplemental Digital Content (available at: http://links.lww.com/JNCQ/A909) provides descriptive statistics and/or reliability indices for GMW factors and quality and safety outcomes. McDonald's Ω ranged between 0.74 and 0.89, indicating good internal consistency for 11 workplace factors and acceptable internal consistency for 2 workplace factors (ie, job fit and growth development).

The Table shows the results of the RF analysis identifying the relative ranking of the 13 workplace conditions predicting quality and safety of patient care delivery. Overall, the model accounted for 16% to 44% of the variance across 3 quality and safety outcomes. For all of the 3 models, the RMSE for the test data set was greater than the train data set, suggesting lack of overfit and the appropriateness of RF analysis for prediction purposes. More specifically, workload management (importance score = 56.15), physical safety (importance score = 35.58), and psychological protection (importance score = 27.78) in the workplace were ranked as the most important predictors of patient safety grade. For quality of nursing care, the most important predictors were workplace engagement (importance score = 27.96), workload management (importance score = 26.76), psychological protection (importance score = 15.40), and physical safety (importance score = 12.11). In addition, whether nurses would recommend their workplace to family and friends and/or colleagues was predicted by workload management (importance score = 39.46, most important), psychological protection (importance score = 34.14), and workplace engagement (importance score = 34.14). Finally, the direction of associations between predictors and each outcome was examined. All predictors were positively associated with quality and safety outcomes. To be more specific, nurses with higher/healthier ratings of workplace conditions were more likely to rate patient safety grade and quality of nursing care in their workplace higher and also more likely to recommend their workplace to others for care and work.

DISCUSSION

To our knowledge, this is the first study to evaluate nurses' workplace conditions predicting quality and safety of patient care using the comprehensive and theoretically founded GMW. Our findings showed that effective workload management, psychological protection, physical safety, and engagement opportunities are among the most important workplace conditions influencing nurses' ability to deliver quality and safe patient care.

As an important predictor of all quality and safety outcomes, effective workload management reflects a workplace with adequate staffing and resources where assigned tasks and responsibilities can be accomplished successfully within the time available.8 This GMW factor is present within all 3 nursing work environment bodies of evidence and is a known determinant of nurse and patient outcomes.1,4,5,19 Consistent with our findings, previous research suggests that when nurses do not have the support or resources to complete their nursing tasks and responsibilities, patient care suffers.1,5 More specifically, MacPhee and colleagues20 have found that heavy workloads at multiple levels (unit level, job level, and task level) were associated with negative nurse and patient outcomes including medication errors and falls.

The second GMW factor associated with the 3 quality and safety outcomes in this study is psychological protection, which reflects workplaces where employees' psychological safety is “protected” by preventing unnecessary stress, such as workplace violence and discrimination.8 Although this workplace factor is absent from the 3 models of nursing work environment, an extensive body of nursing literature has linked workplace violence exposure to negative nurse21 and patient outcomes including medication errors and missed care.22 These findings suggest that patient care suffers in workplaces that fail to protect the health and safety of their nursing employees. Previous research has also linked adverse nurse mental health outcomes to poor performance and negative patient outcomes.23–25

Physical safety is another important predictor of quality and safety outcomes in this study, particularly nurses' ratings of their unit's safety grade and quality of nursing care. In the GMW model, physical safety is described as eliminating physical hazards in the workplace through offering adequate access to equipment and supplies, training, and incident reviews. This factor is best reflected in the structural empowerment theory, particularly access to information, resources, and professional development opportunities are required workplace factors for effective performance.4,19 Previous research has linked these empowering structures to better job performance among nurses.26

Furthermore, in this study, engagement was found to be an important predictor of quality of nursing care and the likelihood of recommending the workplace to others for care and/or work. Described as a workplace where employees have a sense of connection and commitment to their colleagues and the organization,8 engagement is reflected only in the areas of worklife model.5 Previous worklife model research showed that a lack or limited sense of community among nurses was related to negative nurse outcomes such as burnout27,28 and turnover intent,28 which, in turn, compromise nurses' ability to deliver high-quality and safe patient care.29,30

Further to enlightening our theoretical understanding of healthy work environments, this study offers a direction for practice, policy, and research that best support nurses' provision of quality and safe patient care. The study findings suggest that there is a critical need for interventions addressing key aspects of nurses' work environments. In the context of a stressful, demanding, and highly infectious COVID-19 pandemic, implementing workload management tools,31,32 adopting workplace violence prevention strategies,32 employing team-building exercises,33 and offering sufficient access to high-quality personal protective equipment34 are potential strategies that would improve the 4 workplace factors most important to quality and safe patient care.

Given that the GMW model was developed with the intent of optimizing workers' mental health and safety,6–8 we believe nurses' workplace conditions impact patient care through influencing nurse mental health. Poor nurse mental health has been repeatedly identified as an antecedent of negative patient and organizational outcomes.29 More specifically, patient dissatisfaction levels are high in workplaces with highly dissatisfied or burned-out nurses.35 This is further evidenced by previous research with the standard that consistently found workload management, psychological protection, engagement, and physical safety as important determinants of nurse anxiety, depression, PTSD, burnout, and life satisfaction.9 In light of these findings, future research should evaluate the mediating effect of nurse mental health on the relationship between GMW factors and quality and safe care provision.

To better protect nurse mental health in the workplace and subsequently enable a high standard of patient care, occupational health and safety policies must be revisited and refined according to nurses' needs. In BC, nurses have access to confidential counseling services by their employer (Employer and Family Assistance Program), but access is highly inadequate, limited to only 3 to 4 visits per mental disorder.36 This is especially problematic in light of recent research that found concerning rates of various mental health problems among nurses in Canada and provincially.36,37 Furthermore, while nurses were finally included in the presumptive legislation in 2019, giving them access to resources for treating mental illnesses such as PTSD, the legislation does not cover the most prevalent nurse mental health problems such as burnout.36

While these policies enable supports and resources for nurses with negative mental health experiences, they are not preventive in nature. We believe preventing mental health injury requires investing in workplace conditions most important to nurse mental health and care delivery. To develop preventive workplace interventions that are evidence-based, health leaders must conduct routine and confidential assessments of nurses' mental health and workplace conditions using validated measures. For example, this study used empirical evidence from BC nurses to show specific workplace conditions were important predictors of nurse quality and safe patient care delivery. Obtaining this information at a unit or organizational level would be critical to informing workplace health and safety interventions that are responsive to the needs of nursing staff and the unique context of their workplace.

Strengths and limitations

To examine the relative importance of 13 different workplace conditions in predicting quality and safe patient care delivery, this study used a machine learning technique, RF analysis. The use of RF analysis in this study provided an opportunity to introduce a contemporary method of data analysis, from other disciplines, to nursing research. Although RF methods do not provide any information related to clinical significance, they are a more appropriate method than linear regression for analyzing and evaluating the relative importance of large numbers of predictors due to yielding more stable results than conventional regression.16,17 Furthermore, the GMW survey is a comprehensive and validated measure of workplace conditions.2 Despite these strengths, the study also has some limitations including convenience sampling and low response rate, which suggest the possibility of sampling bias. However, a descriptive comparison of our sample with the provincial nursing workforce demonstrated less than 10% difference with respect to nurse demographics including gender and professional designation.38 Despite this finding, the study results should be cautiously generalized to other samples and contexts. We also refrain readers from making any cause-and-effect conclusions due to the cross-sectional nature of the study.

CONCLUSION

This is the first study to examine the most important workplace predictors of quality and safe patient care using a validated and comprehensive measure of workplace conditions developed by the Mental Health Commission of Canada. Workload management, psychological protection and engagement, and physical safety in nurses' workplace were found to be among the most important determinants of quality and safe patient care provision. These workplace conditions are believed to impact patient care through influencing nurse mental health. Thus, there is a critical need for better occupational health and safety policies and workplace interventions that are data-driven and responsive to nurses' needs and workplace conditions.

Table.

Relative Importance Ranking of the 13 Workplace Conditions Regressed on Quality and Safety Outcomes Using Random Foresta

	Patient Safety Grade (+)	Quality of Nursing Care (+)	Recommendation (+)
Psychological support	18.14	8.61	16.85
Organizational culture	20.41	10.77	26.43
Leadership expectations	14.02	10.88	21.41
Civility and respect	20.33	8.87	20.46
Psychological job fit	20.40	10.82	15.87
Growth and development	14.94	4.48	14.67
Recognition and reward	20.79	11.79	14.08
Involvement and influence	18.71	11.97	17.70
Workload management	56.15	26.76	39.46
Engagement	18.17	27.96	34.04
Balance	14.38	7.19	12.67
Psychological protection	27.78	15.40	34.14
Physical safety	35.58	12.11	18.44
R 2	0.36	0.16	0.44
RMSE (train)	0.79	0.59	0.58
RMSE (test)	0.85	0.60	0.67

Abbreviation: RMSE, root mean square error.

aThe models are adjusted for demographics including age, gender, experience, health care sector, and geographical region. The positive signs refer to the direction of the bivariate association between predictors and outcomes.