Exploring the utility of the Nursing Role Effectiveness Model in evaluating nursing contributions in primary health care: A scoping review.

AIMS: To inform a discussion for the applicability of using the Nursing Role Effectiveness Model (NREM) in the primary health care setting through a synthesis of the literature that has used the model in all health care sectors.
DESIGN: Scoping Review.
METHODS: Articles were considered for inclusion if they discussed any aspect of the NREM in health care research that presented information related to any nursing regulatory designation, such as nurse practitioner (NP), registered nurse (RN), licensed/registered practical nurse (LPN/RPN) and considered both quantitative and qualitative study designs, including expert opinions and reports.
RESULTS: A total of 22 articles that cited and/or used the NREM were identified in this review. Only two studies were focused in the primary health care setting. There is precedence for the use of the NREM to guide research in primary health care. The NREM should be modified to incorporate the unique characteristics of the primary health care setting.

INTRODUCTION

High‐quality care is a priority in health care systems and is described as the provision of appropriate, efficient and effective services that result in optimal outcomes for patients (Sidani & Braden, 1998). Nursing is well positioned in the health care system to contribute to optimal outcomes for patients and families (Smolowitz et al., 2015). In Canada, there are currently over 400 000 nurses who play important roles in the delivery of health care services in all sectors of the health care system (Canadian Institute for Health Information, 2015). Evaluating the contribution of the nursing profession to the quality and cost of care is necessary for demonstrating the importance of the nursing workforce in different health care settings, for holding nurses accountable for their contribution to patient care and for providing evidence to inform discussions by key stakeholder groups regarding the implementation and optimization of nursing in practice (Dubois, D'Amour, Pomey, Girard, & Brault, 2013; Sidani & Irvine, 1999). The Nursing Role Effectiveness Model (NREM) was developed to assist with the challenging tasks of assessing the contribution of nursing to quality care and outcomes in a multidimensional health care environment (Irvine, Sidani, & Hall, 1998b) (Figure 1). The overall aim of this paper is to synthesize literature that has used the NREM to assess its applicability for use in the primary health care setting. We have focused the discussion on nursing in the primary health care setting as policy‐makers and health care administrators are seeking evidence to inform the integration and optimization of registered nurses (RNs) in primary health care teams, specifically to address the needs of complex patients.

Figure 1

Adapted from the Nursing Role Effectiveness Model (Irvine, Sidani, & Hall, 1998b).

Research examining the contribution of nurses to health care has been primarily conducted in the acute care setting and has focused on staffing, occurrence of adverse events, role enactment and work environment (Doran, Sidani, Keatings, & Doidge, 2002). In acute care, there is substantial evidence demonstrating the positive effects of nursing on reducing adverse patient outcomes (Doran et al., 2002; Dubois, D'Amour, Tchouaket, et al., 2013). For example, reduced adverse events are significantly associated with a higher number of hours of care delivered by RNs (Dubois, D'Amour, Tchouaket, et al., 2013; Kane, Shamliyan, Mueller, Duval, & Wilt, 2007; Needleman, Buerhaus, Mattke, Stewart, & Zelevinsky, 2002). Nurses have continuously shown their ability to meet many objectives of primary health care (Norful et al., 2017). They have responsibilities in primary health care, including patient engagement, leading teams to improve care of patients, coordinating care and promoting population health (Bodenheimer & Mason, 2016). Yet, in the primary health care setting, little research has examined the contributions of nurses to patient, provider or system‐level outcomes. There are several challenges associated with assessing nursing role effectiveness in the primary health care setting, including the multifaceted nature of outcomes, lack of nursing specific data (e.g. unique billing/fee codes for patient care delivered) and the need to understand factors that precede favourable or unfavourable outcomes (Irvine, Sidani, & Hall, 1998a). Therefore, we will use the information gathered from this review to inform our discussion about the use of NREM in the primary health care setting.

BACKGROUND

Nursing role effectiveness model

The NREM was developed based on the Donabedian (1966) structure–process–outcome model of quality care and a comprehensive literature review on nursing‐sensitive patient outcomes and the effectiveness of nursing in acute care (Donabedian, 1966; Doran, 2003,2011; Irvine et al., 1998b; Figure 1). In comparison with the Donabedian (1966) structure–process–outcome model of quality care, it includes more variables and incorporates several nursing specific components allowing for the conceptualization of the nursing contribution to patient outcomes (Irvine et al., 1998b).

Structure component

The structure component consists of patient, nurse and organizational variables that influence the processes and outcomes of care (Irvine et al., 1998b). Patient variables include demographic and health‐related characteristics such as age, gender, education, type and severity of illness and comorbidities. Nurse variables include professional characteristics such as education, experience and skill level. Organizational variables include staffing assignment patterns, staffing mix, work environment characteristics and workload (Doran, 2011; Irvine et al., 1998b).

Process component

The process component is focused exclusively on nursing processes, or interventions, which are treatments, procedures or actions that the nurse performs to enhance the patient's health status or patient's behaviour to move towards a desired outcome (Bulechek & McCloskey, 1999; Sidani & Braden, 1998). Nurses' activities are grouped into three roles that they assume to deliver patient care, namely independent, medical care‐related and interdependent roles. The independent role is described by activities that nurses initiate and undertake autonomously, without a physician's order, to respond to patient needs. Independent role activities include those where only nurses are held accountable, such as patient triage and assessment, nursing interventions and evaluation of patient/family education (Doran, 2003, 2011; Doran et al., 2006b; Irvine et al., 1998b). The medical care‐related role is characterized by activities that are part of the expanded scope of nursing practice, initiated by nurses in response to physician orders. This involves implementation and coordination of care, evaluation of the patient's response to care and the use of nurses' clinical judgements (Irvine et al., 1998a, 1998b; Sidani & Irvine, 1999). The interdependent role consists of activities that nurses perform in association with other health care providers. It also includes activities that nurses undertake which other health care professionals are dependent on for accomplishing their own activities, including team communication, coordination of care and health system maintenance and improvement (Doran, 2003, 2011; Irvine et al., 1998b).

Outcome component

The outcome component consists of nursing‐sensitive patient outcomes. Nursing‐sensitive outcomes are defined as “those that are relevant, based on nurses' scope and domain of practice and for which there is empirical evidence linking nursing inputs and interventions to the outcomes” (Doran, 2003). The nursing‐sensitive outcomes that are included in the model were identified through a literature review and organized into six different categories, namely: (i) prevention of adverse occurrences, such as injury or nosocomial infections; (ii) clinical outcomes, including symptom control and management; (iii) patient's knowledge of the disease, its associated treatments and management; (iv) functional health outcomes, including physical, social, cognitive and mental functioning and self‐care abilities; (v) patient satisfaction with care; and (vi) cost outcomes. Psychological distress, health care use and mortality have recently been identified as nursing‐sensitive outcomes (Doran, 2011); however, they have yet to be incorporated in the original model.

DESIGN

Scoping review methodology was used (Grant & Booth, 2009; The Joanna Briggs Institute, 2014). The aim of the scoping review is to inform a discussion for the applicability of using the NREM in the primary health care setting through a synthesis of the literature that has used the model in all health care sectors.

METHOD

Search methods

The search strategy aimed to find both published and unpublished studies/reports. The following databases were searched in July 2018: Allied and Complementary Medicine, CINAHL, Embase, Mosby's Nursing Consult, ProQuest Dissertations and Theses, PsycINFO, PubMed and MEDLINE (Indexed Citations Only). Google Scholar, key nursing websites (e.g. Canadian Nurses Association) and reference lists of all relevant articles were also searched. The phrase used in all searches was “Nursing Role Effectiveness Model” and/or “NREM.”

ANALYSIS

Inclusion/Exclusion criteria

Table 1 summarizes article selection criteria applied to the article screening process. Articles were considered for inclusion if they discussed any aspect of the NREM in health care research, in any capacity. Any literature published before September 2018 was included. Articles were not limited by location of publication, study design or setting. Only articles written in the English language were included. Two independent reviewers screened the titles and abstracts of articles for relevancy. Full‐text articles deemed relevant were assessed for inclusion using the pre‐established selection criteria. Any disagreements that arose between reviewers were resolved through discussion, or with a third reviewer. This scoping review considered both quantitative and qualitative study designs, including expert opinions and reports. According to scoping review methodology, a formal quality assessment of articles is not required (Grant & Booth, 2009; The Joanna Briggs Institute, 2014); thus, articles were not excluded based on concerns with methodological quality.

Table 1

Article selection criteria

Discussed any aspect of the NREM Published before September 2018 Any location (international literature) No restriction on study population No restriction on nursing regulatory designation No restriction on health care setting English language only

Data synthesis and analysis

Given the heterogeneity of the studies included and the use of different methodologies, study populations, interventions and outcomes, findings are reported as a narrative summary and include tables and figures to aid in data presentation where appropriate. Variables and relationships examined in studies are mapped according the NREM (e.g. structure–process–outcome). Additional details extracted from articles included design, setting and information regarding how the NREM was applied in the study and overall conclusions. Results are organized by health care setting, namely, acute care, long‐term care, home care, ambulatory care, unspecified or multiple locations and primary health care.

RESULTS

Summary of included literature

A total of 22 articles met the inclusion criteria (Figure 2). Most literature identified was conducted in the acute care setting (N = 12). Other settings included long‐term care (N = 1), home care (N = 1), ambulatory care (N = 2), primary care (N = 2) and few were not specific to a single setting (N = 4). Most the studies were conducted in Canada (N = 12) and the United States (N = 6), with few from other countries (e.g. Singapore, Portugal). Table 2 provides an overview of each study, including how the NREM was applied. Table 3 lists each specific variable examined (organized according to the structure–process–outcome components of the NREM).

Figure 2

Search strategy and results

Table 2

Utilization of the Nursing Role Effectiveness Model (NREM) within included studies

Author and date	Purpose	Study design	Setting and sample	Application of the NREM
Curnew, D. 2017 Canada	Examines existing evidence related to nursing roles and resources in primary care settings	Scoping review	Registered Nurses (RNs) and nurse practitioners (NPs) in primary care across Atlantic Canada (Newfoundland and Labrador, Prince Edward Island, Nova Scotia, New Brunswick)	The NREM was the organizing framework for the review
Doran, D. 2006a Canada	Explores whether nursing interventions provided during hospitalization are associated with patients' therapeutic self‐care and functional health outcomes	Repeated measures	Acute care nurses and a voluntary sample of 574 patients	Validates and addresses the structural relationships in the NREM, particularly using structural equation modelling to examine nursing interventions and patient outcomes
Doran, D. 2006b Canada	Addresses the lack of information in administrative databases relevant to nursing as it relates to the reliability of instruments measuring nursing‐sensitive outcomes and patient outcomes	Repeated measures	Acute care and long‐term care staff nurses, sample of 890 patients	Examines NREM nursing interventions and patient outcomes including symptom frequency and severity, therapeutic self‐care, and functional status on admission and discharge
Doran, D. 2014 Canada	Investigates the relationship between evidence‐based practice and client pain, dyspnea, falls and pressure ulcer outcomes to address gaps in the knowledge of home care nursing practice based on best evidence	Cross‐sectional	13 home care offices; 338 home care RNs and Registered Practical Nurses (RPNs); and 939 de‐identified client charts	NREM used to guide the selection of variables for investigation, including client and nurse structural variables, process variables, and evidence‐based nursing interventions
Doran, D. 2002 Canada	Investigates the propositions of the NREM, in which structural variables are expected to influence nursing role performance which in turn is expected to influence patient outcomes	Cross‐sectional	26 general medical‐surgical units in a tertiary care hospital; 372 patients, 254 RNs, RPNs and patient care assistants	Validates the relationships that are proposed within the NREM; provides support that the NREM is a well‐defined framework to guide nursing care evaluation
Dubois, C. 2013 North America	Investigates many framework models to help guide the Nursing Care Performance Framework, a concept of key indicators to judge nursing performance	Systematic review	All settings No specific sample	NREM is one of many framework models included in this review
Endacott, R. 2009 Australia & United Kingdom	Determines activities and outcomes of intensive care unit Liaison Nurse/Outreach Services	Integrative review and meta‐synthesis	Intensive care units (ICU) and acute care units; ICU/Liaison Nurses that provide outreach services	NREM is used as an a priori model to underpin the meta‐synthesis
Gazarian, P.K. 2008 United States	Examines what cues RNs identify as an indicator for patient adverse events and what factors influence their decision to interrupt	Qualitative	Acute care staff nurses	Decision‐making is highlighted as an NREM variable that will influence patient outcomes
Irvine, D. 1998a Canada	Illustrates the use of the model in quality improvement and research activities	Literature review	All settings All nurses	Original paper establishing the NREM
Irvine, D. 1998b Canada	Presents the NREM and describes the components (structure, process, outcome)	Literature review	All settings All nurses	Original paper establishing the NREM
Kossman, S.P. 2008 United States	Addresses community hospital nurses' use of electronic health records (EHRs) on nurses' role performance and patient outcomes, and the views of EHR impact on job performance and patient outcomes	Descriptive qualitative	ICU, community hospitals with the same electronic health system; convenience sample of nurses that worked on either the medical‐surgical floor or ICU, and had used the EHR system for at least 6 months	The NREM suggests that the effect of EHRs (structural) on patient outcomes is mediated through nurses' use of EHRs and how the use of EHRs influences nurses' performance
Landesman, A. 2003 Canada	Develops guidelines for oral care, highlighting the importance of nurses' role in quality improvement initiatives	Descriptive paper of a unit‐based quality improvement initiative	Long‐term care; facility staff trained in assisting with oral care, patients who were able to participate in their own care, and family members who demonstrated interest	Guidelines supported by and based on the NREM framework
Manojlovich, M. 2005 United States	Aims to investigate direct and indirect relationships among the practice environment, nurse–physician (RN‐MD) communication and job satisfaction	Survey	Acute care hospital; 332 hospital nurses in Michigan, United States	Relationships examined are presented by the NREM framework
Manojlovich, M. 2009 United States	Determines the relationships between patient's outcomes and nurses' perceptions of elements of communication between nurses and physicians, and characteristics of the practice environment	Cross‐sectional survey	25 ICUs, 462 hospital nurses who anonymously answered the survey	The NREM is the organizing framework to guide selection of study variables
Mok, W.C. 2015 Singapore	Explores factors surrounding vital sign monitoring in detecting and reporting deterioration in nursing practice	Integrative literature review	Articles including the acute care unit, in‐hospital General ward patients and nurses	Literature reviewed was synthesized based on the structural components of the NREM
Rondinelli, J.L. 2014 United States	Attempts to describe the perceived impact of the staff RN role components on specific activities and outcomes	Descriptive, self‐report survey	Ambulatory Care; 187 RNs from various primary and specialty care clinics in Southern California	NREM structure, process and outcome variables guided the study's design and relationships
Salgueiro, A.F. 2014 Portugal	Tests the NREM structure, process and outcome variables in two different hospital settings	Cross‐sectional, longitudinal	Medicine and surgery units of Central Hospitals (26 units of four hospitals); 364 nurses excluding head nurses, 1,764 patients	Presents the model and tests relationships proposed within the NREM via structural equation modelling to determine the value of the model in assessing nursing care
Saunders, S. 2011 Canada	Aims to determine whether nurse‐driven protocol for anaemia management in a haemodialysis setting is as safe and effective as physician‐driven approaches	Retrospective, non‐equivalent case–control group design	Haemodialysis units in Western Canada Nurse‐Driven Protocol vs. Physician‐Driven Protocol in patients	NREM was used as the organizing framework to guide selection of variables within study
Seabra, P.R.C. 2017 Portugal	Aims to identify factors that contribute to better outcomes in drug users	Correlational, cross‐sectional	Out‐patient drug unit; nurses from nursing ambulatory treatment units and drug users in a methadone programme from three community services	NREM is used to determine how effective nursing interventions are in relation to variables of patient outcomes
Sidani, S. 1999 Canada	Proposes a conceptual framework for evaluating the role and effectiveness of acute care NPs	Literature review	Acute care setting NPs	Proposes an adaptation of the NREM for establishing the acute care NP framework
Tarlier, D.S. 2006 Canada	Examines nurses' primary care practice, continuity of patient care, and clinical health outcomes in a First Nations community	Ethnography	Primary care setting; all nurses in the First Nations Community	NREM is the conceptual framework that underpins the study
White, D.E. 2015 Canada	Aims to describe the amount of time nurses spend on key clinical roles and activities in the workplace	Observational	Medical Units of Tertiary Hospitals; RNs and Health Care Aides	Uses the NREM framework to categorize the recorded activities of nurses

Table 3

NREM variables examined in the articles

Structure	Process	Outcome
Patient Characteristics: Age1,3,4,9,10,12,13,15 Sex3,4,10,12,15 Ethnicity3,10 Medical Diagnosis3,5,9,10,12,15 Severity of Illness4,5,8,9,14,15 Cognitive Status1,13 Comorbidities10,13,15 Severity of Illness13 Baseline Functional Ability13 Nurse Characteristics: Age9,7,14 Sex7,14 Ethnicity7,14 Education3,9,7,13,14,15 Experience4,5,7,13,14,15 Skill Level9,13,15 Proportion of visits made by an RN12 Length of Employment3,7 Organizational Characteristics: Work Hours9,14 Staffing Patterns/Staff Mix3,13,14,15 Workload6,14,15 Work Environment7,9,15 Hospital Characteristics4,7,8 Structural Empowerment7,8 Clinic‐Community Relationships14,15 Technology6,14	Independent Role: Self‐Care Assistance1,2,11 Physical Comfort Promotion1,2 Elimination Management1,2 Immobility Management1,2,11 Patient Education1,2,14 Activity & Exercise Enhancement1,2,14 Drug Management2 Respiratory Management2 Health System Mediation1, 2 Mouth Care13 Support for Patient/Families4,11 Assess Patient History and Chief Complaint14 Nurse Advice14 Triaging14 Developing Plan of Nursing Care14 Evaluation of Patient/Family Education14 Evaluation of Treatment/Procedure14 Nursing Physical Assessments14 Reviewing Discharge Instructions14 Goal Setting with Patient14 Arranging for At‐Home Equipment14 Consistency of Nurse Provider12 Nursing Interventions12,15 Medical Care‐Related Role: Initiating and Managing Medication4,11,14 Initiating & Managing Treatment14 Assessing Laboratory Test Results14 Coordinating Next Laboratory Test14 Coordinating Diagnostic Procedure14 Interdependent Role: Coordination of Care3,14,15,19 Quality of Communication3,7,8,9,10,15 Consult with Clinicians on Plan of Care14 Evaluation of Discharge Readiness14 Evaluation of Medication Effect14 Assist Other Clinicians14 Case Management15	Patient: Functional Status1,2,9,13 Therapeutic Self‐Care1,2,3,9 Symptom Frequency and Severity2 Pressure Ulcer8, 12 Patient Satisfaction9,13,16 Adverse Events4,5 Clinical Outcomes13 Costs of treatment and medications13 Pain12, 13 Dyspnea12 Falls12 Otitis Media15 Prenatal Care15 Care of Newborn 1st Year15 Hypertension15 Mental Health16 Substance Addiction16 Nurse: Knowledge of Guidelines15 Responsibility for Autonomous Practice15 Communication15 Time for Communication15 Mechanism for Follow‐up15 Standard of Care15 Appropriate Referrals15 Appropriate Interventions15 Referral Notes15 Managing Follow‐Up with Consultant15

1Doran et al. (2006a); 2Doran et al. (2006b); 3Doran et al. (2002); 4Endacott et al. (2009); 5Gazarian, Moineddin, and Agha (2008); 6Kossman and Scheidenhelm (2008); 7Manojlovich (2005); 8Manojlovich et al. (2009); 9Salgueiro et al. (2014); 10Saunders (2011); 11White et al. (2015); 12Doran et al. (2014); 13Landesman et al. (2003); 14Rondinelli et al., 2014; 15Tarlier (2006); 16Seabra et al. (2017).

Note: original Nursing Role Effectiveness Model articles and review papers are not included in table.

Acute care

Thirteen studies that used the NREM were conducted in the acute care setting (Doran et al., 2006a; Doran et al., 2006b; Doran et al., 2002; Endacott, Eliott, & Chaboyer, 2009; Gazarian, 2008; Kossman & Scheidenhelm, 2008; Manojlovich, 2005; Manojlovich, Antonakos, & Ronis, 2009; Mok, Wang, & Liaw, 2015; Salgueiro, Lopes Fereira, Lucilia Cardoso, & Vidinha, 2014; Saunders, 2011; Sidani & Irvine, 1999; White, Jackson, Besner, & Norris, 2015), including six in Ontario, Canada (Table 2). Typically, these studies used the NREM as an organizing framework to guide the selection of study variables and there was great variability with respect to the specific variables from each component of the NREM that were investigated. Only a few studies examined relationships across variables in different domains of the model (i.e. structure, process and outcome).

Most studies explored variables in the structural component of the model, including patient, nurse and organizational characteristics. The most commonly measured structural variables included age, patient medical diagnosis, severity of condition, nurse's education, nurse's experience and hospital characteristics. Process variables included a wide array of independent role functions (e.g. self‐care assistance, immobility management, patient education), medical care‐related role functions (e.g. prescribing medication) and interdependent role functions (e.g. quality of communication, coordination of care). With respect to the outcome component of the NREM, patient outcomes (e.g. functional status, therapeutic self‐care), adverse events (e.g. occurrence of falls) and nursing outcomes (e.g. nursing confidence, nursing job satisfaction) were examined in the acute care setting.

Doran et al. (2002) empirically tested the relationships between variables that were proposed in the model using a cross‐sectional study design (Doran et al., 2002). Data related to each domain in the model were collected from 254 RNs and LPNs and 372 patients from a medical–surgical unit in an acute care hospital. Using structural equation modelling, many of the relationships proposed in the NREM were supported by the findings of this study. For example, longer length of employment (structural) had a positive effect on coordination of patient care (process) (β = 0.21, χ 2 = 23.81, p = 0.69), but also resulted in poorer communication between nurses (process) (β = −0.24, χ 2 = 23.81, p = 0.69). Additionally, greater job autonomy (structural) was positively related to communication (process) (β = 0.29, χ 2 = 23.81, p = 0.69). Further validation of the NREM in the acute care setting was provided in two follow‐up studies that used a repeated measures design (Doran et al., 2006a, 2006b). Similar to Doran et al. (2002), data were collected that related to the structure, process and outcome components of the NREM. Both studies explored a broader set of variables and included more specific nursing interventions as a measure of the independent nursing role. Doran et al. (2006a) found that in the acute care setting, patient positioning and self‐care interventions related to oral hygiene had a positive effect on functional status (p < 0.01). Similarly, Doran et al. (2006b) found that improved functional status was related to specific nursing interventions, including energy management, exercise therapy, urinary bladder training, bed rest care, positioning and self‐care assistance (p < 0.05).

Long‐term care

Two studies used the NREM in the long‐term care setting (one being a multi‐setting study not specific to long‐term care) (Doran et al., 2006b; Landesman, Murphy, Richards, Smyth, & Osakue, 2003). In long‐term care, Doran et al. (2006b) found that nursing interventions (process) for bowel incontinence, bed rest care and position in a wheelchair were significantly related to functional status (outcome) (p < 0.05). For example, functional status was most strongly correlated with self‐care assistance (r = 0.43). Relationships between nursing interventions (process) and symptom control outcomes, including dyspnea, pain and fatigue frequency and severity, were also noted (Doran et al., 2006b). Landesman et al. (2003) found that the independent nursing role of mouth care was related to improved clinical outcomes and patient satisfaction.

Home care

There was only one study conducted in home care. Doran et al. (2014) found that nursing interventions (process) consistent with Registered Nurses Association of Ontario (RNAO) Best Practice Guidelines were associated with patient outcomes (Doran et al., 2014). There were associations between structural variables, such as patient age and medical diagnosis and outcome variables, such as pain and dyspnea (Doran et al., 2014). For example, the nursing intervention implemented to observe the pain (outcome) in patients was to screen patients at least once a visit for pain (process) (p < 0.0001). When nurses documented their practice of this intervention more frequently (56.2% documented in all pertinent cases), it was associated with reduced pain in 44% of cases (p < 0.0001). Other patient outcomes observed included falls during the episode of care, presence and improvement of dyspnea and having/being at risk of developing a pressure ulcer.

Ambulatory care

Two studies that used the NREM were conducted in the ambulatory care setting. In the ambulatory care setting, the NREM provided a conceptual structure for variable selection in a descriptive study. Rondinelli, Omery, Crawford, and Johnson (2014) described RNs diverse and complex patient care activities (process variables). Specifically, nurse‐initiated independent processes such as self‐care facilitation, exercise enhancement and nutritional support were examined. Structure components were also examined (e.g. nurse demographics, education, settings, skill mix). Outcome variables included clinical outcomes, prevention of complications, knowledge of disease and treatment, functional status and patient satisfaction.

The second article that used the NREM in ambulatory care was completed by Seabra, Amendoeira, and Sà (2017). Specifically, they examined the NREM in the context of an outpatient drug unit for patients with a methadone addiction to determine how effective nursing interventions are in relation to the patient outcomes when treating patients with substance addictive behaviours. Results showed that all functional patient outcomes were sensitive to nursing care. That is, the nursing interventions contributed to the improved patient outcomes (p < 0.05).

Primary health care

Two studies used the NREM as an organizing framework to guide selection of the study variables in primary health care. Tarlier (2006) conducted an ethnographic study in a remote First Nations Community in Manitoba, Canada, and explored the effect of NP practice on maternal–infant health outcomes. Furthermore, the NREM was used to guide data extraction and map findings from a scoping review, which investigated the current knowledge regarding nursing roles and resources in primary health care in Atlantic Canada (Curnew, 2017). NREM components that were investigated in the primary health care setting included patient characteristics (age, sex, medical diagnosis, comorbidities, severity of illness), nurse characteristics (education, experience, skill level), organizational characteristics (staffing patterns/mix, workload, work environment, clinic–community relationships), the independent role (nursing interventions), the interdependent role (coordination of care, quality of communication, case management), patient outcomes (Otitis Media, prenatal care, care of newborn for the first year, hypertension) and nursing outcomes (knowledge of guidelines, responsibility for autonomous practice, communication, mechanism for follow‐up, standard of care, appropriate referrals, appropriate interventions, referral notes, managing follow‐up with consultant). Neither study using the NREM in primary health care examined relationships proposed in the model to generate evidence related to contributions to patient or system‐level outcomes.

DISCUSSION

The aim of this paper was to synthesize literature using the NREM across all health care sectors and discuss the use of the NREM in the primary health care setting. It was not surprising that most research using the NREM was conducted in acute care, as the model was initially developed for evaluating nursing effectiveness in this setting. Only two studies related to primary health care were found. Although many variables contained in the NREM that were examined in other settings (e.g. home care, long‐term care) could be applicable to primary health care, there are certain variables that would not likely be applicable to primary health care. For instance, nurse's independent role of patient education and patient functional status outcome could be considered reasonable to examine in primary health care. However, nurse's roles related to elimination management, immobility management and physical comfort promotion, as well as patient outcomes such as pressure ulcers, are not as obviously transferable to the primary health care setting. While most studies used the NREM as a guiding framework for variable selection/extraction with a breadth of variables investigated, few studies validated the relationships proposed in the model and no validation of relationships has been performed in the primary health care setting.

Strengths of the NREM

Several strengths associated with using the NREM to evaluate nursing outcomes were identified, particularly with strong evidence for its application in acute care. The NREM postulates that structural variables can affect the processes performed by nurses. For example, effective communication among health care professionals, as a measure of the interdependent nursing role, is fostered by educational preparation and work autonomy which are structural variables (Doran et al., 2002). Perhaps most importantly, the NREM proposes that the process of care can directly influence the outcome. For example, the quality of nursing care, as a measure of the independent nursing role, has been shown to directly affect functional health outcomes (Doran et al., 2002). Structure components are also proposed to both directly and indirectly influence outcomes. For example, patient age and immune status can directly affect the rate of nosocomial infections (Irvine et al., 1998a; Lipsett, 2008; Sidani & Irvine, 1999). The indirect effects of structural variables on outcomes are mediated through the process component. For example, the effect of nurses' job autonomy on the achievement of functional health outcomes, including functional status, mood disturbance and therapeutic self‐care, is mediated through nurses' independent and interdependent role functions (Doran et al., 2002). Therefore, the NREM can serve as a framework to guide the organization and selection of variables in studies evaluating nursing‐sensitive outcomes.

The multiple interventions that health care professionals undertake as part of their regular practice pose a significant barrier to the evaluation of quality of care (Sidani & Braden, 1998). The NREM assists in overcoming this barrier by incorporating a broad set of variables that are reflective of the multidimensional nature of nursing care (Irvine et al., 1998b; Sidani & Irvine, 1999) and has the ability to address the unique contributions of the nurse in independent and interdependent roles (Doran et al., 2002). Although these variables have theoretical applicability to multiple settings, its applicability across various health care settings has yet to be confidently validated.

Another strength of the model is that the NREM attempts to delineate relationships among structure, process and outcome variables and emphasizes the importance of conceptualizing outcomes in terms of the factors that precede them (Sidani & Irvine, 1999). For instance, given that structural variables can indirectly affect outcomes through process variables (Doran et al., 2002), if process variables are neglected then there is the potential for confounding factors to influence study results. Neglecting structure or process components in the NREM can result in limitations associated with the interpretation and generalizability of study results (Sidani & Braden, 1998), and therefore, it is important to examine relationships among all of the components in the NREM (Sidani & Irvine, 1999). The NREM subjects nursing research to address all factors involved with nursing care simultaneously. Unlike the Donabedian (1966) structure–process–outcome model of quality care, the NREM explicitly incorporates patient characteristics, such as age, income, education and health status, which have been shown to influence outcomes (Donabedian, 1966; Sidani & Braden, 1998). This model is further strengthened by the incorporation of patient outcomes which have been empirically found to be sensitive to nursing care (Irvine et al., 1998b). These nursing‐sensitive outcomes allow researchers to explore and delineate nursing specific contributions to patient outcomes, which can be used to hold nurses accountable for their professional practice.

Limitations of the NREM

As for limitations of the model, the NREM does not explicitly reference the importance of ongoing assessment to evaluate outcomes and makes no direct reference to time. As a result, many of the studies identified in the literature review used a cross‐sectional study design to evaluate nursing outcomes and therefore could only measure associations, rather than evaluate the nature or dose of the specific nursing interventions (i.e. no causal relationships can be examined) (Doran et al., 2002; Manojlovich, 2005; Manojlovich et al., 2009; Saunders, 2011). The NREM also does not acknowledge the potential for reciprocal interactions between components or indirect interactions between variables in a given component. For example, although not depicted in the diagram that represents the NREM, Doran et al. (2002) identified that there were interactions between variables in the process component. The NREM also does not explicitly show the direct relationship between the structure component and patient outcomes, which has been identified in other studies (Doran et al., 2006a; Sidani & Irvine, 1999).

Evaluating nursing roles in interdisciplinary primary health care teams

A strong point of the NREM is that it embraces the concept of interprofessionalism. Over recent years, there has been an increasing emphasis placed on delivering primary health care services through interprofessional health care teams in Canada (Health Canada, 2006; Hutchison, Levesque, Strumpf, & Coyle, 2011). Team‐based models of care that incorporate allied health care professionals, such as nurses, social workers and pharmacists, can improve access to health care services, use resources more efficiently and improve chronic disease management in the primary health care setting (Dahrouge et al., 2012; Hogg et al., 2009; Liddy, Singh, Hogg, Dahrouge, & Taljaard, 2011; G. Russell et al., 2010; G. M. Russell et al., 2009). In the structural component of the model, organizational variables include various measures that capture the extent to which a practice functions using an interprofessional model of care. Furthermore, in the process component of the model, the nurse's role in an interprofessional team can be captured by examining their interdependent role functions (e.g. communication with other health care professionals) (Doran, 2003, 2011). Thus, the model offers a framework, strategy, approach and foundation to effectively evaluate distinct nursing roles in the context of interdisciplinary teams, a hallmark of the primary health care reform.

Modifying the NREM for primary health care

There is little direct evidence to support the use of this model to evaluate nursing‐sensitive outcomes in the primary health care setting. When considering the utility of the NREM in the primary health care setting, it is important to first understand the unique needs of this clinical environment in comparison with the acute care facilities that the NREM has been previously been applied to. Acute care focuses on providing health care services in the hospital environment for the necessary treatment of a disease or a severe episode of illness (Canadian Institute for Health Information, 2012), whereas primary health care focuses on health promotion and the prevention, diagnosis and treatment of illness and injury (Health Canada, 2006; World Health Organization, 1978). The relationships proposed in the NREM and the nursing‐sensitive outcomes that are included in the model are supported by empirical evidence synthesized from the literature in acute care (Doran, 2011). It is unclear whether these relationships and outcomes apply to the primary health care setting. A comprehensive review of the literature and consultation with experts would therefore need to be performed to determine which variables identified in the NREM are the most relevant to nursing outcome evaluation in primary health care before the NREM could be used in this setting (Haggerty, 2007; Hogg, Rowan, Russell, Geneau, & Muldoon, 2008). Recently, a systematic review conducted by Norful et al. (2017) synthesized international literature related to the roles/processes of primary care RNs and made recommendations for optimizing their contributions in primary health care teams. This review included 18 studies from six countries. Fundamental roles/processes of the RN in primary care identified were assessment, monitoring and follow‐up of patients with chronic diseases (Norful et al., 2017). Similarly, a systematic review synthesizing outcome measures and the effectiveness of this important and growing role in primary health care teams is needed and currently being conducted by the present research team (Lukewich et al., 2018). Such a review would inform the modification of the outcome domain in NREM. For example, with respect to the nursing‐sensitive outcomes identified in the NREM, functional status, as a measure for readiness of discharge from a health care institution or service, may not be relevant in the primary health care setting, where patients receive care periodically as needed over the course of their life‐time and are not discharged. Many studies have found that nurses in primary health care are extensively involved with chronic disease management and can positively affect patient outcomes (Dahrouge et al., 2012; Denver, Barnard, Woolfson, & Earle, 2003; Hogg et al., 2009; Kleinpell, 2009; Laurant et al., 2009; Loveman, Royle, & Waugh, 2003; Lukewich, Corbin, et al., 2014; Lukewich, Edge, VanDenKerkhof, & Tranmer, 2014; Lukewich, Edge, VanDenKerkhof, Williamson, & Tranmer, 2016; Renders et al., 2001; Schadewaldt & Schultz, 2011; Vrijhoef, Diederiks, Spreeuwenberg, Wolffenbuttel, & van Wilderen, 2002). Thus, there is precedence for the modification of the NREM to meet the unique requirements of different health care environments, particularly primary health care (Redekopp, 2007; Sidani & Irvine, 1999; Yoon, 2011).

Limitations

A limitation to this review is that studies were only included that specifically used the NREM as a guiding model for outcome evaluation. Studies that investigated the relationships proposed in the NREM without referencing the NREM are not included. Thus, specific relationships in the model may be validated outside of the context of the model's use. Despite utilizing a comprehensive search strategy, it is possible that there are articles that used the NREM that were not captured in this review.

CONCLUSION

Although nurses are well positioned to influence patient outcomes in primary health care, the unique contributions of nurses to patient outcomes have yet to be firmly established in this health care setting. One of the barriers hindering the evaluation of nursing‐sensitive outcomes in primary health care is the lack of a comprehensive model that can be used as a framework to guide such studies. Given the success of the NREM in the acute care setting, it is likely that this model will be useful in the development of studies assessing similar relationships in primary health care. However, before this model can be successfully used to facilitate primary health care nursing research, it must first be modified to incorporate the unique characteristics of the primary health care setting. Once modified, the NREM has the potential to identify nurse‐specific outcomes that can be collected and contributed to national health database initiatives, giving the nursing profession accountability and justify their importance in health care policy decision‐making.

CONFLICT OF INTEREST

No conflict of interest has been declared by the authors.

AUTHORS' CONTRIBUTION

JL, JT, MK and AW: Substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; drafting the manuscript or revising it critically for important intellectual content; given final approval of the version to be published. Each author should have participated sufficiently in the work to take public responsibility for appropriate portions of the content. All authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

RESEARCH ETHICS COMMITTEE APPROVAL

Research Ethics Committee approval was not required for this review.

PATIENT CONSENT

No patient consent was required for this review.