Development of nurse practitioner metaspecialty clinical practice standards: A national sequential mixed methods study.

AIM: To achieve profession-wide consensus on clinical practice standards for six broad Australian nurse practitioner specialty areas (termed metaspecialties).
DESIGN: Sequential mixed methods with initial interpretive study (Interpretive Phase) followed by modified three-round Delphi study (Survey Phase).
METHODS: Participants from all Australian jurisdictions were recruited. Main eligibility criterion was current endorsement as a nurse practitioner for 12 or more months. Interpretive Phase comprised in-depth interviews of purposeful sample of nurse practitioners to identify clinical care activities and practice processes. Six sets of draft clinical practice standards relevant to six previously identified metaspecialties were developed. Outcome informed Round 1 of Survey Phase (six nested web-based Delphi surveys), with draft standards reviewed profession-wide. Responses comprised scoring using Likert scales to calculate content validity indexes for individual standards with qualitative responses supporting decision-making. For Rounds 2 and 3, participants rated relevancy of original or revised standards after consideration of individual and group feedback. The study was conducted 2014-2017.
RESULTS: Interpretive Phase: Analysis of interview data with 16 nurse practitioners provided 75 draft standards. Survey Phase: 221 nurse practitioners completed Round 1 (20% of then eligible Australian nurse practitioners). Weighted respondent retention was 92%. Seventy-three standards were validated, with final content validity indices of 92-100%. Scale-level indices were 98%, strongly validating metaspecialty taxonomy.
CONCLUSION: A research-derived, professionally endorsed suite of nurse practitioner clinical practice standards was developed. This provides a broad clinical learning structure with metaspecialties guiding nurse practitioner student clinical education. IMPACT: The clinical practice standards and metaspecialty taxonomy strengthen nurse practitioner clinical education and professional development nationally and internationally. These novel study methods and findings are applicable to advanced specialty roles in other health professions.

INTRODUCTION

Globally, there is increasing emphasis on innovation and extended scope of practice for health professionals to improve workforce efficiency and patient outcomes. One rapidly expanding role worldwide is the nurse practitioner. Nurse practitioners are registered nurses with additional education, experience, and qualifications. They provide complete episodes of care, often in contexts with little or no healthcare access. In Australia, their practice is underpinned by a nursing model of care enhanced by ability to independently prescribe medicines, request, and interpret diagnostic testing and refer to medical and allied health specialists.

An increasing number of countries have nurse practitioner title protection, enabling role regulation, and workforce standardization. Title protection may be state based or national, denoting an advanced practice nurse with additional training and authorization to work within an extended scope of practice. Title protection ensures a consistent level of professional practice in those countries. However, practice focus varies across countries. For example, the United States of America (USA) and Canada have nurse practitioner education and authorization focussing on mutually exclusive populations, whereas Australian nurse practitioner authorization is general (referred to as ‘endorsement’), without reference to a specialty area or population focus and is regulated at a national level.

There is agreement through the International Council of Nurses that preparatory nurse practitioner education is at master's level. Worldwide, master's course design is customized to delivering skills, knowledge, and expertise supporting the broad professional attributes of nurse practitioner practice. However, nurse practitioner student clinical education generally conforms to development of either generalist or specialist services. For both education modes, curricula need to ensure provision of clinical education that will enable the student to develop the advanced clinical skills, knowledge, and expertise required immediately on authorization and with capacity to adapt future practice to emerging evidence, as well as changing patient, service, and community expectations. The focus of this research report is the clinical education component in Australian nurse practitioner master's degree courses.

BACKGROUND

The master's degree format varies with some countries having curriculum design that meets a single set of standards for entry to practice as a nurse practitioner, for example, Australia and New Zealand (Australian Nursing & Midwifery Accreditation Council, 2015; Nursing Council of New Zealand, 2017). Other countries have separate post‐graduate degrees for each nurse practitioner population focus or speciality, for example, the USA (National Organisation of Nurse Practitioner Faculties, 2013) and Canada (University of Windsor, undated). There is much diversity across European countries with not all countries requiring master's level preparation (Sánchez‐Gómez et al., 2019). Notwithstanding course format, there are core regulatory and professional requirements that govern and inform course content and design (American Association of Nurse Practitioners, 2013; Nursing & Midwifery Board of Australia, 2014; Sánchez‐Gómez et al., 2019). Educational preparation must bridge academic and clinical requirements for this intensely clinical role. Ensuring consistency of level and content of clinical education for nurse practitioners has been problematic in many countries (Pulcini et al., 2010). Indeed, there are few peer‐reviewed clinical learning frameworks addressing advanced specialty‐level clinical education and training for any healthcare profession (Gardner et al., 2016).

Population‐focussed clinical standards for nurse practitioners have been in existence in the USA for many years, but their application is limited internationally because of jurisdictional variations in law and practice. Moreover, the language and content of USA clinical standards is often inconsistent with some content focussing on categories of very specific task acquisition, with other content addressing higher‐level practice (e.g., Hoyt et al., 2010). Few other countries have published speciality standards for advanced practice and these are not research based or validated (Gardner et al., 2014a).

A range of nomenclature is used in the literature to refer to these clinical educational standards. While ‘competency’ is the most commonly applied term, critics argue that competence is a somewhat simplistic and behaviouristic approach (Cairns & Mulloch, 2016) and not appropriate for learning and teaching at an advanced level. The term ‘standard’ is used for this research and throughout this study.

In Australia, the nurse practitioner title has been protected nationally since 1998. A master's degree is the entry‐level qualification for practice (Helms et al., 2017a). While Australian nurse practitioners work across acute and primary care settings and very diverse geographical locations (Middleton et al., 2016), their educational preparation is governed by a core set of national accreditation standards (Australian Nursing & Midwifery Accreditation Council, 2015) supported by national regulatory standards for professional practice (Nursing & Midwifery Board of Australia, 2014). This educational and regulatory approach has enabled nurse practitioner student clinical learning to be tailored and contextualized for the diverse Australian healthcare needs. However, this flexibility can result in inconsistencies for workplace‐based student clinical learning with an entry‐level nurse practitioner workforce that is inconsistently clinically prepared (Schwartz, 2019).

The growth of clinical specialities and subspecialities has increased over the past 20 years and presents challenges for clinical education in many health professions, including nursing. In Australia, while educational preparation for the nurse practitioner role has had some success in bridging workplace learning and academic requirements, many gaps remain in structure and governance of learning and teaching in clinical settings where most clinical education takes place. In a country like Australia, with a very large land mass and relatively low population, advanced speciality clinical learning must be sufficiently flexible to meet diverse educational needs, while conforming to a nationally agreed level of practice.

Palliative care and nephrology are two areas where clinicians have developed practice standards to support student clinical education. The Victorian Palliative Care Nurse Practitioner Collaborative developed a set of clinical practice standards for nurse practitioner students in palliative care (Quinn et al., 2011). Douglas and Bonner reported a consensus statement developed for clinical education of nurse practitioner students in nephrology settings. They identified the ‘absence of clearly documented information about expected clinical learning outcomes to support the nephrology nurse practitioner student’ (Bonner & Douglas, 2011, p. 14).

More recently, a national study focussed on Australian emergency nurse practitioners, the specialty employing the highest proportion of nurse practitioners (Middleton et al., 2016). In‐depth exploratory research followed by consensus techniques enabled development of research‐based specialty standards for emergency nurse practitioners, with key indicators for each standard (O'Connell, 2015; O'Connell & Gardner, 2012; O'Connell et al., 2014). O’Connell et al. were the first Australian researchers to publish empirical research that developed a practice framework and speciality clinical practice standards for nurse practitioners. The pedagogical implications from their research go beyond the context of the emergency specialty to the clinical educational preparation required of all nurse practitioners in Australia and other countries where a master's degree is required for authorization to practice. These authors argued the capability framework they used is important for demonstrating criteria required by the Australian Qualifications Framework Council for Masters Degrees are met (Australian Qualifications Framework Council, 2013). This level of education is consistent with the trend away from a behaviourist and competency‐based approach in postgraduate education more generally, both in Australia and internationally. Their theoretically informed research (O'Connell et al., 2014), specific to emergency nurse practitioner roles and education, also highlighted the need for similar research in other specialty areas of practice for Australian nurse practitioners. An Australian framework with nationally agreed, broad clinical domains and associated clinical standards would promote national consistency of clinical education, in the same way that core nurse practitioner standards have achieved consistency at the professional level.

In response to this need, a focus group of nurse practitioner clinical experts and leaders identified six broad nurse practitioner clinical practice areas for the Australian context (Gardner et al., 2014b). These practice areas were called metaspecialties. A metaspecialty ‘groups specialties with similar skill‐sets, knowledge and/or expertise, which comprehensively reflect the diverse healthcare needs of population groups. They are not intended to be mutually exclusive’ (Helms et al., 2017a). The six metaspecialties comprised Ageing and Palliative Care; Child and Family Health Care; Chronic and Complex Care; Emergency and Acute Care; Mental Health Care; Primary Health Care. They were reviewed by the Australian nurse practitioner profession using a national Delphi survey (Helms, 2017; Helms et al., 2017a). A two‐phase study then sought profession‐wide consensus on clinical practice standards for each of the six metaspecialties. This article reports on that study.

THE STUDY

Design

The study used a sequential mixed methods design. An initial interpretive study (Interpretive Phase) was followed by a modified three‐round Delphi study (Survey Phase):

Interpretive Phase involving in‐depth interviews for development of draft clinical practice standards and,

Survey Phase involving a modified Delphi study for validation of the above clinical practice standards.

Aim – interpretive phase

The qualitative phase used an interpretive research approach, guided by the following research question:

What skills, knowledge, and expertise are employed by Australian nurse practitioners to manage specialty patient care?

Participant eligibility and recruitment interpretive phase

The study population was endorsed of nurse practitioners practising in Australia at the time of the study. The main inclusion criteria were 12 or more months’ endorsement (authorization) and employment for at least 0.5 fulltime equivalent hours in nurse practitioner positions. Given that standards for emergency nurse practitioners had been developed and validated previously (O'Connell, 2015; O'Connell & Gardner, 2012), recruitment for this specialty was not included. This exclusion was clarified initially when potential participants expressed interest. Recruitment was undertaken through the Australian College of Nurse Practitioners website.

Data collection interpretive phase

In‐depth telephone interviews were conducted between February 2014 ‐May 2015. Interviews were unstructured and directed by the research question. Participants were guided by questioning to provide vignettes of clinical care. In preparation, each participant was asked to bring to the interview two de‐identified complex patient case studies that demonstrated their specialty practice. Interviews were transcribed from direct audio‐recordings. Limited demographic and clinical role data were collected, sufficient only to enable description of service delivery contexts. Recruitment ceased on reaching data saturation. Draft standards were reviewed in entirety to identify gaps and ensure consistency of complexity and terminology.

Data analysis interpretive phase

Demographic and professional characteristics were summarized using descriptive statistics. For each interview, case studies of nurse practitioner practice were deconstructed to identify and code inherent skills, knowledge, and expertise required to manage care. Data were subjected to a process of reverse engineering where a final product is taken apart or deconstructed to identify component parts, a process previously applied to software systems (Chikofsky & Cross, 1990) and development of other nurse practitioner standards (Gardner et al., 2006, 2008). Based on this analysis, researchers developed draft standards, which were then mapped to the six metaspecialties.

Aim – survey phase

A modified three‐round Delphi study was conducted, using a web‐based survey format, to validate draft standards developed in the Interpretive Phase. In Round 1, participants were provided with these previously developed draft standards. Subsequent rounds followed a classical Delphi approach.

Participants survey phase: Eligibility and recruitment

The only eligibility criterion was current endorsement as a nurse practitioner by the Nursing and Midwifery Board of Australia (NMBA) for 12 or more months. At the time, the total Australian population of nurse practitioners endorsed for 1 or more years was estimated to be approximately 1,210 (Nursing & Midwifery Board of Australia, 2015).

Participants were recruited between January and April 2016. Primarily, recruitment was conducted through the Australian College of Nurse Practitioners, whose membership at the time included 75%–80% of all endorsed nurse practitioners in Australia. Australian College of Nurse Practitioners membership may have been over‐represented constituting a recruitment bias. Additional recruitment was undertaken, particularly through the Australian College of Mental Health Nurses and the Older Persons Collaborative, to reduce this potential bias. Convenience and snowball sampling techniques promoted wide distribution of recruitment messages across the eligible population. This mixed recruitment was necessary because there was no accessible national nurse practitioner population database. Recruitment processes had been validated in the earlier, related nurse practitioner study (Helms et al., 2017a). Once consented, participants were invited to provide input to one or two metaspecialties that matched their areas of clinical expertise. Limitation to two metaspecialties reduced participant attrition and researcher burden and was fixed at the first round. Follow‐up reminder processes used previously applied approaches (Helms et al., 2017a).

Instrument survey phase

The Round 1 survey instrument had two sections. Section A comprised questions about participant demographic and professional characteristics. Section B was divided into six nested surveys, corresponding to the six metaspecialties. For analysis, each was managed as a separate Delphi study. For Round 1, Section B of each nested survey comprised the relevant set of draft clinical practice standards. Pilot testing of the Round 1 survey instrument, for content and face validity, was completed with a six‐member panel of clinicians and researchers and deemed sufficient for all rounds.

Data collection survey phase

The survey took place between April ‐June 2016. It was managed using Qualtrics software (Qualtrics, 2013), which enabled sophisticated data linkage between rounds while ensuring anonymity between participants (Helms et al., 2017b). In Round 1, for each metaspecialty between 10‐14 draft standards were reviewed. Participants rated relevancy of each standard on a Likert scale as ‘highly relevant’, ‘quite relevant’, ‘somewhat relevant’, or ‘not relevant’. Participants could provide additional information supporting their decisions using prepared options and open text boxes, again based on the earlier, related study (Helms et al., 2017a). Finally, in Round 1 only, additional questions for each metaspecialty, explored whether any standards could be combined, whether participants had suggestions for additional standards or had other feedback.

In Round 2, findings from Round 1 were provided to participants both as individualized and summarized group feedback based on content analysis of responses (see Helms et al., 2017b for more information). Participants were asked to consider feedback and rate relevance of original or revised standards. No qualitative feedback was collected. For Round 3, one metaspecialty required minor wording amendment for one standard only, applying the same principles for feedback and revision. Invitation was confined to participants who had completed that metaspecialty in Round 2.

Data analysis survey phase

Quantitative data were downloaded to the Statistical Package for Social Sciences (2016, version 23, IBM Corps, Armonk, NY) and qualitative data to Microsoft Excel (2010, version 14.0). Demographic and professional profile data were analysed using descriptive statistics appropriate to data collected. An individual‐level content validity index was calculated for each standard. Likert scale ratings were recoded as ‘relevant’ or ‘not relevant’. A content validity index of 85% or greater was used to define consensus on individual standards. Scale‐level content validity indices established validity for each metaspecialty construct in entirety. These were calculated by summing individual‐level indices and dividing by number of standards in that metaspecialty.

Round 1 qualitative data were analysed to identify consistent suggestions for:

development of new standards;

combination of one or more standards and

revision of wording in any standard.

Guiding principles for changes to, combination of, and deletion of standards in preparation for Round 2 were as follows:

more than five participants indicating ‘needs minor or major rewording’;

relatable justification provided in free text responses;

consistency of participant responses suggesting combination of standards; and

a content validity index of less than 90% for revisions to wording for the standard under consideration.

Ethical considerations

Human research ethics approval was obtained from the Australian Catholic University (2014 42V) and Queensland University of Technology (1400000403). Once ethics approval was received, administrative approval was given by the Australian College of Nurse Practitioners to use their membership list and website for recruitment. For the Survey Phase, consent was implied through survey completion. Survey participants had the option to be named as contributors for professional development purposes, to reduce attrition, and increase vestment in the process (see Appendix S1).

Validity, reliability, and rigour

The research team had extensive research and clinical expertise. Study processes ensured trustworthiness of draft standards (Interpretive Phase). Draft standards were assessed and revised by the wider nurse practitioner community (Survey Phase).

For the Interpretive Phase, a purposeful sample of nurse practitioners was recruited to ensure representation of all metaspecialties and maximum variation of sub‐specialities, locations of practice and employment arrangements. Two research team members, experienced in qualitative research, undertook interviews, independently analysed data, and then presented findings for confirmation by all research team members.

For the Survey Phase, the reactive Delphi processes had been previously validated (Helms et al., 2017a, 2017b). Processes were embedded that enabled participants to provide additional information in Round 1 and receive individualized and summarized feedback in subsequent rounds, thus, minimizing loss of depth and richness of individual opinion (Helms et al., 2017a). The high response rate and retention across rounds enhanced generalizability of findings. A content validity index of 85% or greater defined consensus. A content validity index of greater than 78% has long been established as correlating with ‘excellent’ agreement among 10 or more experts and corrects for chance agreement (Polit et al., 2007). Scale‐level content validity indices established validity for each metaspecialty construct as a whole.

RESULTS

Interpretive phase participant demographics

Sixteen nurse practitioners completed in‐depth interviews. Most were aged 50 years or over and nearly 90% were women. A wide range of workplace characteristics was represented. While most nurse practitioner roles were publicly funded and located in metropolitan areas, there was representation from privately funded services and from rural and remote locations. Participant scopes of practice encompassed all metaspecialties. Mean interview duration was 55 min (range 45 to 66 min).

Interpretive phase draft clinical practice standards

An early, not unexpected, finding was that nurse practitioner work was often mapped across more than one metaspecialty and this influenced data analysis. Results confirmed that metaspecialties were not mutually exclusive with workforce flexibility being an important aspect of nurse practitioner scope. Speciality skills, knowledge, and expertise of most participants encompassed at least two metaspecialties with care delivered in diverse contexts.

The process of reverse engineering allowed skills, knowledge, and expertise required for specific circumstances to be identified and clustered at the metaspecialty level according to aspects of care such as assessment or discharge planning. Examination of modes of practice (defined by O'Connell, 2015, p. 5, as ‘practice features that are common across all service models and levels of patient acuity’) uncovered high‐level knowledge in ordinary descriptions. This analytical process informed development of draft standards that included both empirical skills and knowledge as well as expert care delivery. Examples of two draft standards are as follows:

Demonstrates expert, compassionate judgment, and knowledge of legal implications of end‐of‐life care for person and family.

Conducts holistic and advanced assessment of the child and family caring for the child, including social and cultural history using in‐depth knowledge of child development.

Seventy‐five draft standards were developed, with 10 to 14 draft standards per metaspecialty.

Survey phase response rate and participant demographics

A total of 221 endorsed Australian nurse practitioners, who met the study inclusion criterion, completed Round 1 (Figure 1). For Section B, 102 nurse practitioners (46%) elected to respond to one metaspecialty with most of the sample responding to two metaspecialties (BN = 119, 54%). This provided 340 individual responses across all metaspecialties in Section B (with between 22 and 81 responses for each metaspecialty). The most common metaspecialty combinations were as follows:

FIGURE 1

Tree diagram of recruitment, completion, and non‐response in each round of Survey Phase, the modified Delphi study

Primary healthcare and Emergency and Acute Care (12%)

Chronic and Complex Care and Ageing and Palliative Care (9%)

Chronic and Complex Care and Primary Healthcare (8%)

Primary Healthcare and Child and Family Care (7%)

Two hundred and five nurse practitioners completed Round 2 with a total of 319 individual responses across all metaspecialties. Round 3 comprised a single Delphi study with 66 completed responses (88% of 75 potential participants from Round 2 for that metaspecialty). Overall, weighted retention was 92%. Figure 1 presents information about participant retention across rounds.

Table 1 provides a demographic and professional profile for Round 1 participants. The median number of years as a nurse practitioner was 5 years. Over 25% of participants were employed in Queensland and the smallest numbers employed were across Tasmania and the two territories (Table 1, data aggregated to preserve anonymity). Work locality of most participants was in major cities and inner regional sectors (n = 180, 81.4%), with 18.6% (n = 41) working in remote or very remote locations as categorized by the Australian Bureau of Statistics Remoteness Structure (Australian Bureau of Statistics, 2011). Most nurse practitioners were based in hospitals (data not shown). The participant profile was similar across all metaspecialties, except Mental Health Care, where fewer worked in outer regional, remote, or very remote areas.

TABLE 1

Demographic and professional profile of Round 1 participants by work locality, defined by ABS remoteness structure

Participant characteristic	Remoteness areas: major cities and inner regional	Remoteness areas: outer regional, remote, or very remote	Total (%)
	N (%)	N (%)
Number of participants	180 (81.4)	41 (18.6)	221 (100)
Employment sector
Public sector	52 (23.5)	11 (5.0)	63 (28.5)
Private sector	127 (57.5)	29 (13.1)	156 (70.6)
Not employed or retired	1 (0.5)	1 (0.5)	2 (0.9)
Median years as a nurse practitioner	3	6	5
Median years as a registered nurse	30	32	30
Principal area of main nursing job
Critical care and emergency departments	16 (7.2)	10 (4.5)	26 (11.8)
Community health	37 (16.7)	6 (2.7)	43 (19.5)
General practice	17 (7.7)	7 (3.2)	24 (10.9)
Mental health	17 (7.7)	1 (0.5)	18 (8.1)
Other b	93 (42.1)	17 (7.7)	110 (49.8)
State or territory
Queensland	48 (21.7)	11 (5.0)	59 (26.7)
New South Wales	41 (18.6)	8 (3.6)	49 (22.2)
Victoria	37 (16.7)	7 (3.2)	44 (19.9)
Western Australia	19 (8.6)	11 (5.0)	30 (13.6)
South Australia	19 (8.6)	1 (0.5)	20 (9.0)
Tasmania, Australian Capital Territory, and Northern Territory	16 (7.2)	3 (1.4)	19 (8.6)
Have served on state committees (% yes)	82 (37.1)	20 (9)	102 (46.2)
Have published in peer‐reviewed journals (% yes)	70 (31.7)	10 (4.5)	80 (36.2)
Have been invited speaker at conference (% yes)	106 (48.0)	22 (10.0)	128 (57.9)
Have presented paper or poster at conference (% yes)	121 (54.8)	28 (12.7)	149 (67.4)
Have served as supervisor for nurse practitioner or nurse practitioner student (% yes)	136 (61.5)	32 (14.5)	168 (76.0)

Australian Bureau of Statistics.

Other includes nurse practitioners who nominated ‘other’ because their perceived principal area was not included in the validated Health Workforce Australia (Health Workforce Australia, 2012) list as well as small numbers of nurse practitioners who specifically nominated aged care; education; family, maternal, and child health; management; medical; midwifery; mixed medical/surgical; paediatrics; peri‐operative; rehabilitation and disability; research; and surgical.

Clinical practice standards

In general, standards were strongly validated in Round 1 with all standards in five metaspecialties achieving content validity indices of 85% or more and most participants suggesting no or minor wording changes. Most qualitative feedback comprised suggestions more relevant for very specific practice contexts, rather than at the metaspecialty level. The need to keep standards applicable across a wide range of clinical contexts was fed back to participants at the commencement of Round 2.

Based on guiding principles previously outlined, no new standards were needed. The high level of consensus for all standards and generally confirmatory qualitative participant feedback supported the breadth of clinical practice encompassed by the draft standards.

Two standards were deleted after Round 1 because participant feedback suggested that key aspects could be incorporated into other standards (deletions in Primary Health Care and Emergency and Acute Care). Round 1 feedback suggested minor wording changes for six standards (one Primary Healthcare standard; one Chronic and Complex Care standard; two each in Ageing and Palliative Care and in Emergency and Acute Care).

In Round 1, for the Emergency and Acute Care metaspecialty, three standards achieved content validity indices of less than 85%. In this last metaspecialty, acute care standards were developed to complement the Emergency Department standards previously established (O'Connell, 2015). However, qualitative feedback indicated that there was strong participant focus on emergency department responsibilities to the exclusion of acute inpatient needs, for example:

The standards need to take into consideration routine practice of Emergency Nurse Practitioners rather than a focus on (advanced life support, medical emergency) teams, Resuscitation care, etc.

After admission, follow up no longer required, care taken over by another specialty.

Examples of summaries of Round 1 feedback and explanation of proposed changes for review in Round 2 are presented in Table 2.

TABLE 2

Excerpts of summaries of Round 1 and Round 2 feedback and proposed changes to standards for participant review

Round and Metaspecialty	Excerpts of summaries	Example of changed wording for specific standards a	Rationale for changed wording
Round 1 Feedback: Ageing and palliative care	While the combination of ageing and palliative care in the metaspecialty requires resolution external to this Delphi study, the general feedback for all standards grouped here was very positive. There were very high relevancy ratings for most proposed standards so only minor wording changes have been made, no standards were combined and no new standards were added. The phrase ‘nearing or surpassing anticipated life expectancy’ was challenged by a small number of respondents usually reflecting a specific palliative care focus or a specific healthy ageing focus. However, the standards that included this phrase were very highly validated and so we have concluded that the phrase should be retained. We consider that this phrase encompasses people's ‘goals and anticipated life expectancy’.	Educates person and carers about the correct use of opioids and other medications in ageing and palliative care	A small number of respondents suggested that the focus on opioids was too specific so the phrase ‘and other medications’ has been added.
Round 2 Feedback: Emergency and Acute Care	Congratulations on staying with us for the long haul. Almost all clinical practice standards were finalized in Round 2. This third round comprises only the Emergency and Acute Care Metaspecialty so you have received this invitation because it was your chosen, or one of your two chosen metaspecialties in previous rounds. The relevancy of only one standard remains in question at the completion of Round 2. As before, may we remind you that this metaspecialty includes standards for nurse practitioners who are not emergency nurse practitioners so some standards focus on acute inpatient care responsibilities. Please also remember that we intend these metaspecialty standards to complement the existing emergency nurse practitioner specialty standards (web link to O'Connell et al provided: http://www.azille.com.au/standards.pdf).	Assesses risk and initiates pharmacological and non‐pharmacological preventative therapies for the sequelae of immobilization during the acute phase of illness	60% of those who rated this standard as ‘not or somewhat relevant’ had a position title that indicated they did not work solely as emergency nurse practitioners. The phrase ‘due to surgery and/or intensive care therapy’ has been replaced by ‘during the acute phase of illness’. This change acknowledges that the standard previously excluded the acutely ill who had not needed surgery or ICU care.

Underlining denotes new words added.

On completion of Round 2, content validity indices for almost all standards were maintained or increased. Virtually all minor wording changes were validated and achieved content validity indices of 95% or more (Table 3). One Emergency and Acute Care standard did not achieve 85%, necessitating a third round for this metaspecialty only, when the final revised standard was validated.

TABLE 3

Summary of completed sample sizes for each round and scale‐level content validity indexes across all metaspecialties.

	Primary Healthcare	Emergency and Acute Care	Chronic and Complex Care	Ageing and Palliative Care	Child and Family Health	Mental Healthcare
Round 1 completed sample size	93	81	75	43	27	22
Round 2 completed sample size	84	75	73	40	23	22
Attrition between Round 1 and Round 2: n (%)	9 (10)	6 (7)	2 (3)	3 (4)	4 (15)	0 (0)
Round 3 completed sample size	Not applicable	66	Not applicable	Not applicable	Not applicable	Not applicable
Attrition between Round 2 and Round 3: n (%)	Not applicable	9 (12)	Not applicable	Not applicable	Not applicable	Not applicable
Final number of standards validated	13	9	14	14	12	11
Round 1 scale‐level content validity index	96%	87%	97%	96%	96%	97%
Round 2 scale‐level content validity index	98%	98%	98%	98%	98%	98%

The metaspecialty scale‐level content validity index was 98% for all six metaspecialties at the completion of Round 2 (Table 3). See Figure 2 for metaspecialty framework and number of validated standards for each metaspecialty.

FIGURE 2

Metaspecialty framework with number of validated clinical practice standards for each metaspecialty

In summary, over a period of 6 weeks, a major three‐round Delphi study was completed comprising six nested surveys. Participant retention was very high. Seventy‐three standards across six metaspecialties were validated with individual standard content validity indices ranging from 92‐100% and all scale‐level indices of 98%. The full set of standards is published online (Gardner et al., 2019).

DISCUSSION

Clinical learning and teaching for nurse practitioner students, to date, has been subject to a wide range of influences not necessarily pedagogically determined. The research reported here addressed this issue with development of an evidence‐based, comprehensive suite of clinical practice standards that will guide work‐based learning for nurse practitioner education and professional development.

This two‐phase study enabled Australian nurse practitioners to have direct input to development of clinical practice standards across the six metaspecialties. The standards encompass all facets of care delivery across all specialties and were developed and validated directly by clinicians in partnership with the research team. To our knowledge, this is the first framework with a comprehensive national suite of nurse practitioner clinical practice standards for learning and teaching.

The participant sample in the Interpretive Phase provided great diversity of clinical specialty, location, and type of practice. The level of participation for the Survey Phase was high with the completed Round 1 survey sample comprising approximately one fifth of all eligible nurse practitioners. Participant demographics closely matched the nurse practitioner population endorsed for at least 1 year at the time of the study (Nursing & Midwifery Board of Australia, 2015), suggesting the sample was representative of the Australian nurse practitioner population. National data about nurse practitioner professional characteristics are not available publicly but the profile is consistent with another recent Australian Delphi study of nurse practitioners (Helms et al., 2017a) and other survey‐based research with Australian nurse practitioners (e.g., Currie et al., 2018). Retention across rounds was high, again matching similar web‐based Delphi studies (Gill et al., 2013; Helms et al., 2017a). Therefore, we argue that the results are generalizable to the Australian nurse practitioner population nationally at the time of publication.

Development and validation of specialty clinical practice standards

Internationally, while there are several sets of nurse practitioner clinical practice standards for speciality practice, most have been developed from the literature and/or expert opinion rather than from original primary research. Often, these clinical practice standards are limited to specific lists of health conditions to be treated and skills to be mastered. Emphasis on specific skill acquisition has the potential to restrict relevance of clinical practice standards over time: a specific skill may be redundant when new technology is introduced or a therapy is no longer supported by evidence. More importantly, a focus on narrow, skills‐based clinical learning has the potential to undervalue the complexity of care delivery by nurse practitioners and lead to what are arguably task‐based nurse practitioner roles, for example, the endoscopy nurse practitioner role (Duffield et al., 2017).

The clinical specialty learning and teaching structure promoted here uses conceptual rather than prescriptive language. The higher‐order language is consistent with a capability learning context (Gardner et al., 2006b, 2008; O'Connell et al., 2014). This approach enables each nurse practitioner student with their clinical mentorship team to use metaspecialty‐based clinical practice standards to guide development of mutually agreed clinical speciality skills, knowledge, and expertise. The standards can be framed as individual learning objectives with key activities specified in clinical learning contracts. The structure ensures a consistent level of attainment while preparing graduates who will have workforce flexibility. Standards can be selected across metaspecialties, supporting learning for diverse scopes of practice. This flexibility contrasts with nurse practitioner clinical education in many other countries.

In the Australian context, this proposed structure of a national nurse practitioner clinical learning and teaching framework that comprises a metaspecialty taxonomy and clinical practice standards is complementary to other Australian work. Most particularly, the findings complement research by O’Connell et al (O'Connell, 2015; O'Connell & Gardner, 2012). O’Connell et al found that, while there was no single definable model of emergency nurse practitioners, there were common practice features across diverse emergency nurse practitioner models. The high standard content validity indices in our study across all nurse practitioner metaspecialties mirror this finding of common practice features within a specialty and while O’Connell et al's research was broader than clinical learning and teaching, their work is already informing nurse practitioner clinical practice in the specialty of emergency nursing.

These research outcomes have international and cross‐discipline application for development of specialty clinical practice standards. Methods and findings can be applied in other contexts, both nationally and internationally. This flexibility supports future novel role development internationally and is consistent with nurse practitioner roles in meeting unmet healthcare needs. Furthermore, there is potential for the methods and research findings to contribute to structure and content of professional development for endorsed nurse practitioners wanting to consolidate or expand their clinical practice. Notably, the standards have relevance for emerging advanced practice models in other health professions, given the lack of research on which to base development of other specialty roles and the lack of research‐based corresponding educational frameworks (Coombes et al., 2011; Morris et al., 2015).

Strengths and limitations

This complex two‐phase study comprised qualitative data collection and analysis to develop draft standards and six nested Delphi surveys in a single sophisticated web‐based survey platform with multiple rounds and high recruitment and retention of participants. However, the study had some limitations. Survey Phase eligibility criteria and recruitment have been tested previously (Helms et al., 2017a) but, similar to that study, no single accessible national nurse practitioner population database existed so it was not possible to confirm unequivocally the representativeness of the sample. Furthermore, the Survey Phase response rate may have been compromised because a few respondents encountered problems with the online format, related to older versions of computer operating systems. Importantly, identification of these difficulties has implications for the reliability of web‐based Delphi survey software generally (Helms et al., 2017b). The short timeframe between rounds may have reduced retention across rounds somewhat.

RECOMMENDATIONS AND CONCLUSION

The outcome of this research is profession‐wide consensus on clinical practice standards for each of six Australian nurse practitioner metaspecialties. This research‐derived suite of standards provides a broad metaspecialty‐based clinical learning and teaching structure to guide nurse practitioner student clinical education. The full list of standards is available under Creative Commons licence (Gardner et al., 2019) and is already being used to guide clinical learning for Australian nurse practitioner students. The standards will strengthen the quality of nurse practitioner clinical education in Australia and will have wide applicability internationally. Given the dearth of clinical educational structures for other similar advanced specialty healthcare roles both in Australia and internationally, the findings have wide‐ranging international relevance.

CONFLICT OF INTEREST

No conflict of interest has been declared by the authors.

AUTHOR CONTRIBUTIONS

All authors have agreed on the final version and meet both of the following criteria (recommended by the ICMJE; http://www.icmje.org/recommendations/):

substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data;

drafting the article or revising it critically for important intellectual content.

Appendix S1

Click here for additional data file.