Attributes of innovations and approaches to scalability - lessons from a national program to extend the scope of practice of health professionals.

The context for the paper was the evaluation of a national program in Australia to investigate extended scopes of practice for health professionals (paramedics, physiotherapists, and nurses). The design of the evaluation involved a mixed-methods approach with multiple data sources. Four multidisciplinary models of extended scope of practice were tested over an 18-month period, involving 26 organizations, 224 health professionals, and 36 implementation sites. The evaluation focused on what could be learned to inform scaling up the extended scopes of practice on a national scale. The evaluation findings were used to develop a conceptual framework for use by clinicians, managers, and policy makers to determine appropriate strategies for scaling up effective innovations. Development of the framework was informed by the literature on the diffusion of innovations, particularly an understanding that certain attributes of innovations influence adoption. The framework recognizes the role played by three groups of stakeholders: evidence producers, evidence influencers, and evidence adopters. The use of the framework is illustrated with four case studies from the evaluation. The findings demonstrate how the scaling up of innovations can be influenced by three quite distinct approaches - letting adoption take place in an uncontrolled, unplanned, way; actively helping the process of adoption; or taking deliberate steps to ensure that adoption takes place. Development of the conceptual framework resulted in two sets of questions to guide decisions about scalability, one for those considering whether to adopt the innovation (evidence adopters), and the other for those trying to decide on the optimal strategy for dissemination (evidence influencers).

Introduction

The catalyst for this paper was the evaluation of a national program, the Expanded Scopes of Practice (ESOP) Program, to extend the scope of practice of health professionals in Australia through introducing new multidisciplinary models of care. The focus of the program was to investigate the extent to which each role could be implemented on a national scale and the conditions under which the roles would be most likely to succeed.

Twenty-six organizations were funded across four sub-projects, involving extended roles for paramedics in primary care, physiotherapists in emergency departments (EDs), nurses conducting endoscopies, and nurses in EDs. In total, 224 health professionals participated, at 36 sites.

The primary aim of this paper is not to report on the results of the evaluation, which are available in a series of comprehensive reports.1–4 Rather, it is to describe how the results and lessons learned from evaluating the program were used to develop a conceptual framework for determining how to scale up innovations. Planning for “scaling up” is often overlooked.5

The framework (the “Conceptual Framework for Scaling up Innovations”) draws in part on a systematic review of the literature on the diffusion of innovations.6 We illustrate the use of the framework with case studies from the program evaluation. The relevant terms used in the paper are defined in Table 1.

Table 1

Definitions of terms

Term	Definition
Diffusion	The passive, untargeted, unplanned, and uncontrolled spread of new interventions.33
Dissemination	An active approach of spreading evidence-based interventions to the target audience via determined channels using planned strategies.33
Innovation	An idea, practice, or object that is perceived as new by an individual or other unit of adoption.12
Scaling up	Efforts to increase the impact of innovations successfully tested in pilot or experimental projects so as to benefit more people and to foster policy and program development on a lasting basis.34
Scaling up strategy	The means by which the innovation is communicated, transferred, or otherwise promoted.13

Evaluation methods

Evaluation of the ESOP program took place between 2012 and 2014, using a mixed-methods approach. The evaluation was based on a framework that has been used for over 10 years to evaluate a wide variety of health programs. The framework recognizes that implementation is influenced by the setting in which it takes place, the individuals involved, and the processes by which implementation is accomplished. The framework includes an evaluation hierarchy that focuses on outcomes for consumers, providers, and the care delivery system and is structured according to six domains: program delivery, impact, sustainability, capacity building, generalizability, and dissemination.7

Quantitative data were obtained from routine administrative data sets at three time points (baseline, implementation, and postimplementation) for more than 29,000 patients treated by the extended scope practitioners. Surveys were distributed to a sample of patients to collect data on patient experiences and satisfaction with the care provided. A 20-item questionnaire was sent to those working in extended roles to elicit their experiences, including job satisfaction, relationships with other staff, consumer acceptability, and their opinions on the sustainability of the roles. Professional colleagues of those working in extended roles were surveyed with a 15-item questionnaire designed to collect data on their understanding, opinions, and attitudes regarding the extended roles. Qualitative data were collected from documentation produced by each project and during site visits. Semistructured interviews were conducted with purposively sampled key stakeholders and each extended scope practitioner.

Interviews were recorded and transcribed. Transcripts were imported into NVivo software to facilitate data analysis. Thematic analysis was undertaken, starting with an initial set of codes informed by the evaluation framework that was amended and refined as data analysis progressed with inclusion of additional codes developed inductively to ensure the best fit with the data. Quantitative data were analyzed using Excel, SAS 9.2, SPSS (Chicago, IL, USA) and relevant comparisons made across time and implementation site. Data for each survey were compiled into one worksheet and checked by members of the evaluation team prior to analysis. The evaluation was approved by the University of Wollongong/South Eastern Sydney and Illawarra Area Health Service Human Research Ethics Committee and consent was implied by the completion of the survey.

Conceptual framework

Taking an innovation that has been tested in one or more locations and getting that innovation adopted more widely can be framed in various ways. From the perspective of researchers, the issue is one of generalizability, the process of conducting studies so that general conclusions can be drawn from particular instances,8 of which there are three types: statistical generalization, analytic generalization, and case-to-case translation (transferability).9 Within the context of the ESOP program, the most relevant type was that of transferability, ie, an innovation in one setting is considered for adoption in another setting.

From the perspective of policy makers and decision makers, the issue of scaling up innovations essentially involves a decision about their role. Do they act passively, allowing the innovation to be implemented in an uncontrolled, unplanned, way, or do they take a more active role by “pushing” the innovation by the various means at their disposal. It has been suggested that this can be conceptualized as a continuum, ranging from pure diffusion to active dissemination, involving three main mechanisms:

“Let it happen”: Allowing innovations to be adopted in a “natural” way, with individual organizations making their own decisions about whether to adopt an innovation. This approach is unpredictable and self-organizing.

“Help it happen”: The process of innovation adoption is facilitated, influenced, and enabled.

“Make it happen”: The adoption of innovations is managed formally, typically by a central agency.6

Øvretveit,10 drawing on the international health literature, also identified three approaches: facilitated evolution, participatory adaptation, and hierarchical control. Facilitated evolution has been equated with “let it happen”, participatory adaptation with “help it happen”, and hierarchical control with “making it happen”.11

From the perspective of those thinking of adopting an innovation, there are many issues to consider. For example, do they have the capacity to implement the innovation, how will the innovation “fit” with existing services, and how acceptable will the innovation be to other members of the health care team or the wider health care professions? The evidence suggests that certain attributes of an innovation can influence adoption and implementation:

Relative advantage – The degree to which the innovation is better than what is in place already.

Compatibility – The innovation is compatible with the values and perceived needs of the adopting organization.

Complexity – The innovation is relatively simple. If the innovation is relatively complex, it helps if it can be broken down and implemented in stages.

Trialability – The innovation can be “tried out” before full adoption.

Observability – The benefits of the innovation (to either consumers or staff) are visible.

Adaptability – The innovation can be adapted for local use.

Risk – The innovation is perceived as low risk.6,12

These different perspectives indicate that three stakeholder groups should be involved in scaling up innovations, presented schematically in Figure 1:

Figure 1

Conceptual framework for scaling up innovations.

Evidence producers: Those involved in researching or evaluating an innovation, who not only have to conduct their work rigorously but also present their findings in a way that is useful to others.

Evidence adopters: Those considering implementation of an innovation that has been tested elsewhere.

Evidence influencers: Those in a position to facilitate scaling up.

The field of international health has examples of approaches to “scaling up” innovations, particularly in public health, that align with this conceptual framework.13 Support for the framework also comes from a literature review which identified three categories of barriers hindering dissemination: 1) the research or evaluation design; 2) characteristics of the innovation; and 3) characteristics of the target setting.14 The underlying premise of the framework is that understanding innovation attributes and what is involved in implementation are part of deciding whether to make it happen, help it happen, or let it happen. In the context of the evaluation, it was particularly important to understand the impact of the innovation on other health care providers and members of the multidisciplinary health care team.1–4

Case study 1: extending the role of paramedics

This model centered on extended care paramedics (ECPs) managing lower acuity patients in their usual place of residence, with the potential to reduce the number of patients transported to hospital. Existing evidence generally supports the model but is primarily from overseas, particularly the United Kingdom, and more research is required to establish the effectiveness and safety of the model.15,16

The ECP role was relatively standardized, with local variations. At three sites, it operated in a solo capacity using a vehicle without the capacity to transport patients; at one site, the ECP worked with another paramedic as part of an existing emergency response service; and at another site, the ECP role was combined with another paramedic role. Training included a structured program of face-to-face teaching, simulations, clinical placements, and supervised practice.

The evaluation findings, framed in terms of “innovation attributes” are summarized in Table 2. The role worked well for patients (eg, less likelihood of being transported to hospital unnecessarily) but presented some difficulties for individual paramedics (eg, maintaining intensive care skills) and the ambulance service more generally (eg, the role requires sufficient throughput to reduce costs). The main requirements for successful implementation were as follows:

Table 2

Innovation attributes of the ECP model

Innovation attributes	Evaluation findings
Relative advantage	A high proportion (72.5%) of patients seen by ECPs did not require transport to hospital. Cost-effectiveness depended on sufficient throughput of suitable patients identified in an efficient manner. The costs of implementing the model were met by ambulance services, but any cost savings accrued to the health system more generally, particularly hospitals (because of reduced transfers to hospital).
Compatibility	The practice of ECPs was compatible with current practice of ambulance paramedics. From an organizational perspective, the major issue of “compatibility” related to throughput. With sufficient throughput, a sole ECP can work in a specially equipped vehicle with no patient transport capability, quite separate from existing emergency response crews. If throughput is less, the ECP role has to be combined with another role (eg, the existing emergency response service).
Complexity	The ECPs managed patients with diverse, and often ill-defined, signs and symptoms. Although much of this work can be considered as “low acuity”, it can also be quite complex, requiring advanced clinical reasoning. This requires relatively in-depth training, with mentoring and supervision by medical practitioners or experienced ECPs, which may be difficult to provide adequately. The ECP role required highly experienced ambulance paramedics with appropriate qualifications.
Trialability	The model is difficult to “try out” without a significant investment of time, money, and stakeholder engagement. The cost of training each ECP was estimated at AUD30,000. In the absence of prior experience implementing the role, the results of the evaluation indicated that 12–18 months are required to establish systems, structures, and processes before any patients benefit.
Observability	The benefits of the model were “visible” to ECPs and those they treated, with strong agreement among ECPs that their role improved quality of care for specific patient groups and very high levels of consumer satisfaction with the model. Few patients refused treatment by an ECP.
Adaptability	At most sites, the caseload was too small to warrant a full-time, stand-alone position. A hybrid role was seen by most ECPs as more satisfying and efficient in rural and regional locations, with the added advantage of allowing ECPs to maintain their intensive care skills. The stand-alone ECP model may be more viable in large metropolitan locations that generate higher caseloads or in localities where the supply of other primary health practitioners is limited.
Risk	The model is low risk, with small likelihood of adverse outcomes, as long as strict clinical governance arrangements are in place, particularly supportive medical supervision. In addition to being highly experienced, carefully selected, and comprehensively trained, ECPs had a distinct set of personal characteristics and attributes that were seen to promote safe practice. Key stakeholders were satisfied that the model operated safely and offered a very high level of quality in patient care. This was reinforced by the available information from administrative data sets.

Abbreviation: ECPs, extended care paramedics.

A receptive context for change.

Selecting personnel with the necessary skills, experience, and personal characteristics for the role.

Overcoming structural barriers such as funding models and role classification.2

A receptive context for change, a term used throughout this paper, includes factors such as a supportive organizational culture, key people leading change, clear goals, and appropriate infrastructure.17,18 The role of medical mentors was pivotal to successful implementation, providing ongoing support, clinical supervision, telephone advice, and backup for the ECPs.2

There were some legislative barriers to realizing the full potential of the role. For example, in some states and territories, amendments to legislation would be required for ECPs to prescribe medications and remove restrictions on the use and mobile storage of pharmaceuticals such as antibiotics. The inability to supply medicines meant that some patients had to be transported to an ED or general practitioner for this purpose.

Case study 2: extending the role of physiotherapists in emergency departments

Traditionally, physiotherapists assess and treat patients in EDs after initial assessment and referral by a medical practitioner (a secondary contact role). In contrast, this sub-project involved primary contact physiotherapists (PCPs) assessing and treating patients with musculoskeletal (MSK) conditions without medical intervention, unless clinically indicated. Published evidence to support the PCP role is quite limited.19–23

The evaluation findings, summarized in Table 3, indicate that the role works well for patients (eg, satisfaction with care by the PCPs). Clinically, the PCP model is compatible with current practice, but from the perspective of the care delivery system, it presented some challenges, primarily the need for sufficient throughput of low-acuity patients with MSK conditions to ensure efficiency. The number of such patients represented a quarter of all ED presentations, suggesting plenty of scope for PCPs to be utilized. However, PCPs treated only 9.5% of these presentations, with the low percentage due to three main factors: 1) patients with MSK conditions presenting when PCPs were not rostered on-duty; 2) occasions when there were too many patients with MSK conditions to be all seen by the available PCPs; and 3) “competition” for patients from other health professionals in the ED (medical staff and nurse practitioners).

Table 3

Innovation attributes of the PCP model

Attribute	Findings
Relative advantage	The waiting times, treatment times, and lengths of stay for MSK patients treated by PCPs were shorter than for patients treated by other clinicians. Evaluation of cost-efficiency was limited by the lack of available data. The model may help reduce resource use in the area of X-ray ordering by facilitating more prompt and expert assessment of patients with suspected fractures. On weekdays when PCPs were rostered on, ED performance improved and patient throughput was higher.
Compatibility	The practice of PCPs is compatible with current physiotherapy and ED practice. The model requires physiotherapists to change their thinking from one of accepting referrals to one of seeking out referrals. The PCP model can be introduced as a separate model, or combined with an existing secondary contact physiotherapy service.
Complexity	The practice of the PCPs was largely restricted to a well-defined group of patients with MSK conditions. The training is relatively complex, but can be broken down into smaller parts. This can include an early focus on key competencies to facilitate commencement of PCP practice and reduced need for supervision.
Trialability	The model can be “tried out” by slowly increasing the skills and expertise of existing staff to take on increasing responsibility for the patient cohort as their competencies develop.
Observability	There was strong agreement among PCPs that their role improved quality of care for MSK conditions. The PCP role was strongly endorsed by colleagues who were satisfied that the model was safe and improved quality and efficiency. Patients reported good experiences and high levels of satisfaction with the care they received.
Adaptability	The arrangements for supporting the PCP model can be adapted for local use. The available training pathways were appropriate, but there is the potential for the pathways to be more flexible so as not to limit the number of physiotherapists who are suitable for the role. Medical staff can be replaced as assessors of clinical competence by an experienced and suitably qualified PCP.
Risk	Based on limited data, re-presentations to the same ED for the same health condition were similar for PCPs and other practitioners. The number of unexpected deaths was similar for the baseline and implementation periods and decreased postimplementation. All PCPs were experienced clinicians. Stakeholders were confident that the model was safe and that PCPs were working within their scope of practice. Some senior doctors emphasized the importance of medical oversight and PCPs themselves demonstrated willingness to seek advice and refer as needed.

Abbreviations: PCP, primary contact physiotherapist; MSK, musculoskeletal; ED, emergency department.

The evaluation indicated that the main requirements for successful implementation were a receptive context for change and the availability of staff with the necessary skills.1 The key group for facilitating a receptive context was medical staff in the ED, by providing general support for the model and practical assistance in the form of mentoring, supervision, and assessment of clinical competencies.

Implementation required close collaboration with other disciplines working within the EDs. Most clinicians working in EDs are “generalists”, ie, they are capable of assessing and managing all types of patients who present. Introducing the PCP role, which only assessed and treated a specific patient cohort, introduced a complexity that was not present before.1

Legislation restricted full implementation of the role at some sites, eg, limitations on administering and prescribing medications.

Case study 3: extending the role of nurses to conduct endoscopies

The aim of this subproject was to train nurses to perform endoscopies previously only undertaken in Australia by doctors. Research from other countries indicates that for simple endoscopies nurses can achieve similar results for efficacy and safety to those achieved by doctors.24

Nurses were trained at five sites where they functioned as part of multidisciplinary teams, primarily medical, and other nursing staff. The focus of their training was almost entirely on colonoscopies. Two models of practice were implemented: one framed in terms of advanced practice nursing, and the other involving nurses training to become nurse practitioners. The net cost of training each nurse was estimated to be almost A$90,000.3

By the time the evaluation concluded, full implementation was not achieved, with some trainees still working to complete the required number of procedures and be assessed as competent. The relative advantage (effectiveness and cost effectiveness), observability (visible benefits), and risks of the model could therefore not be properly evaluated. The extensive training requirements mean that the role cannot be “tried out”, but once competency is achieved, the role is relatively straightforward. The role can be varied to meet local circumstances, but there are core elements that cannot be varied, particularly the training requirements, need for medical mentoring and supervision, and dedicated resources for performing endoscopies. Both models of nursing practice are acceptable, with the preferred model depending on local factors, particularly the need for nurses to work in an extended role (eg, prescribing, ordering pathology).

The main factors influencing implementation were a receptive context for change, selecting appropriate personnel, and the availability of a well-structured, well-resourced training program with strong medical supervision and mentorship supported by an ongoing credentialing process. Important characteristics of a receptive context were management support at all levels of the organization, engagement and support of key medical leaders, and adequate resources.3

The evaluation indicated two fundamental problems limiting wider implementation. First, training requires a critical mass of medical proceduralists to provide the necessary mentoring and clinical supervision. The presence of a critical mass reduces the need for nurse endoscopists once they are trained. Locations lacking medical proceduralists may have greater need for nurse endoscopists but are not well placed to provide the training. Second, only about a quarter of same-day colonoscopies are performed in public hospitals in Australia, with the remainder performed in private facilities. Doctors interviewed for the evaluation indicated fierce opposition to nurses performing colonoscopies (or any type of endoscopy) in the private sector.

Case study 4: advancing the role of nurses in emergency departments

This subproject involved a diverse range of organizations implementing different models of care to meet local needs with the common goal of improving patient flow through EDs. Of the eight projects, three targeted mental health patients, two targeted pediatric patients, two addressed the needs of rural hospitals, and one established a “review” clinic to follow-up patients initially seen in the ED. Six projects involved registered nurses working within a framework of clinical guidelines, protocols, and pathways, which were little different from the incremental expansion of nursing roles that has been occurring for many years. Two projects focused on the employment of nurse practitioners or nurses training to become nurse practitioners.

The projects were relatively small in scale, with the diversity and limited reach of the projects preventing a comparative analysis of incremental costs and consequences. Hence, the relative advantage and observability of each model was not established. The evaluation indicated that the models are compatible with accepted practice; the changes are relatively simple and can be “tried out”; the models can be adapted for local use; and the risks are low, as long as suitable clinical governance arrangements are in place. Any barriers to implementation were largely intraorganizational, rather than arising from the broader economic, policy, and legislative environment. An important enabling factor was the ability to work with other members of the health care team, either within the EDs or from other services, eg, liaison psychiatry service.4

Discussion

Across all four case studies, the evaluation findings indicated two key factors facilitating implementation; first, a receptive context for change, particularly the support of key medical staff and availability of appropriate infrastructure. Some of this “receptivity” can be influenced from outside but much of it relies on the intraorganizational environment. Second, the characteristics of the people working in the extended role, including their knowledge, skills, and experience. These findings, and our overall experience conducting the evaluation, became the basis for developing two sets of questions to guide decisions about scalability, one for evidence adopters and the other for evidence influencers (Table 4). For evidence influencers, answering the questions can assist in deciding whether to support the innovation and how to support the innovation. The questions have a different focus and use of language, and are framed in a different way, but the intent is consistent with the approach taken in the Institute for Healthcare Improvement’s “spread planner”.25 For example, the “spread planner” includes questions about the availability of resources and leadership to support the spread of innovations and the alignment of an innovation with organizational goals and incentives.

Table 4

Questions for scalability

Questions to be answered by evidence adopters
Is the “adopting” organization likely to be receptive to the innovation?
Can the innovation deliver care that is as safe as, and of equivalent quality (or better) than, usual care?
Will the innovation lead to greater efficiency or productivity?
Is the innovation cost-effective or at least cost neutral?
Is there a critical mass of appropriately qualified and experienced personnel who can fill, or be trained to fill, the expanded role?
Are appropriately qualified personnel available to provide supervision and support when required?
Is the innovation compatible with current practice?
How will the innovation “fit” with current service provision?
Does the innovation need to be adapted to meet local circumstances (including any adaptation to ensure sufficient throughput)?
Can the innovation be “tried out” before full adoption?
Questions to be answered by evidence influencers
Are there health services that are likely to be receptive to the innovation eg, management support, support from clinical leaders, recognized need for change?
Are there health services with the necessary infrastructure (eg, resources, structures, training capability) to support the innovation?
Will the innovation lead to greater efficiency or productivity?
Is the innovation cost-effective or at least cost neutral?
Are any legislative changes required to facilitate the innovation?
Are there any potential economies of scale, eg, in the provision of training and skills development?
Does the current funding system support the innovation?
How does the innovation align with current policy priorities?
Do professional bodies support the innovation?
Are there industrial implications arising from the introduction of this innovation?

The conceptual framework, answers to the scalability questions, and the evaluation findings suggest appropriate scaling up strategies for each model.

Extending the role of paramedics

In Australia, ambulance services are organized at the level of states and territories, managed fairly directly by the relevant government, ie, there are close links between evidence adopters (ambulance services) and evidence influencers (government). The evaluation findings provided support for the ECP role but indicated that adoption of the role requires careful thought regarding potential impact on individual paramedics and how the role fits within existing services. These findings suggest that decisions to adopt the role should be taken centrally by government and ambulance service leaders with a “make it happen” approach, leaving room for local variation in how the role is implemented.

Extending the role of physiotherapists in emergency departments

The evaluation findings indicated that the PCP role has the potential to provide a “relative advantage” over existing practice, with a caveat regarding the lack of evidence of cost effectiveness. The importance of local factors for successful implementation suggests that trying to impose the PCP role with a “make it happen” approach is likely to be unsuccessful. The specialist nature of the role requires careful consideration about how it “fits” with the existing practices to ensure efficiency.

One option for evidence influencers is to take a “let it happen” approach, leaving it up to evidence adopters to decide at a local level whether they wish to implement the role. However, another option is to take more of a “help it happen” approach. EDs in Australia are located almost entirely within publicly funded hospitals organized on a geographic basis. In the case of the PCP role, evidence influencers are likely to be found in government health departments or senior roles within local health services, with knowledge of hospitals likely to be receptive to the role. They will be in a position to identify any funding or legislative changes required to support the role and judge any potential economies of scale in training physiotherapists.

The significant resources developed in the ESOP program to train PCPs will require constant updating to ensure relevance, another example of where “help” can be provided. Other potential ways to “help” wider implementation include funding for hospitals that have already implemented the role to support other sites to implement the role. The need for further evidence to support wider implementation indicates a role for evidence producers (to conduct the research and disseminate the results) and evidence influencers (by funding the research and supporting dissemination).

Extending the role of nurses to conduct endoscopies

The evaluation findings indicate that wider implementation of nurse endoscopists would require considerable medical support and involve large training costs. The evaluation highlighted a fundamental contradiction – health services in the best position to train nurse endoscopists are in the least favorable position to justify the role once the nurses are trained.

This situation indicates a role for evidence influencers to take a “help it happen” approach with various forms of practical assistance, including funding to support implementation and development of a strategy, where some health services may train nurse endoscopists to work in other health services once they are trained.

The evaluation left some of the scalability questions unanswered, particularly regarding the relative advantage and productivity of the role, primarily because full implementation was not achieved during the timeframe of the program. Further research to answer these questions would support more informed decisions about wider implementation of the role, again indicating a role for evidence producers and evidence influencers.

Advancing the role of nurses in emergency departments

These models were diverse and relatively small in scale, limiting the ability to judge the merits of each model. However, the issue of increasing the scope of practice of nurses in EDs is an important one, particularly in rural and remote locations where securing adequate medical coverage can be problematic. There is an extensive literature on the merits of extending the role of nurses working in EDs, either as nurse practitioners26 or other extended roles.27,28 Rather than continuing to conduct small-scale local projects, it may be preferable to take a more strategic “make it happen” approach by implementing a particular model on a larger scale. This is already happening in one state (New South Wales) where registered nurses working in small rural hospitals are expected to have completed the First Line Emergency Care Course for Registered Nurses and function as advanced clinical nurses in accordance with specially formulated clinical guidelines.

Conclusion

The Conceptual Framework for Scaling up Innovations was developed iteratively during the evaluation of the ESOP program, based on our existing knowledge of the literature and evolving understanding of the projects we were evaluating. Many conceptual frameworks for disseminating research findings currently exist, with a tendency to emphasize the role of evidence producers.29–31 There is a lack of emphasis in existing frameworks on the tripartite nature of knowledge exchange (evidence producers, evidence influencers, and evidence adopters) and little basis for helping evidence influencers to decide what they should do to facilitate the scaling up of innovations. This is the key contribution of our paper to improving knowledge of this challenging topic. Although developed within the context of a program on workforce reform, the framework has the potential to be applied to other health service innovations.

The “questions for scalability” were based on the conceptual framework and informed by the evaluation findings, providing a practical means of assisting evidence adopters and evidence influencers. For the evidence influencers, answering the questions helps the process of deciding what approach to take on the continuum from pure diffusion (“let it happen”) to active dissemination (“make it happen”). Examples of how this might be done were presented in the form of the four case studies. Answering the scalability questions can improve planning for the introduction of new extended roles that can help to realize the potential for such roles.32

Implementing effective health workforce reform is hard work. The right investment has the potential to deliver better care for consumers, optimize the skill mix and job satisfaction of the health workforce, and generate productivity benefits for the health system. The wrong investment has the potential to waste valuable resources. When policy makers consider the potential for scaling up health workforce innovations, they need to consider three main issues: 1) the attributes of an innovation (eg, relative advantage, adaptability); 2) the optimal sites for implementation; and 3) the broader economic, policy, and legislative environment within which implementation will take place. Our framework provides policy makers with a tool that can be used to consider these three issues in a systematic way.

The knowledge that attributes of innovations such as relative advantage and complexity can influence adoption is well known. This suggests that, from the perspective of potential adopters, it would be advantageous if the findings from research and evaluation were framed in this way. Our experience over many years of reading the results of research studies and evaluations is that this does not occur. For the evaluation of the ESOP program, we found this to be a useful exercise, not only in part to synthesize the results of a long and complex evaluation in a concise and useable way but also because of the direction this gave regarding scalability to policy makers.