Development of the PriCARE classification for potentially preventable emergency department visits by ambulance: a RAND/UCLA modified Delphi study protocol.

INTRODUCTION: Ontario ambulances are restricted from patient transportation to sub-acute levels of care when these facilities may be more suitable than emergency departments for non-emergent conditions. There is no known patient classification specifically constructed to inform ED diversion protocols and guidance for sub-acute centre transportation for primary care-like patient conditions.
OBJECTIVE: To construct a novel patient classification of potentially preventable emergency department visits following transport by ambulance, and analyse patient-level characteristic associations with this classification based in Ontario secondary data. METHODS AND ANALYSIS: The Primary Care-like Ambulance transports following Response for 911-Emergencies (PriCARE) patient classification will be constructed using a two-phase RAND/UCLA modified Delphi design. All experts included are physicians with relevant experience in emergency and/or primary care in Ontario. The first phase of the study will determine consensus of the expert committee on which ED interventions performed on patients with non-emergent acuities could be conducted in sub-acute healthcare centres. The second phase will assess consensus of which patient, hospital and acuity factors are most appropriate to be incorporated into a PriCARE classification. We will also investigate secondary outcomes on consensus of which ED interventions could be transferred to a paramedic context given an expanded scope of practice and patient-level characteristics of PriCARE classified individuals. ETHICS AND DISSEMINATION: This study received a research ethics board exemption waiver from the Hamilton Integrated Research Ethics Board; review reference 2020-11451-GRA. Results will be submitted for publication in a peer-reviewed journal and presented at relevant conferences. The results will be shared with Ontario paramedic services and governing institutions. This study will be used to inform patient classification protocols and clinical decision tools for ambulances to transport to sub-acute healthcare centres. TRIAL REGISTRATION NUMBER: ISRCTN22901977. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The primary objective of this study will be to develop and construct criterion of an ambulance-specific patient classification for potentially preventable emergency department transports.

The RAND/UCLA modified Delphi method is used comprehensively in healthcare fields, but this protocol represents the first application in the paramedicine field for constructing a patient classification.

Expert confidentiality will be maintained in round results by providing feedback as aggregate data only, which will minimise individual pressure to conform to groups.

Purposive sampling will be used to identify experts interested in out-of-hospital care and patient classifications, which may differ from those who decline to participate.

Introduction

Ontario’s emergency departments (EDs) are increasingly overburdened with demand for healthcare services that exceed their ability to provide consistent high-quality and timely care.1 2 In addition, many EDs are congested with non-emergent patients, when alternative care centres for their conditions may be more beneficial in aligning chronic condition care, strengthening their relationships with primary care3 4 and are more cost-effective.5 Despite advancements in administrative, political, health policy, access to primary care and public awareness initiatives for primary care conditions, EDs continue to remain overused with longer time intervals in initial physician assessment, care received, discharge and overall wait time benchmarks.1 6 7

Ambulance transport diversion from emergency departments to sub-acute centres do not exist in Ontario, Canada for patients who call for a 911 response and yield non-emergent conditions. Conceptually, this model of care could improve several healthcare domains such as hospital utilisation, ambulance utilisation, patient navigation and quality of care. Urgent care centres and other sub-acute healthcare units have been found to decrease the proportion of low-acuity ED diagnoses,8 and conceptually provide similar services to EDs but in a more limited capacity.9

In November 2019, the Ministry of Health in Ontario amended Regulation 257/00 under the Ambulance Act to incorporate new models of care for paramedics. New models could include transportation to specialty clinics or community-based providers based on patient needs.10 The objective was to aid in alleviating ED workload by redirecting patients with non-emergent or primary care–like conditions to alternative care units. To this end, a new model of care could include granting paramedics the authority to make systematised ambulance transport decisions for potentially preventable ED patients. The Institute for Clinical Evaluative Sciences (IC/ES) database reports ambulance-transported patients account for approximately 20% of all ED visits (1 January 2014 to 31 March 2018). Sixty per cent (2 001 108/3 345 991) of all ambulance-transported patients were scored non-emergent acuities following an emergency 911 response request, with 74% discharged the same day (1 474 212/2 001 108). The extent to which paramedics could reliably identify potentially preventable ED patients using a validated clinical pathway or classification algorithm in the out-of-hospital setting is unknown nor developed.11

There are several published classifications to categorise patients as a potentially preventable ED visit in a multitude of settings; however, there is no consensus on a definition of a ‘potentially preventable ED visit’.12–15 In addition, high heterogeneity exists among potentially preventable classification criteria and do not uniformly account for criterion such as the main intervention a patient requires. Several existing classifications rely heavily on diagnostics for inclusion, which may not apply to the specific nature of a presenting complaint where a 911 call was placed or be known to paramedics in the out-of-hospital setting. The validity and indirectness of applying classifications that were constructed only for in-hospital contexts to the out-of-hospital setting are also in question.16

In the absence of a gold standard to classify ambulance ED visits which could be potentially preventable for primary care–like conditions, a novel classification would be beneficial for increasing the understanding of out-of-hospital patient categorisation and informing future clinical protocols. To streamline the applicability and feasibility of such a patient classification, construction must first be undertaken for epidemiological purposes and prior to prospective validation. Furthermore, an ambulance patient classification should focus on ambulance-transported patients primarily, the specific nature of the ED visit and the units of differing alternative healthcare facilities (not treat-on-site without transport). A new ambulance-relevant classification for potentially preventable ED visits would, if proven valid and reliable, support new models of preventative care as well as ambulance diversion to sub-acute centres.

Study objectives

The primary objective of this study is to construct the patient classification Primary Care–like Ambulance transports following Response for 911-Emergencies (PriCARE) for potentially preventable ED ambulance transported visits based on the consensus of an expert physician committee. This study will examine which main ED physician interventions could be conducted in sub-acute centres of Ontario by assessing the determinations of an expert committee on a comprehensive list of Canadian Classification of Health Interventions (CCI) codes. In addition, expert consensus will be assessed on which variation of a PriCARE classification, assembled based on results of CCI codes, is considered most appropriate for representing ambulance transported patients who could be considered to have a primary care–like visit, as opposed to an emergency care visit. Expert assessment of the most appropriate PriCARE classification should yield the highest clinical specificity to identifying patients that have a visit disposition of same-day discharge, no hospital admission and no mortality in ED. The sub-acute centres of this study will include urgent care centres, walk-in medical centres and nurse practitioner–led clinics.

Secondary objectives of this study will analyse which adult patient-level characteristics associate with a PriCARE classification in Ontario secondary data using the National Ambulatory Care Reporting System (NACRS) ED database. In addition, consensus from experts will be analysed for which ED interventions could be transferred to a paramedic context given a reasonable increase in paramedic scope of practice and training.

Methods

Study design

This study will use a RAND/UCLA modified Delphi methodology to assess the consensus of a technical expert committee through a two-phase development and evaluation model.17 Phase I will determine which ED interventions could be conducted in Ontario sub-acute healthcare centres. Phase II will evaluate criterion that should be incorporated into a PriCARE classification that is appropriate for representing a primary care–like visit and yields the highest specificity of visit disposition of a same-day discharge ED visit, with no hospital admission or mortality. This study was initiated in May 2020 and is anticipated to be completed by March 2021.

Study sample and eligibility criteria

The expert committee will be composed of experienced physicians with extensive knowledge in emergency medicine and/or primary care. Inclusion as an expert on the technical expert committee must meet the requirement of being an active healthcare physician who, at the time of the study, is involved in the care of patients in an ED and/or primary care centre in Ontario. Physicians were deemed the only clinician appropriate for inclusion as an expert on the committee as the ED CCI codes of this study are conducted by physicians.

Recruitment and consent

The expert committee will be recruited by study investigators using purposive sampling. Investigators will contact the potential expert committee participants individually to introduce the study and gauge their willingness to participate. An information letter and consent form will be provided for each potential participant. All experts will be given the opportunity to review, ask questions and sign the consent to participate. Once a participant has accepted, a recruitment questionnaire will be distributed to obtain demographic information in order to report aggregate characteristics of the technical expert committee.

Sample size

There is no set standard sample size for Delphi groups but is generally agreed that more members will increase the reliability of group judgements.18 The investigators estimate at least seven ED physicians and three primary care physicians will participate. The technical expert committee will likely consist of 10–18 participants, providing a well-rounded representation of their expertise in emergency and primary care.19

Designations

Primary care–like is defined as any healthcare provided by a trained clinician that is considered basic, as opposed to specialised, in a patient’s first point of contact for treatment.17 20 21 Primary care–like for this study will be concentrated on facilitating care when it is first needed, should include quality features such as effectiveness and safety, and should address personal healthcare needs while practising in a family or community context.14–16

Emergency departments are defined as units of a hospital where emergency medicine physicians primarily provide care for patients requiring expeditious evaluation, diagnosis and/or treatment for medical, surgical or psychiatric care.22 EDs will contain all infrastructure (professional and technological) required for assessment, resuscitation, stabilisation and admission of emergently ill or injured patients.22

Although sub-acute centres have provided healthcare in Ontario for over a decade, the standardisation of services in urgent care centres, walk-in medical centres and nurse practitioner–led clinics have not been uniformly defined. Definitions differ within centre-to-centre comparisons and between groups of similar sub-acute centres, but are consistent in their mission statements to treat conditions that are urgent to non-urgent in acuity and ambulances are not accepted.9 23 Thus, for the purposes of this study, a standardised definition of each sub-acute centre is required when granulations in models of care and services may exist. All definitions are constructed based on the Ontario Ministry of Health and sub-acute centre expert consultations.

Sub-acute care units are defined as healthcare facilities or centres that provide primary care and/or multidisciplinary care for patients who do not require immediate or acute hospital attention.

Urgent care centres are defined as healthcare units (either located in a hospital facility or stand-alone) that are primarily staffed by emergency physicians and provide healthcare services for the treatment of illnesses or injuries that would not result in disability or death when not receiving immediate care.9 Urgent care centres will contain essential healthcare personnel and diagnostic equipment (including X-ray, ultrasound and blood laboratory services) to address personal healthcare needs while practising in primary care contexts, and will concentrate on facilitating care when it is first needed.16–18

Walk-in medical centres are defined as healthcare units that address non-urgent patient conditions requiring care for minor illnesses or injuries.23 Walk-in medical centres will contain a physician, will address and treat primary care patient conditions, and have use of diagnostics, referrals, prescriptions and medical interventions while operating with limited diagnostic technological capabilities.23

Nurse practitioner–led clinics are defined as healthcare units that focus on providing ongoing healthcare, education and illness prevention for patients with primary care conditions.24 Services include diagnostics, treatment, prescriptions and testing from nurse practitioners exclusively.24

ED interventions

The comprehensive list of ED interventions used in phase I will be constructed by the investigators from ~21 000 potential CCI codes of the NACRS 2018 index manual.25 The CCI codes included in this study will represent a comprehensive list of the most frequently used interventions for the treatment of adult patients aged 18 years or greater, with a Canadian Triage and Acuity Scale (CTAS) score of 3 to 5, and have been transported by ground ambulance to ED between years 2014 and 2018 in Ontario. The CTAS score is assigned on entry to ED by an emergency room or triage nurse. The year range of 2014 to 2018 was the most recently available 5-year period of the NACRS database at study initiation. External selection validity is preserved as NACRS contains a specific collection of hospital-based ambulatory care records, representing all available healthcare administrative reports from ED visits of the Ontario population. The total number of interventions to be included will be determined based on the most frequently used CCI codes that encompass 95% or greater of all interventions performed on this cohort. Also, 95% was deemed to be a comprehensive representation of the cohort by the investigators.

All access to the NACRS database will occur via the Institute for Clinical Evaluative Sciences IDAVE system.

Development of questionnaires

In the development phase (I) of this study, questionnaires will contain the comprehensive list of interventions to be ranked by each expert independently. Each expert will make a determination of which sub-acute centre, ranging from the ED to a nurse practitioner–led clinic, a given intervention could be conducted. Phase I questionnaires will also include a dimension asking experts to determine whether or not this ED main intervention could be transferred to a future paramedic context given an expansion in training and scope.

Following the completion of phase I, the evaluative phase (II) will assemble multiple variations of PriCARE based on differing degrees of patient, ED and acuity characteristics, as well as the interventions that received consensus from experts in phase I. Patient characteristics will include variables of age, sex, primary complaint category and comorbidities. ED characteristics will include ED length of stay, if a specialised physician is consulted and the ED intervention (consensus from phase I). The acuity characteristic will be the triage CTAS score assigned on entry to ED. Phase II questionnaires will contain several variations of a PriCARE classification to be ranked by each expert independently for appropriateness of the classification to capture primary care–like ED visits with high specificity to visit disposition outcome. Specificity will be addressed as a visit outcome of no admission, same-day discharge (within 12-hours) and no mortality in ED. Specificity will be considered by the experts for their rankings based on

True positive—a patient classified as PriCARE with a visit disposition of same-day discharge (no admission, or mortality).

False positive—a patient classified as PriCARE with a visit disposition of admitted, not same-day discharged or died in department.

True negative—a patient not classified as PriCARE with a visit disposition of admitted, not same-day discharged or died in department.

False negative—a patient not classified as PriCARE with a visit disposition of same-day discharge (no admission or mortality).

The ranking scale of phase II will use a 5-point Likert scale26 for appropriateness to correctly classify patients using PriCARE as meeting the expert’s judgement within a range from ‘Not Appropriate’ to ‘Highly Appropriate’.

Data collection

Data collection for this study will be primarily conducted using three methods: questionnaires, feedback and discussion with the technical expert committee. For both phases of the RAND/UCLA modified Delphi, questionnaires will be distributed to the technical expert committee by email.

For all questionnaires, a comment space will be available on each item for the expert to identify their rationale for determinations and rankings. After completing a questionnaire, the rounds will be returned to the lead investigator. A maximum of two rounds of questionnaires will be permitted for phase I, and as many rounds as required to reach consensus will be permitted for phase II. After phase I round 1, all interventions that received consensus will be removed from the subsequent round.

Feedback will be distributed by the investigators to the technical expert committee following each round with the overall aggregate results. The feedback form will include no less than a table with each item of individual result, overall ranking distribution, percentage agreement and reasoning. The feedback form will be anonymised; only their individual results and overall aggregate will be displayed.

Discussion with the technical expert committee will commence once a round has been completed (questionnaire and feedback form). All experts will be asked to participate in an expert committee videoconference debrief, which will be facilitated by the investigators. The videoconference will not be recorded or transcribed; however, the investigators may take notes.

Study steps

This study will use a two-phase RAND/UCLA modified Delphi methodology to complete the classification development of PriCARE, shown in the following steps of table 1.

Table 1

Description of the two-phase RAND/UCLA modified Delphi study steps to develop the PriCARE patient classification

Phase	Study step	Description
	Phase I questionnaire development	The investigators prepare a drafted version of the phase I questionnaire based on a comprehensive list of CCI interventions used in the NACRS database
	Phase I questionnaire refinement	The drafted version will be distributed to a pilot committee and investigators for edit and comment. Revisions made by the investigators
	Recruitment of expert committee	See Recruitment and consent for the details of recruitment
I	Round 1	The round 1 questionnaire will be distributed to each expert. Each will be asked to rate all interventions based on their applicability to be conducted in a sub-acute centre
	Analysis of round 1 and feedback	Results of round 1 will be collected by the investigators and transcribed into a feedback form that will be distributed to each expert committee participant (their individual scores compared against the aggregate)
	Videoconference debrief 1	A videoconference debrief will be held to discuss the rating results of round 1. Particular attention will be given to the interventions which did not receive consensus
	Round 2	The round 2 questionnaire containing all subsequent interventions which did not receive consensus will be distributed to the experts
	Analysis of round 2 and feedback	Results of round 2 will be collected by the investigators and transcribed into a feedback form that will be distributed to each expert committee participant
	Videoconference debrief 2	A videoconference debrief will be held to discuss the rating results of round 2
	Phase II questionnaire development	The investigators will develop the phase II questionnaire, which will use the results of phase I, with the addition of factors on patients, ED and acuity to create several variations of a PriCARE classification
II	Round 1	The round 1 questionnaire will be distributed to each expert. Experts will be asked to rank each variation of a PriCARE classification for appropriateness of capturing primary care–like ED visits that have a visit disposition of the desired outcomes. See Intervention and Data collection for further details
	Analysis of round 1 and feedback	Results of round 1 will be collected by the investigators and transcribed into a feedback form that will be distributed to each expert committee participant
	Videoconference debrief 1	A videoconference debrief will be held to discuss the ranking results of round 1
	Subsequent rounds	Subsequent rounds of questionnaires, analysis of results, feedback and videoconferencing will commence until a consensus of a PriCARE classification is reached
	Analysis of the classification	The PriCARE classification will be analysed in the IC/ES ED NACRS dataset for associations in patient-level characteristics

Analysis

Determining consensus

Phase I consensus will be reached when an intervention reaches 70% of expert ratings for a given healthcare centre. At consensus, this intervention will not receive any further discussion or rounds. Less than 70% agreement indicates an intervention will be present on the subsequent round of rating until either consensus is reached or a maximum of two rounds occurs. Phase II consensus will be reached when a PriCARE classification reaches 70% of experts within two levels, for as many rounds as required. Rankings of phase II questionnaires will be completed by each of the experts using a 5-point Likert Scale.26 Two-level achievement is defined as any ranking of 1 or 2, 2 or 3, 3 or 4, or 4 or 5.

Statistical analysis

The PriCARE classification will be analysed in a NACRS dataset for adult patients aged 18 years or greater and transported to ED by ambulance for patient-level characteristics using general frequency and central tendency descriptive statistics.

Data management

All data collected by the investigators will be analysed for producing results of each round. The RAND/UCLA modified Delphi ending result in phase I is a focused list of interventions that will build the PriCARE classifications in phase II. The investigators will have the only access to the raw data to process, analyse and prepare descriptive reports. McMaster University will act as the sole custodian of the data and will not be sent to any external institutions. All data are to be stored electronically with the lead investigator.

Each expert participant will be assigned a unique identification (ID) number, and all data will be stored under this ID. No participant-level data will be shared with anyone outside of the participant themselves, and all other data from this study will be presented as aggregate. No personal information or patient-level data are to be transferred within the study.

Patient and public involvement

The necessity to research an ambulance-specific patient classification was brought forth from a series of meetings with clinicians pertaining to paramedicine and primary care in late 2019, at the time when ambulance legislation was amended. Potential implications of this study’s findings were discussed with Ontario paramedics to gauge their satisfaction with this study’s methodologies. This protocol received input from paramedic services across Ontario, where frontline paramedics were asked focused questions on their perspectives of alternative healthcare transport centres. In addition, paramedics were asked for the estimations of patients they have assessed on whether or not ambulances should transport to alternative healthcare centres. All input helped to modify the study design.

Ethics and dissemination

Risk to participants

No known risks to experts are anticipated as a result of participating in this study. The technical expert committee will be asked to rate secondary data interventions and rank variations of a classification in a two-phase RAND/UCLA modified Delphi study design. The investigators do not anticipate this being difficult for the experts as they are practising physicians. It is unlikely that this will cause any psychological distress. Experts will be asked to provide consent before participation and will have an opportunity to ask questions and have them answered. The experts will be able to withdraw from the study at any time for any reason and will be able to have their data withdrawn from the study.

As there are no known or anticipated risks, the safeguards in place are minimal. All experts will be informed of their rights and/or that they can terminate their participation without any consequences.

Confidentiality

All participants of this study will have their anonymity maintained by the researchers. All documents will be stored securely and are only assessable by the investigators.

Ethics approval

This study received a research ethics board exemption waiver from the Hamilton-Integrated Research Ethics Board (HiREB), review reference 2020-11451-GRA. This study protocol is registered with ISRCTN, reference ID no. 22901977.

Results dissemination

The results of this study will be made public through peer-reviewed publication, study registries, conference publications and thesis manuscripts. Communications will be sent to relevant stakeholders with the study’s results for distribution in reports and newsletters.

Discussion

The results of this study will add a new classification to the scientific literature for categorising patients who are transported to ED by ambulance, receive primary care–like visits and could be considered to have a potentially preventable ED visit. This classification will potentially support further research into new models of preventative care as well as ambulance diversion to sub-acute centres. In addition, this study will provide patient-level evidence to inform prospective research to validate the PriCARE classification.

Progress to date

Recruitment of experts was completed in October 2020. A total of 25 physicians were invited to participate in this study, with 20 accepting. The phase I round 1 questionnaire was distributed in November 2020 for determinations of the technical expert committee. One hundred fifty ED NACRS interventions were included in the round 1 questionnaire.