Examining consensus for a standardised patient assessment in community paramedicine home visits: a RAND/UCLA-modified Delphi Study.

OBJECTIVE: Community paramedicine programme are often designed to address repeated and non-urgent use of paramedic services by providing patients with alternatives to the traditional 'treat and transport' ambulance model of care. We sought to investigate the level of consensus that could be found by a panel of experts regarding appropriate health, social and environmental domains that should be assessed in community paramedicine home visit programme.
DESIGN: We applied the RAND/UCLA Appropriateness Method in a modified Delphi method to investigate the level of consensus on assessment domains for use in community paramedicine home visit programme. SETTING AND PARTICIPANTS: We included a multi-national panel of 17 experts on community paramedicine and in-home assessment from multiple settings (paramedicine, primary care, mental health, home and community care, geriatric care). MEASURES: A list of potential assessment categories was established after a targeted literature review and confirmed by panel members. Over multiple rounds, panel members scored the appropriateness of 48 assessment domains on a Likert scale from 0 (not appropriate) to 5 (very appropriate). Scores were then reviewed at an in-person meeting and a finalised list of assessment domains was generated.
RESULTS: After the preliminary round of scoring, all 48 assessment domains had scores that demonstrated consensus. Nine assessment domains (18.8%) demonstrated a wider range of rated appropriateness. No domains were found to be not appropriate. Achieving consensus about the appropriateness of assessment domains on the first round of scoring negated the need for subsequent rounds of scoring. The in-person meeting resulted in re-grouping assessment domains and adding an additional domain about urinary continence.
CONCLUSION: An international panel of experts with knowledge about in-home assessment by community paramedics demonstrated a high level of agreement on appropriate patient assessment domains for community paramedicine home visit programme. Community paramedicine home visit programme are likely to have similar patient populations. A standardised assessment instrument may be viable in multiple settings. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

An international panel with expertise in paramedicine and/or assessment practices participated in a multi-round process to find consensus.

The assessment domains that were examined covered social, functional, cognitive and medical assessment categories.

The study process could be applied to find common approaches to assessment in community paramedicine in spite of localised differences in community paramedicine programme design.

The assessment domains that we presented did not provide detail with respect to the number of assessment items that could be included in a domain or the depth of detail.

An investigation of what assessment items are aligned with the assessment domains that have been described in this study would address the uncertainty about the amount of detail community paramedicine programme are including in their patient assessments.

Introduction

Increasing demands on the health system by an ageing population have contributed to novel approaches to service delivery in paramedicine.1–5 Community paramedicine programme are often designed to address repeated and non-urgent use of paramedic services by providing patients with alternatives to the traditional ‘treat and transport’ ambulance model of care.5–12 For example, some community paramedicine programme partner with primary care providers to assist patients with chronic disease management strategies through home visit programme that integrate patient coaching, patient monitoring processes and point of care diagnostics through scheduled, non-emergency visits.7 10 Community paramedicine programme are in the early stages of development meaning that many components of programme delivery are unclear, including the role of the paramedic in providing integrated non-emergency care. For example, it is unclear how community paramedics identify or prioritise patients, whether they are establishing consistent care plans, or if their training is sufficient to provide evidence-based treatments. In order to provide safe, consistent and evidence-based care, paramedics must complete a patient assessment to guide their decision making.13–15 If such an assessment is valid and reliable, then the paramedic is supported in their care planning and provision of treatments. Providing safe, consistent and evidence-based care should improve patient outcomes, but it is also an indicator of a health system that is working well. Standardised assessment instruments play an important role by ensuring that consistent approaches are followed and by generating the data necessary to establish the evidence-base.16–23

Community paramedicine home visit programme commonly combine a variety of discrete assessment scales to create comprehensive patient assessment forms.10 24 In this way, the focus of community paramedicine assessments are tailored to suit local clinical decision-making needs, and reflect the goals and target populations for individual community paramedicine home visit programme.25 26 Yet, standardised assessment instruments are used across multiple healthcare settings as a valuable tool for promoting information continuity across the continuum of care.16 23 25 27–29 Other studies have investigated assessment domains in traditional emergency paramedic settings to inform clinical guidelines for paramedics.30 Without further investigation of the viability for community paramedicine assessment practices, it is difficult to determine whether or not pertinent assessment domains are being included in the community paramedic decision making processes for interventions and care planning in different community paramedicine home visit programme. A standardised assessment or minimum assessment standard might clarify common scopes of practice, facilitate clinical training and improve patient care, health system utilisation and clinical communication in community paramedicine. To our knowledge no study has examined the viability of standardised patient assessment across community paramedicine home visit programme.

We sought to investigate the level of consensus that could be achieved for standardised assessment content in community paramedicine home visit programme by an international panel of relevant experts. We expected that consensus could be achieved on the relevance of some assessment domains in the community paramedicine setting despite assumed differences in programme design between settings because common assessment domains in emergency settings were identified from an international sample of paramedic assessments.30 Should such a consensus be achieved, it would provide an opportunity for future testing of a standardised assessment instrument in community paramedicine.

Methods

Design

We applied the RAND/UCLA Appropriateness Method (developed by the RAND Corporation and the University of California Los Angeles) within a modified Delphi method.31 Multiple rounds of surveys were used to ask a panel of experts to rate the appropriateness of assessment domains for community paramedicine home visit programme. An in-person consensus meeting was held to report on survey results to panel members and discuss the consensus among the group.

Ethics

Formal research ethics review was not required for this study based on non-experimental design and low risk to the panel participants. Panel members were under no obligation (real or perceived) to contribute to the work outlined in this paper, and their participation was voluntary.

Patient and public involvement

Patients and members of the public were not involved in this study.

Establishing expert panel

Our panel of experts represented key backgrounds in paramedic services, primary care, geriatric care, home care, assessment development and health services research. The panel coordination process involved national and international networks of professionals and researchers with known interests in community paramedicine or patient assessment practices. A panel coordinator sent invitations to individuals who had participated in the International Roundtable on Community Paramedicine, the Canadian EMS Research Network, the Canadian Standards Association Group Technical Committee on Community Paramedicine, the Ontario Community Paramedicine Forum, interRAI, the Canadian Frailty Network and the Ontario Association of Community Care Access Centres. Panellist selection was based on ensuring representation from multiple Canadian and international jurisdictions, multiple research settings, multiple allied healthcare sectors and multiple paramedic services

Literature search and identification of assessment domains

Throughout this study, the structure of an assessment was conceptualised to be made up of assessment items pertaining to assessment domains within assessment categories. We used a targeted literature review to identify assessment domains for consideration in our Delphi process. We included literature about assessment in community paramedicine programme by drawing on a previously conducted scoping review study on case management and care planning in community paramedicine home visit programme.5 A total of 26 articles about community paramedicine were reviewed for any assessment domains described, either generally or through explicitly named assessment instruments. A list of assessment domains was generated and grouped into assessment categories based on broad themes such as social factors, functional abilities, or ongoing health conditions. When articles named a specific assessment instrument, the domains included in it were added to the respective categories. Grouping was done concurrently by consensus between two reviewers (ML & AC). Given the lack of published research on the topic, panel members were invited to confirm the relevance of the assessment categories and provide suggestions for any other categories that they felt may warrant inclusion. A final list of assessment domains was generated based on the targeted literature review and any input regarding assessment categories that we received. We used this list of assessment domains for the first Delphi questionnaire.

Finding consensus (Delphi rounds)

The Delphi questionnaire presented panel members with each assessment domain in randomised order with a scale ranging from 0 (not appropriate) to 5 (very appropriate). A 6-point scale was used instead of the traditional 9-point scale in order to promote reproducibility (better differentiation between scale choices) and to avoid ambiguity that can result from having a midpoint in a Likert scale.32 Instructions to panel members were to consider the appropriateness of each domain with respect to the context of where, when and how patients might be assessed, what other care providers might be involved in care planning or in providing treatment, and what the aims of the community paramedicine programme specific to their individual experiences. Results from the preliminary rounds of scoring were compiled and revised surveys were distributed to members where lack of consensus (median scores of two or less on the Likert scale) was found. Two authors (ML and AC) acted as co-chairs for the in-person meeting where consensus was finalised. To facilitate discussion at the in-person meeting, assessment domains were grouped into the same assessment categories from the first Delphi questionnaire.

Results

Panel characteristics

Seventeen individuals agreed to participate in the panel (see table 1). Many participants were affiliated with multiple professional networks or associations. Experts who had a background in paramedicine included individuals involved nationally or internationally in paramedic service management, community paramedicine programme, or paramedic education or research. Experts who were familiar with assessment practices in care settings other than community paramedicine had portfolios of primary care, geriatric care, mental healthcare and home and community care.

Table 1

Distribution of panellists by affiliations and by areas of expertise

Expertise in paramedicine	n	Affiliation with professional network/association
		International roundtable on community paramedicine	Canadian EMS Research Network	CSA Group Technical Committee on Community Paramedicine	Ontario Community Paramedicine Forum (Ontario Association of Paramedic Chiefs)	interRAI*	Canadian Frailty Network	Ontario Association of Community Care Access Centres†
Paramedic service management (chief or deputy chief)	3	2	1	1	3	0	0	0
Community paramedicine (supervisor or paramedic)	6	4	3	2	3	0	0	0
Paramedic educator/researcher	4	2	3	0	1	0	1	0
Expertise in assessment
In primary care settings	1	1	0	1	1	0	0	0
In acute geriatric care settings	1	0	0	0	1	1	1	0
In home and community care settings	1	0	0	0	0	0	1	1
In mental health/emergency psychiatric settings	1	0	0	0	0	1	0	0
Total	17	9	7	4	9	2	3	1

*interRAI is a network of clinicians and researchers who develop standardised assessment instruments.

†The Ontario Associations of Community Care Access Centres was an organisation that coordinated provincial agencies who provided home and community care prior to local level restructuring.

Survey creation

Assessment categories reflected in the literature search included lifestyle, dietary and sleeping habits,11 mobility and social needs,33 home safety12 and fall risk.34 Physical examination and reported symptoms of chronic diseases were common.7 10–12 33–35 Two studies mentioned specific assessment instruments; the Canadian Diabetes Risk Questionnaire (CANRISK) tool for diabetes34 and the EQ-5D-3L for quality of life.7 The nine categories presented to panel members prior to distribution of the Delphi questionnaire were living arrangement, psychosocial well-being, cognition, functional ability, nutrition, medical history, ongoing health conditions, existing use of health services and mental health. Panel members confirmed these categories and warranted their responses by indicating that the programme that they were involved with were designed to provide care to community dwelling older adults, palliative care patients, long-term care patients, residents of assisted living, patients with identified mental health issues or members of the general population. The nine assessment categories were then used to formulate a final list of 48 assessment domains for scoring appropriateness (see table 2).

Table 2

Summary of respondent scores reflecting the appropriateness of assessment domains. Domains were ranked from 0 (inappropriate) to 5 (highly appropriate)

Question	Median	Max	Min	Range
Living arrangements and social status
Patients should be asked an open-ended question allowing them to express their personal goals for care.	5	5	4	1
Patients should be asked about their marital status.	4	5	1	4
Patients should be asked about their living arrangement (alone, with spouse, with family, etc).	5	5	4	1
Patients should be asked about changes in their living arrangement.	5	5	3	2
Patients should be asked about their social relationships.	4	5	2	3
Patients should be asked about feeling lonely.	5	5	4	1
Patients should be asked about changes in their social activities.	5	5	2	3
Patients should be asked about the amount of time they are alone during the day.	5	5	3	2
Patients should be asked about the amount of time they are alone during the night.	5	5	2	3
Patients should be asked about major stressors (severe illness, loss of income, victim of crime, loss of license, illness of family, etc).	5	5	3	2
Patients should be asked whether family or close friends feel overwhelmed by their condition.	4.5	5	3	2
Patients should be asked about their home environment (disrepair, safety, inadequate heating or cooling, etc).	5	5	3	2
Patients should be asked about making trade-offs due to finances (food vs shelter, shelter vs clothing, clothing vs medications, etc).	4	5	0	5
Patients should be asked whether they have supportive family or close friends.	5	5	4	1
Function and abilities
Patients should be asked about activities of daily living (ADL) (bathing, dressing, hygiene, walking, etc).	5	5	4	1
Patients should be asked about mobility (how they move about).	5	5	4	1
Patients should be asked about physical activity (exercise).	5	5	4	1
Patients should be asked about recent changes in ability to perform ADL (bathing, dressing, hygiene, walking, etc).	5	5	4	1
Patients should be asked whether or not they drive.	4.5	5	2	3
Patients should be asked about changes in their ability to drive.	4	5	2	3
Patients should be asked about their ability to communicate with others.	4.5	5	3	2
Patients should be asked about their hearing and vision.	5	5	3	2
Cognition, mood and mental health
Patients should be asked about their memory/recall ability.	5	5	4	1
Patients should be asked about changes to their mental status.	5	5	3	2
Patients should be asked about their mood (feeling depressed, anxious or sad).	5	5	4	1
Patients should be asked about disordered thought (irritability, inappropriate behaviours, drug or alcohol intoxication).	4.5	5	3	2
Patients should be asked about insight into their mental health problems (when applicable).	5	5	4	1
Patients should be asked about police involvement in mental health crisis (when applicable).	4	5	2	3
Patients should be asked about ideation for harm to self or others (when applicable).	5	5	3	2
Medical history, medications and ongoing health conditions
Patients should be asked whether they experience medical problems (signs or symptoms of medical conditions that have or have not been diagnosed) (dizziness, fatigue, dyspnea, hallucinations, diarrhoea, etc).	5	5	4	1
Patients should be asked about pain symptoms.	5	5	4	1
Patients should be asked about the stability of their medical conditions.	5	5	3	2
Patients should be asked to self-rate their health.	5	5	3	2
Patients should be asked about tobacco and alcohol use.	5	5	3	2
Patients should be asked about their diet.	5	5	3	2
Patients should be asked about weight loss.	5	5	4	1
Patients should be asked about the prescription medications that they take.	5	5	3	2
Patients should be asked about adherence to prescription medications.	5	5	4	1
Patients should be asked about preventative treatments or procedures (eye exam, dental examination, vaccines, mammography, colonoscopy, etc).	4.5	5	2	3
Patients should be asked about ongoing treatments or procedures (radiation, transfusions, dialysis, etc).	5	5	3	2
Patients should be asked about their continence (urinary).	5	5	2	3
Patients should be asked about their medical history (disease diagnoses).	5	5	5	0
Patients should be asked whether they have recently fallen.	5	5	5	0
Use of health services
Patients should be asked about ongoing formal care (home health aides, homemaking, physical therapy, occupational therapy, etc).	5	5	3	2
Patients should be asked about use of hospital services (inpatient, outpatient, emergency department visit, etc).	5	5	3	2
Patients should be asked about use of paramedic services (transport, non-transport, other).	4.5	5	3	2
Patients should be asked about use of community services (public health, social services, etc).	5	5	4	1

Delphi results

Thirteen members of the panel participated in the preliminary round of scoring. All assessment domains had median scores of four or higher indicating that panel members considered them to be appropriate. No item was considered to not be appropriate by the panel and only nine domains (18.8%) had a range of responses greater than or equal to three. Some domains had isolated responses by individual panellists that they were not considered to be appropriate. The three domains (6.3%) that had responses that were lower than two (indicating a degree of inappropriateness) by more than one respondent pertained to marital status, involvement of police in episodes of mental health crisis and making financial trade-offs. Preventative health measures, urinary continence, driving, social activities and time spent alone were other domains (n=5, 10.4%) that had one respondent indicate as not being appropriate. Overall, the high scores for appropriateness of assessment domains achieved on the first round of scoring negated the need for distribution of subsequent rounds of scoring prior to the in-person meeting.

Eight members of the panel were able to attend the in-person meeting. Distance and time-zone differences were factors that prevented attendance by other panellists. Discussion about pre-meeting scoring during the meeting resulted in re-grouping questions about social relationships and activities and adding an additional domain to improve context about the assessment of urinary continence. Assessment domains about making financial trade-offs, preventative health measures and driving were determined to merit inclusion for testing by sites willing to do so. Table 2 has been re-grouped according to the feedback from panel members at the in-person meeting about assessment categories. All data relevant to the study are included in tables 1 and 2. Anonymised individual survey responses are available as a online supplementary file.

Discussion

This study conducted a Delphi consensus technique to examine which assessment domains were appropriate areas of inquiry in community paramedicine home visit programme. A panel of experts familiar with community paramedicine assessment had a high level of agreement on appropriate patient assessment domains for community paramedicine home visit programme. The high level of agreement was achieved in-spite of differences in backgrounds of panel members, designs of community paramedicine programme that they were familiar with or areas of assessment expertise. Although paramedic training and education (and subsequent certification) varies between jurisdictions, their assessment practices in emergency settings are very similar.15 30 Community paramedicine programme represent a new context for assessment that apply paramedic assessment skills outside of traditional emergency settings and care paradigms. Our findings suggest that similar to emergency settings, the community paramedicine setting requires that paramedics bring together details about medical history, medications and social factors so that they can identify circumstances where patients may be at risk.

Implications

A feature of community paramedicine is to include community engagement in adapting programme operationalisation to local needs.8 36 While this is likely a key component of programme success, it has also led to uncertainty about the role community paramedics may play.1 4 Our findings illustrate that common approaches to assessment in community paramedicine likely exist and may be realised in spite of differences between settings. Conceptually, paramedics must assess patients before they can determine suitable care planning and interventions that may be beneficial.37 Future standardisation of community paramedic education and training as well as the operationalisation of common assessment practices can draw from the high level of agreement about the appropriateness of assessment domains that was achieved by an international panel of experts. In turn, improved evaluation of community paramedicine programme may be possible because commonly assessed domains would likely reflect the results of interventions and care plans. Such evaluation would also provide clarity to the community paramedic role in patient care.4

Strengths and weaknesses

The high level of agreement between experts made it difficult to determine which assessment domains were more important than others. While we purposely included clinicians and researchers with experience in primary care, geriatrics, home care and mental healthcare, as well as paramedics with experience in community paramedicine from multiple regions, our expert panel was assembled through a convenience sample and participation was voluntary for each stage of the process. Assembling a panel through other means would likely mean that dissenting views on which domains are appropriate for paramedics to assess in-home visit programme would emerge. In turn, this could have created more debate and a longer and more challenging process of achieving consensus. However, even if dissenting views had emerged through an alternate strategy for gathering a panel of experts, employing Delphi methods has had demonstrated success when consensus has not been reached immediately due to such evidence of dissent.14 38

The assessment domains that we presented did not provide detail with respect to the number of assessment items that could be included in a domain or the depth of detail. For example, one of the domains that we asked panel members to rate for appropriateness was pain symptoms. All panel members indicated that this was an appropriate domain to assess (minimum score of 4). However, pain is a very complex condition that can affect different patients in different ways. Community paramedics might be expected to follow the same style of pain mnemonic adapted from emergency practice,30 but the detail involved in determining intensity, duration, frequency and severity of pain was not explicitly described in the questionnaire provided. Similar exploration of depth and detail could be ascribed to nearly all of the assessment domains included in the questionnaire.

Future work

Testing specific assessment items within the domains evaluated in this study will be the focus of future work. A comprehensive, multi-domain, standardised assessment instrument will be pilot tested in multiple community paramedicine home visit programme. Such work will consider the role that specific assessment items might have in different community paramedicine settings and may also investigate the implications for emergency low-acuity paramedic response. An investigation of what assessment items are aligned with the assessment domains that have been described in this study will address the uncertainty about the amount of detail community paramedicine programme are including in their patient assessments and contribute to the development of a validated assessment instrument for community paramedicine. Following uptake of a standardised assessment instrument, future work can explore if adding new assessment domains to existing assessment practices improves patient care or patient outcomes and address the uncertainty about case finding in community paramedicine.

Conclusion

A diverse expert panel (in terms of geographical region, experience and clinical background) achieved consensus on domains to be included in the assessment of patients in community paramedicine home visit programme. This consensus suggests that similar assessment practices occur in diverse community paramedicine home visit programme in spite of operational differences. Questions remain about the amount of detail and degree of depth that should be included in each assessment domain.