Measuring patient experience in the emergency department: A scoping review.

INTRODUCTION: Measuring patients' experience in the emergency department can be an avenue through which the patients are able to evaluate their own care experience, and this may provide guidance for healthcare professionals in addressing quality improvement. This scoping review aimed to identify and examine existing tools that measure patients' experience in the emergency department.
METHODS: A scoping review was carried out to synthesize evidence from a range of studies in order to describe the characteristics of each study and their sample, and to describe the tools used to measure patients' experience in the emergency department.
RESULTS: Out of the 308 articles retrieved, results of the first and second level screening yielded 10 articles for inclusion using 9 different experience tools/questionnaire in the emergency department.
CONCLUSION: Measuring patients' experience in the emergency department is a global concern, however research conducted in low-to-middle-income countries is very limited and such research in Africa appears to be absent. Getting consumers of care to evaluate their experience may help healthcare professionals to identify discrepancies in care and plan possible strategies to address them.

African relevance

Measuring patients' experience in the ED is a global concern, however there appears to be a limited number of tools available, and such research in Africa appears to be absent.

This article provides details of specific tools available in the literature for measuring patients' experience in the emergency department, which could be useful for African EDs

Getting consumers of care to evaluate their experience may help healthcare professionals to identify discrepancies in care and plan possible strategies to address them.

Introduction

Patients' experience with healthcare services describes their interaction with hospital personnel, products, services and structure that contributes to care provision (The Beryl Institute, 2017). Measuring patients' experience can be an avenue through which the patient voices are heard, and may also enable patients' to participate in their own care experience, which has been shown to promote higher compliance to treatment and discharge instructions [[1], [2], [3], [4]]. It can also provide an opportunity for benchmarking and policy formulation thus making it necessary for healthcare professionals to identify tools that can accomplish this task [5,6].

Many overlapping concepts such as patients' satisfaction, perceptions, engagement, needs, participation and preference are used to explain what is meant by patients' experience, however most of these terms are not well defined and often lack conceptual clarity which in turn challenges their accurate measurement [7]. Measuring these concepts independently is also problematic as they are often found to be ill-defined to capture the associated complexities [[7], [8], [9]]. These concepts therefore need to be clearly defined and used to develop a valid, reliable, acceptable and relevant tools to measure patients' experience in the hospital setting [10].

The unique environment of the emergency department (ED), which is usually chaotic, rushed, with little or no privacy, is indeed difficult for patients [11,12]. Factors such as increased patient volume, complex patients' needs, lack of beds, intensive treatment within the ED, staff and space shortages, language and cultural barriers are factors that appear to influence patients' satisfaction with care in the ED (Nairn, 2004). The ability of patients' to accurately describe their experience of care can be compromised by this often stressful environment of the ED, which can result in the patient experiencing anxiety, fear and uncertainties [5,13,14]. The often noisy and unpredictable nature of the ED environment might also limit the healthcare professionals' ability to assess patients' experience [14,15]. This scoping review aimed to identify and examine tools that measure patients' experience in the ED. The objectives of this scoping review were to 1) describe the characteristics of each study and their sample and 2) describe the tools used to measure the patients' experience in the ED.

Methods

A scoping review is done to synthesize evidence from a range of studies with the aim of identifying gaps in existing literature about a phenomenon of interest [16]. This scoping review utilised the Arksey and O'Malley's Framework [17] which was further developed by Levac, Colquhoun and O'Brien [18] as well as the Joanna Briggs Institute Framework for Scoping Reviews [19]. This review also adhered to the Preferred Reporting Items for Systematic Reviews and meta-analyses (PRISMA-ScR) search guidelines [20]. The final protocol was registered prospectively with the Open Science Framework on November 26, 2019 at https://osf.io/bcqfg.

Search criteria

The review started by identifying the inclusion criteria guided by the PCC (Population, Concept and Context) [19] that would produce relevant documents to meet the study objectives (Table 1). The search strategy for this study identified any existing literature, both published and unpublished, in the following databases; PubMed, MEDLINE, and CINAHL. The search terms used are listed in Box 1.

Table 1

PCC and inclusion and exclusion criteria.

Population: All adult patient (18 years and older) admitted to the ED.

Concept: All types of tools; these includes scales, questionnaire and surveys etc. used to measure patients' experience in the ED.

Context: The emergency department of a hospital.

Inclusion criteria• Literature published over 20 years (1994 to 2018) was considered. • Studies, reports, and published or unpublished articles that focused on tools to measure adult patients' (18 years and older) experience in the ED. • Articles published using the English language. • Studies that focused on patients'/clients in the ED. • Studies that reported on patients' satisfaction, patient engagement and quality outcomes • Review articles including: systematic reviews, meta-analysis, scoping reviews, peer-reviewed journals and rapid reviews, quantitative studies, and pilot studies, letters and guidelines • Grey literature sources such as documents from government and non-governmental organisations and academic dissertations.

Exclusion criteria• Articles not published in English language • Articles published before January 1994 and after 2018 • Articles focusing on children

Search terms that were used in each database included:

Patient/client experience, patient/client satisfaction; patient “AND” participation; patient engagement “AND” perceptions. Emergency department “AND” emergency room; tools “AND” questionnaire “AND” instruments. (((tools[Title] OR instrument[Title] OR questionnaire[Title]) AND ((“patients”[MeSH Terms] OR “patients”[All Fields] OR “patient”[All Fields]) AND experience[All Fields])) AND ((“emergency service, hospital”[MeSH Terms] OR (“emergency”[All Fields] AND “service”[All Fields] AND “hospital”[All Fields]) OR “hospital emergency service”[All Fields] OR (“emergency”[All Fields] AND “department”[All Fields]) OR “emergency department”[All Fields]) OR (“emergency service, hospital”[MeSH Terms] OR (“emergency”[All Fields] AND “service”[All Fields] AND “hospital”[All Fields]) OR “hospital emergency service”[All Fields] OR (“emergency”[All Fields] AND “room”[All Fields]) OR “emergency room”[All Fields]))) NOT (“child”[MeSH Terms] OR “child”[All Fields] OR “children”[All Fields]). (“trauma centers”[MeSH Terms] OR (“trauma”[All Fields] AND “centers”[All Fields]) OR “trauma centers”[All Fields] OR (“trauma”[All Fields] AND “unit”[All Fields]) OR “trauma unit”[All Fields]) OR (“emergency service, hospital”[MeSH Terms] OR (“emergency”[All Fields] AND “service”[All Fields] AND “hospital”[All Fields]) OR “hospital emergency service”[All Fields] OR (“accident”[All Fields] AND “emergency”[All Fields] AND “department”[All Fields]) OR “accident and emergency department”[All Fields]) OR (“emergency service, hospital”[MeSH Terms] OR (“emergency”[All Fields] AND “service”[All Fields] AND “hospital”[All Fields]) OR “hospital emergency service”[All Fields] OR (“casualty”[All Fields] AND “department”[All Fields]) OR “casualty department”[All Fields]) OR unit[All Fields] OR room[All Fields].

⁎MeSH terms used in PubMed and MEDLINE.

Alt-text: Box 1

Study selection: inclusion and exclusion criteria

All identified studies were exported into EndNote reference management software and duplicates were identified and removed. The eligibility criteria were used by the two authors independently to conduct the first level screening of the articles using titles and abstracts (Table 1). Full text articles were then used during the second level independent screening to determine those to include in the review. Disagreements regarding the eligibility of three articles were resolved by consensus. The same process was followed for both the database and the hand search articles retrieved. Refer to Fig. 1 for the PRISMA flow diagram of the selection process. A data extraction form was developed to extract study characteristics including; author(s), year of publication and country of study, research design, setting, sampling and sample size, name of the tool and number of question items, response format and scoring methods, validity and reliability, and limitations of the study. A narrative synthesis of the results was carried out to answer the objectives.

Fig. 1

PRISMA Flow diagram of selection process.

Results

The results of the first and second level screening (both electronic and hand searching) yielded 10 articles for inclusion using 9 different patients' experience tools/questionnaires in the ED; 7 of those articles came from the hand search.

International scope of studies

The results of the scoping review indicated that measuring patients' experience in the ED is a global concern, with 2 papers originating from The Netherlands [21,22], United Kingdom and the Netherlands (1) [23], Australia (1) [24], Iran (3) [[25], [26], [27]], Canada (2) [4,28] and United States of America (1) [29].

Characteristics of the study and study participants

All of the articles describing these tools were published between 1999 and 2018, with the majority (n = 6) being published between 2011 and 2014. All the studies utilised a cross-sectional survey design (n = 10) (see Table 2). Convenience sampling was used in 4 of the studies, random sampling by 3 and systematic (1) and accidental sampling (1), while 1 article did not report its sampling method. The sample sizes in these studies ranged from 103 to 128,350, with most studies collecting data in state urban and rural hospitals. One study focused on a vulnerable population consisting of elderly, mentally ill and homeless patients [28]. Regarding the mode of tool administration in 5 studies, the participants took part in face-to-face questionnaires while still in the hospital or in their homes 7–10 days after discharge. Four studies collected data via a mail survey and 1 collected the data via mail as well as telephonically. (Table 2). Two studies had a mixed sample of patient and family members [26,27] who answered the same tool.

Table 2

Characteristics of the studies (N = 10).

Author, year of publication and country	Research design/setting/sampling and sample size	Name of tool & number of question items	Response format & scoring methods	Validity and reliability	Limitations
Atari, M & Atari, M (2015)Iran	Survey (face-to-face questionnaire)Urban hospitalConvenience sampling N = 301132 patients and 165 family members. N = 4 missing data	Brief Emergency Department Patient Satisfaction Scale (BEPSS) - 24 question items	Each item was scored from 4 (complete satisfaction) to 1 (complete dissatisfaction) on a 4 point Likert scale. Total ED patient satisfaction score was calculated by adding all the items' scores.	The total alpha coefficients = 0.94. Reliability coefficients of five of the domains range from 0.75 to 0.88. One way ANOVA and t-test = no significant difference (P > 0.05).Face validity of each item was evaluated by panel of experts: 2 hospital managers, 2 quality-improvement officers, 1 physician, and 1 psychometrics expert.	Cultural diversity of the particular participants may have influenced the way they responded to the questionnaire.
Bos, N. et al. (2016)England and The Netherlands	Cross-sectional mail surveyUrban & rural hospital ED of NHS Trust,Random sampling N = 43, 892 (England) and N = 1865(The Netherlands)	National Health Services (NHS) - 50 questions	Secondary analysis of the data from England and Netherlands was performed.Overall mean of the experience scores for each item was computed. Linear mixed effects model was used to examine the association between countries and patients' experience.	Internal consistency of each question item was scored individually. Cronbach's α co-efficient range from 0.634 to 0.877	Differences in sample sizes, number of respondents and selection of hospitals, including variabilities in patients' characteristics between the two countries in the study appears to limit the generalisation of the findings of the study.
Bos, N. et al. (2013)The Netherlands.	Cross-sectional mail survey in urban and rural setting.Systematic sampling N = 128,350 in total @ 850 patients/per trust conducted in 151 trusts.	National Health Services (NHS) - 50 questions	The Principal Components Analysis (PCA), a method out of three of grouping and summarising items, present the best score reliability on six clear and interpretable composites: waiting time; doctors and nurses; your care and treatment; hygiene; information before discharge and overall.	Cronbach's α range from 0.634 to 0.877.	Significant differences between respondents and non-respondents age and sex limits generalisation of finding.
Bos, N. et al. (2015)The Netherlands	Cross-sectional mail surveyUrban hospital Random sampling N = 4883	Consumer Quality Index for the Accident and Emergency Department(CQI A&E) - 78 questions	2, 3 or 4 point Likert scale. Response categories were recoded from 1 to 4, summed up and divided by the number of items in the domain. (i.e. no/big problem/never = 1, sometimes = 2, bit of a problem = 2.5, usually = 3, yes/not a problem/always = 4.	Domains were internally consistent with Cronbach's α score of 0.67–0.84.	Recall bias related to language differences and state of consciousness of patients' could potentially influence the survey results and therefore also their generalizability
Chiu, H. et al. (2014)Canada	Survey (face to face) and administered by mail in 110 EDs across British Columbia, Canada.Convenient Sampling of N = 170 vulnerable patients' who were elderly, aged 75 or above, had low income, were homeless or resided in unstable housing, or were disenfranchised with MH/SU issues.	Picker Canada Patient Experience Survey (condensed version) - 9 questions +1 open-ended question.	Overall patients' experience was measured using the % of positive responses. The % of positive responses was calculated based on the proportion of the number of responses categorized as “positive” to the total number of responses in the question.Responses from open-ended question were coded as positive, negative, both positive and negative or neutral	Not reported.	Administration of the survey alone indicate that it does not provide sufficient information to guide quality improvement activities. Patients' experience should be measured alongside other types of quality indicators to guide overall quality improvement and provide a balanced view of performance.
Davis, B. (1999)Australia	Descriptive research design (face to face survey) One rural and one urban hospital EDConvenience sampling N = 103 patients.	Consumer Emergency Care Satisfaction Scale (CECSS) - 17 questions items + 2 open-ended questions.	5 point Likert scale.Individual item means (M) and standard deviation (SD) were scored.The higher the score, the higher the level of satisfaction with ED nursing care.Qualitative analysis of two open-ended questions produced four and six themes respectively.	Cronbach's α coefficient range from 0.85 to 0.88.	Use of small, convenience sample and that data was collected in only one area of Australia limits the generalisation of findings.
Mohammadi-Sardo, M. R. & Salehi, S. (2018)Iran	Cross-sectional study.Face to face at the hospital and at homes.Random sampling N = 373	Emergency Department Patient Satisfaction Assessment - 24 questions + 1 open-ended question	5 point Likert scale.Items were scored individually. Total scores of items in each component were determined as low, moderate, and high. The higher the score, the higher the level of patients' satisfaction.	Face and content validity were confirmed by experts in the field.Tool was found to be valid and the Cronbach's alpha was 0.995. Translation validity was assessed by expert panel, rechecked by a linguist and piloted in 10 patients.	Some data were missing because it was collected in patients' houses after discharge. It could be helpful if social and cultural factors could be investigated as they could influence patients' satisfaction.
Sari, O. et al. (2011)Canada	Survey (mail)Sampling method not reportedN = 6255	Patient Experience Survey (PES). Question items not indicated.	Each item was scored individually.	Not reported.	Survey had lower response rate because experience of the homeless and mentally ill were very difficult to capture.
Soleimanpour, H. et al. (2011).Iran	Cross-sectional study (face to face). ED of an Iranian hospital Accidental quota sampling.N = 500 (452 patients and 48 family members).	Press Ganey Questionnaire (PGQ) - 30 questions	Total ED patients' satisfaction score was calculated by summing all the items' scores.	Press Ganey Questionnaire is a valid tool. From a previous study, internal consistency reliability ranged from 0079 to 0.96. Content validity was measured by team of ED experts and academic members.	Difficult to generalise findings because of regional differences. Not measuring time spent in the ED and different diagnosis of patients' might have different satisfaction rates.
Weinick, R. et al. (2014)USA	Cross-sectional (mail and telephone survey)USAConvenience samplingN = 829	Centres for Medicare & Medicaid Maryland USAEmergency Department Patient Experience of Care Survey EDPEC (On Site/Admitted Stand Alone) (29 questions).	Using any number from 0 to 10, 0 is the worst and 10 is the best care. Total ED patients' satisfaction score was calculated by summing all the items' scores.	The instrument was found reliable with the score of 0.70.Face validity evaluated by experts in the ED.	Further studies should be conducted since results were based on pilot studies.

Description of the tools

The developed tools ranged in number of items/questions asked; from 78 to 9 items. The most frequently used response format for answering the tools was a Likert type rating scale using a four or five-point scale (n = 7). When scoring the tool, authors added up all the individual items and then classified the total experience score as low, moderate and high. Only 3 tools obtained responses through some open-ended questions [24,25,28].

The tools were examined to identify the different patients' experience domains being investigated and these 10 domains are indicated in Table 3. The most common domains covered by the 9 tools are related to the patients' experience with the doctors and nurses as well as the patients' experience with their care and treatment; these domains appearing in 8 of the 9 tools. The least common (only appearing in 1 tool) are the domains of patients' family and hospital environment and facilities. There were only 4 domains that were included in <5 of the tools namely; doctors and nurses, your care and treatment, medications and leaving the ED. In addition to patients' experience in the ED, 1 article included domains related to empathy, that is, the care and response of hospital personnel to patients [25].

Table 3

Domains of the tools.

Name of the tools	Domains investigated in the tool
	Total	Arrival in the ED	Waiting	Doctors & Nurses	Your care & treatment	Pain	Tests	Medications	Hospital environment & facilities	Patients' family	Leaving the ED	Overall ED experience
National Health Service (NHS) Trust Questionnaire	10	X	X	X	X	X	X	X	X	–	X	X
Emergency Department Patient Experience of Care (EDPEC) Survey (On Site/Admitted Stand Alone	6	X	X	X	X	–	–	X	–	–	X	X
Consumer Emergency Care Satisfaction Scale (CECSS)	7	–	–	X	X	X	X	X	–	–	X	X
Accident and Emergency Department Questionnaire (A&ED)	6	X	X	X	–	X	–	–	–	–	X	X
Picker Canada Patient Experience Survey (Condensed Version)	6	X	X	X	X	–	–	–	–	–	X	X
Press Ganey Questionnaire (PGQ)	5	X	X	X	X	–	–	–	–	–	–	X
Patient Experience Survey (PES).	5	X	X	–	X	–	–	–	–	–	X	X
Brief Emergency Department Patient Satisfaction Scale (BEPSS)	7	X	X	X	X	–	–	X	–	X	–	X
Emergency Department Patient Satisfaction Assessment (EDPSA)	8	X	X	X	X	X	X	X	–	–	X	–
Total		8	8	8	8	4	3	5	1	1	7	8

Validity and reliability

Researchers ensured that most of the developed tools were tested using a pilot study [[21], [22], [23], [24]], however for 2 of the tools, validity or reliability were not reported [4,28]. Four of the questionnaires were mailed to ED patients after being recruited anonymously, to minimise biases [30]. The face validity of each question item in 3 tools were determined by a panel of experts in the field of emergency care and quality improvement [22]. Construct, translation and content validity were all tested in 1 study [25]. The internal consistency of 5 tools were tested for reliability and the results were found to be acceptable as they range from 0.634 to 0.995 [31]. Cronbach's alpha was used in 8 of the studies to gauge reliability. See Table 2 for more details.

Discussion

This scoping review aimed to identify and examine tools that measure patients' experience in the ED.

Characteristics of study and study participants

The patients' experience in the ED tools were all developed and tested in various high income countries, except for 2 tools that were developed in Iran. It is important to note that this serves to highlight the fact that measuring patients' experience in the ED is a global concern, however research conducted in low-to-middle-income countries is limited and such research in Africa appears to be absent.

The fact that all the studies utilised a cross-sectional survey design was not surprising as this design is generally quick, easy and cheap to perform. It would also limit the lost-to-follow-up in the context of a busy and chaotic ED environment, as participants are usually accessed only once [32]. It is also important to note that only 4 of the studies made use of probability sampling; the advantage being that samples are more representative of the target population [33]. The use of a convenience sampling method makes one question the quality of the studies since such a sampling method is highly vulnerable to high level of sampling error, response and selection bias [34]. It is most important to note that most articles included in this study were obtained through hand searching. This is due to that fact that some studies may be published in journals not included in electronic databases used or they may not have included wording in the title and abstracts or be indexed with terms that allowed them to be easily identified [35].

Regarding the mode and timing of tool administration; some of the studies administered face-to-face questionnaires while the patient was still in the hospital or in their homes shortly after discharge. A number of studies also made use of mail surveys to collect their data. It is important to consider which is the most appropriate method to administer such tools in the challenging ED environment? The pen and paper method of administering such tools may not be suitable in a highly transitory ED environment and leveraging technology and social media platforms to obtain information on patients' experience in the ED may be quicker and easier when compared to the traditional method of survey administration. The WhatsApp Messenger application which has been found very useful in Africa can be an avenue to obtain real-time responses from patients' regarding their ED experience [36,37].

It is very important to determine the best time to administer the tool. Administering the tool to measure patients' experience while the patient is still in the ED might delay treatment or have health consequences for the patient; the ever busy ED environment might not be conducive for such an exercise and doing so after the ED experience might reduce the amount of information patients can remember. Question items in a patients' experience tool should therefore be limited, concise and not time-consuming for patients' to complete.

Description of the tools used in measuring patients' experience.

Nine different tools were identified in this scoping review. The NHS Trust Questionnaire that was adapted from the United Kingdom and used in The Netherlands revealed significant differences in the within-in country results compared to the result obtained in The Netherlands using the same tool. The language and the wording of question items may have limited the interpretation of the questions and therefore produced different results in the different cultural settings. The Press Ganey Questionnaire, used in Iran was adapted from the United States. Cultural diversity of the particular participants may have influenced the way they responded to the questionnaire. Differences in patients' characteristics may cause difficulties in adapting the tools across different countries and also difficult to generalise findings because of regional differences [38,39]. The most frequently used response format for answering the tools was a Likert type rating scale. This method is quick, efficient and inexpensive for data collection and can be sent through mail, over the internet or given in person [40].

Only 3 studies included some open-ended, qualitative type of questions and future research should consider incorporating such questions to allow for more in-depth information that could be helpful to guide decisions and policies about health care and social welfare [34]. The internal consistency of most of the tools was measured using the Cronbach's α, thus attempting to justify the psychometric measurement [41]. Reporting the validity and reliability of the tools is essential as this could guide fellow researchers to evaluate the tool [42]. It is important to consider that administration of the patients' experience tool alone does not provide sufficient information to guide quality improvement activities. Patients' experience should be measured alongside other types of quality indicators to guide overall quality improvement and provide a balanced view of performance [43].

Study limitation

The first screening was done by reading the title, and browsing the abstract, thus the precision of this procedure is entirely dependent on the terminology used in the titles and the abstracts. There is the risk that relevant articles may have been overlooked for this reason. There may also be questionnaires used by governments and organisations that our internet searches did not find. Studies not indexed in PubMed, CINAHL and MEDLINE may have been missed, as well as studies using different terminology in the title and abstracts. Only literature published in English language was used for this review and thus research published in other languages were not included. Many overlapping concepts such as patients' satisfaction, perceptions, engagement, needs, participation and preference are used to explain what is meant by patients' experience, however most of these terms are not well defined and often lack conceptual clarity which in turn challenges their accurate measurement.

Implications for practice and conclusion

Measuring patients' experience in the ED is a global concern, however there appears to be a limited number of tools to measure patients' experience in the ED and especially within low-to-middle-income countries in Africa. Getting consumers of care to evaluate their experience may help healthcare personnel to identify discrepancies in care and plan possible strategies to address them.

Authors' contribution

Authors contributed as follow to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and drafting the work or revising it critically for important intellectual content: YO contributed 50% and PB contributed 50% each. All authors approved the version to be published and agreed to be accountable for all aspects of the work.

Declaration of competing interest

The authors declare no conflict of interest.