Appropriateness of emergency care use: a retrospective observational study based on professional versus patients' perspectives in Taiwan.

OBJECTIVE: The objectives of this study are to refine the measurement of appropriate emergency department (ED) use and to provide a natural observation of appropriate ED use rates based on professional versus patient perspectives.
SETTING: Taiwan has a population of 23 million, with one single-payer universal health insurance scheme. Taiwan has no limitations on ED use, and a low barrier to ED use may be a surrogate for natural observation of users' perspectives in ED use. PARTICIPANTS: In 7 years, there were 1 835 860 ED visits from one million random samples of the National Health Insurance Database. MEASURES: Appropriate ED use was determined according to professional standards, measured by the modified Billings New York University Emergency Department (NYU-ED) algorithm, and further analysed after the addition of prudent patient standards, measured by explicit process-based and outcome-based criteria. STATISTICAL ANALYSES: The area under the receiver operating characteristic curve (AUC) was used to reflect the performance of appropriate ED use measures, and sensitivity analyses were conducted using different thresholds to determine the appropriateness of ED use. The generalised estimating equation model was used to measure the associations between appropriate ED use based on process and outcome criteria and covariates including sex, age, occupation, health status, place of residence, medical resources area, date and income level.
RESULTS: Appropriate ED use based on professional criteria was 33.5%, which increased to 63.1% when patient criteria were added. The AUC, which combines both professional and patient criteria, was high (0.85).
CONCLUSIONS: The appropriate ED use rate nearly doubled when patient criteria were added to professional criteria. Explicit process-based and outcome-based criteria may be used as a supplementary measure to the implicit modified Billings NYU-ED algorithm when determining appropriate ED use. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The analysis was based on administrative data of one million national representative samples over 7 years under a universal single-payer system.

The study provides evidence of medical professional perspectives on the appropriateness of emergency department (ED) use based on a modified Billings New York University Emergency Department algorithm.

Inappropriate and unclassifiable group ED visits were further reclassified, including process-based (specific diagnostic tests and treatments) and outcome-based (inter-hospital transfer, hospitalisation within 7 days and mortality within 30 days) criteria to reflect users’ views on the appropriateness of ED use.

National Health Insurance data were collected for routine administrative purposes with natural attrition due to migration and death.

Primary ED diagnosis may not reflect all causes of ED visits for those with multiple ED diagnoses.

Introduction

Appropriate use of emergency department (ED) care is an urgent health policy research issue and is associated with a need to increase emergency care delivery effectiveness, efficiency and safety.1 Previous reports in the literature include prospective studies applying explicit non-urgent criteria to refuse ED care,2 retrospective studies using the chief complaint to make reimbursement decisions3 and studies applying the ED algorithm to classify ED visits.4 However, there is still no consensus regarding how to best measure the appropriateness of ED use (A-EDU).5

In the real world, developing ED-specific diagnoses, procedures or treatment appropriateness criteria is not easy.6 The Billings New York University Emergency Department (NYU-ED) algorithm is well known worldwide in determining A-EDU based on medical professionals’ perspectives.4 5 7 8 The original algorithm was designed by a panel of emergency physicians to classify ED utilisation and to monitor inappropriate ED use with regard to the failure of ambulatory sensitive conditions in primary care and the use of the ED as a safety net.4 Nevertheless, some researchers have suggested that A-EDU should not only be judged by the medical professional implicitly but should also meet the patients’ needs and even the perspective of society as a whole.9 A patient’s drive to use the ED is based on the relative weights of the benefit and harm. The decision regarding the A-EDU should match patient demands with the complexity of the tasks of the physicians.10 The ideal measurement of appropriate ED use requires reflecting on the patient’s perceptions and decision to initiate an ED visit, the provider’s estimation of the complexity and severity of the patient’s condition, and a retrospective review on the part of the payer.11

In 1997, the US Congress endorsed the Prudent Layperson Standard (PLS); this legislation, with the intent of balancing Medicare and Medicaid managed care plans, established a patient standard for determining appropriate ED use.12 Taiwan has no limitations on ED use, and a low barrier to ED use may be a surrogate for real-life observations in terms of their ability to reflect the patients’ perceptions of emergency medical conditions and the need to initiate ED visits. The objectives of this study were to refine the method of A-EDU measures and to report the A-EDU rates from professional and patient perspectives.

Methods

Setting

Taiwan has a population of 23 million, under a single-payer universal National Health Insurance (NHI) scheme, covering 99.7% of Taiwan’s population. The national health expenditures as a per cent of the GDP ranged from 6.2% in 2005 to 6.6% in 2017. Accessibility to physician services is high; patients can easily receive care during the night or even on holidays in urban areas. Therefore, the average outpatient visits per beneficiary was higher than most OECD countries—it was 14.0 in 2006 and 15.3 in 2017. Out-of-office mobile ambulatory care and integrated healthcare delivery systems are even available in remote areas. However, ED utilisation is not very high, and the average ED visit was 0.29 per beneficiary per year in 2017.13 Taiwan’s healthcare system is described as offering ‘inexpensive and comprehensive care’.14 The patient satisfaction rate on NHI is typically higher than 80%, but the mean continuity of care score is relatively low (0.31 in 2006).15

Study design and data sources

This study was a retrospective observational study using secondary data analysis that included all ED visits between 1 January 2005 and 31 December 2011 based on the Longitudinal Health Insurance Database (LHID2005), which contains one million national representatives randomly sampled files from the whole population in Taiwan. The database includes the subjects’ medical and enrolment information, providers’ characteristics and medical professional information. The encrypted unique personal identification can link all databases, making longitudinal follow-up feasible. The study identified ED visits using the ED visit case type code and case revenue code. Each ED visit was analysed as an independent event to determine the appropriateness of its use. Only one event per visit date was used to avoid having the same record separated into more than one record.

Measurement: appropriateness of the ED visit

We adopted the methodology of Agency for Healthcare Research and Quality (AHRQ) method16 to refine medical professionals’ criteria of A-EDU by adding patients’ perspectives, as measured by explicit process-based and outcome-based criteria.

Medical professionals’ view of the appropriateness of ED visits

In the original Billings NYU-ED algorithm, ED visits are classified as follows: (1) non-emergency, (2) emergency/primary care treatable, (3) emergency/ED care required but preventable/avoidable (EDCNPA), (4) emergency/ED care required, not preventable/avoidable (EDCNNPA) or (5) unclassifiable. Whether the cause of the visit was an emergency was measured using the summed probability method developed by Billings et al.4 The NYU-ED algorithm categorised ED visits into two arms—‘ED care needed’ or ‘primary care treatable’—based on the optimal care setting. ICD-9 codes related to injury and behavioural health were excluded in the original paper.4 Because of the increasing percentage of ED visits in the unclassifiable group,16 the modified Billings NYU-ED algorithm was developed to update the algorithm with ICD-9 codes added since 2001.17 Empirical data support the validity of the Billings NYU-ED algorithm, which can predict hospitalisations and mortality.10 We used the modified Billings NYU-ED algorithm to measure A-EDU based on professional perspectives4 to increase the face validity of the measures. This study summed the probabilities of the EDCNPA and EDCNNPA categories based on the principal diagnosis of each ED visit: if the probability was greater than or equal to 0.50, then the visit was considered an ‘appropriate use’;7 otherwise, as it was considered an ‘inappropriate use’.

Patients’ view of the appropriateness of ED visits

ED visits classified as inappropriate and unclassifiable by the modified Billings NYU-ED algorithm were further reclassified as ED visits based on process and outcome criteria to reflect users’ views on the A-EDU. Process indicators referred to specific diagnostic tests,18 treatments19 and the level and intensity of care,18 including laboratory tests such as blood cultures,20 21 CTs,19 22 MRIs22 23 and intravenous infusions22 23 that are not often available in the primary care setting. Outcome criteria include inter-hospital transfer,22 hospitalisation within 7 days24 and mortality within 30 days.7 An ED visit was considered appropriate if it met the process and outcome criteria.

The Sydney Health Policy Analysis Authority Recommendation regarding the ‘classification systems for emergency care’ has a three-tiered structure.25 The first tier pertains to whether the ED visit is urgent; the second tier considers the ED principal diagnosis and the third tier reflects the levels of severity and complexity. Our modified A-EDU measures mentioned above can enhance construct a validity of measures because, for the first tier, we used the modified Billings NYU-ED algorithm to classify the urgency of the patients’ ED visits, as described;4 for the second-tier, urgency was determined based on the principal diagnosis according to the modified Billings NYU-ED algorithm and for the third tier, we considered contextual factors, such as process and outcome care indicators, to improve the assessment of professional appropriateness.26 Social and medical safety net factors in remote and under-served regions were also taken into further consideration.

Outcome variable and covariates

The major outcome variable for the generalised estimating equation (GEE) model was the A-EDU based on process and outcome criteria for the inappropriate and unclassifiable visit measured by modified NYU-ED algorithm. The predictive variables of the model were sex, age, occupation, health status, place of residence, medical resources areas, date of ED visit and income level of the beneficiary. Income was measured by the monthly amount of the insurance premium. Health status was measured by the Charlson Comorbidity Index (CCI). The ‘list of areas lacking medical resources’ obtained from the Ministry of Health and Welfare was used to determine whether the hospital was in an under-served area. Residential areas were classified by the urbanisation level.

Data and statistical analyses

Baseline characteristics are presented as frequencies and were compared with χ² tests. A GEE model was used to assess the associations between covariates mentioned above and the process-based and outcome-based A-EDU measures for inappropriate and unclassifiable visits as measured by the modified NYU-ED algorithm. A receiver operating characteristic (ROC) curve classification model was also used to test the performance of the combination of the modified NYU-ED algorithm and explicit process-based and outcome-based criteria in measuring A-EDU. The area under the ROC curve (AUC) is a summary statistic that reflects the accuracy in the classification, generally ranges from 0.50 (no discriminative power) to 1.0 (perfect prediction).27 A sensitivity analysis was conducted using different summation probabilities of the EDCNPA and EDCNNPA thresholds.28 Frequent ED user29 and year 2009 influenza effects30 were also analysed.

Results

Characteristics of subjects and A-EDU

A total of 1 931 451 ED patient visits between 2005 and 2011 were identified from one million samples. Of these, 95 591 events were excluded because there were two visits on the same date but in different hospitals. Thus, 1 835 860 events constituted the study sample (figure 1).

Figure 1

Emergency visits flow chart.

The initial ED visits were grouped by the modified Billings NYU-ED algorithm (online supplementary table 1). The appropriate ED visits accounted for 14.5%, the inappropriate ED visits accounted for 44.4%, unclassified ED visits was 14.1%, and visits due to injuries and behavioural causes was 26.9% (figure 2).

Figure 2

Regrouped the inappropriate and unclassified emergency department visits to the appropriate use group by process-based and outcome-based criteria.

The inappropriate and unclassified groups were further regrouped based on explicit process and outcome criteria, which resulted in 48.6% of the inappropriate group being reclassified as appropriate and 70.5% of the unclassified group being reclassified as appropriate (figure 2). After excluding those with injuries and behavioural diagnoses, the results revealed that from the professional perspective, 33.5% of the visits were appropriate (including the original appropriate group and the unclassified group based on the NYU-ED algorithm being reclassified as appropriate), while from the patient perspective, the per cent of appropriate ED visits was 63.1%. The unclassifiable group decreased from 12% to 4% following the reclassification by process-based and outcome-based criteria.

Table 1 summarises the characteristics of ED visits according to the four reclassified NYU-ED categories. Visits made by older participants, CCI >1, those made by patients living in rural and inadequate medical resources areas, those made by patients who are lower income, civil servants, teachers, military personnel and veterans and those occurring on weekdays tended to be appropriate (table 1). However, visits made by the dependents of insured individuals tended to be more inappropriate. Characteristics of ED visits all differed significantly among the four reclassified NYU-ED categories. The top 10 diagnoses in the appropriate ED visit group were organ system-related diseases, such as cardiovascular-related chest pain, syncope and palpitation; respiratory system-related pneumonia and asthma; gastrointestinal tract bleeding and urinary calculus (table 2). The diagnoses in the inappropriate ED visit group were mostly symptom-based diagnoses, such as fever and abdominal pain. The most common diagnoses in the group reclassified from inappropriate to appropriate were end-stage renal failure. The common diagnoses in the group reclassified from unclassifiable to appropriate were perinatal complications (table 2).

Table 1

Baseline characteristics of emergency department visits in the four reclassified NYU-ED categories*

	Total ED visits		NYU-ED reclassified categories								P value
			Appropriate ED visits		Inappropriate ED visits		Unclassified ED visits		Alcohol-related, drug-related, injury-related or mental health-related ED visits
	n	%	n	%	n	%	n	%	n	%
Sex
Female	891 018	48.5	425 579	47.8	217 351	24.4	35 753	4.0	212 335	23.8	<0.001
Male	944 842	51.5	420 768	44.5	202 036	21.4	40 707	4.3	281 331	29.8
Age, years
<18	330 350	18.0	89 212	27.0	150 099	45.4	12 171	3.7	78 868	23.9	<0.001
18–64	1 090 000	59.6	469 991	43.0	232 592	21.3	44 577	4.1	346 259	31.7
≥65	412 091	22.4	287 144	69.7	36 696	8.9	19 712	4.8	68 539	16.6
Charlson Comorbidity Index (CCI)
CCI≤1	1 450 000	79.0	563 324	38.8	388 719	26.8	55 032	3.8	444 046	30.6	<0.001
CCI>1	384 739	21.0	283 023	73.6	30 668	8.0	21 428	5.6	49 620	12.9
Place of residence
Urban	467 936	25.5	200 028	42.7	119 697	25.6	21 709	4.6	126 502	27.0	<0.001
Suburban	569 488	31.0	258 260	45.3	133 426	23.4	23 572	4.1	154 230	27.1
Rural	777 272	42.3	379 490	48.8	159 525	20.5	30 387	3.9	207 870	26.7
Missing	21 164	1.2	8569	40.5	6739	31.8	792	3.7	5064	23.9
Regional resources
Adequate	1 500 000	81.4	682 026	45.6	345 744	23.1	62 920	4.2	404 455	27.1	<0.001
Inadequate	340 715	18.6	164 321	48.2	73 643	21.6	13 540	4.0	89 211	26.2
Day of visit
Weekday	1 200 000	65.1	577 421	48.3	240 687	20.1	45 630	3.8	332 229	27.8	<0.001
Weekend	639 893	34.9	268 926	42.0	178 700	27.9	30 830	4.8	161 437	25.2
Income level
Quintile 1	365 753	19.9	149 746	42.9	83 649	23.9	13 122	3.8	102 834	29.4	<0.001
Quintile 2	404 274	22.0	168 628	49.8	67 361	19.9	13 686	4.0	88 826	26.2
Quintile 2	359 437	19.6	132 452	47.9	58 670	21.2	11 150	4.0	74 378	26.9
Quintile 4	309 054	16.8	225 696	47.6	101 861	21.5	20 861	4.4	125 598	26.5
Quintile 5	376 941	20.5	162 898	43.2	101 769	27	16 871	4.5	95 403	25.3
Missing	20 401	1.1	6927	34.0	6077	29.8	770	3.8	6627	32.5
Occupation
Dependents of the insured individuals	690 692	37.6	288 822	41.8	201 943	29.2	27 736	4.0	172 191	24.9	<0.001
Civil servants, teachers, military personnel, veterans	105 037	5.7	62 400	59.4	16 031	15.3	5367	5.1	21 239	20.2
Nonmanual workers and professionals	361 534	19.7	141 456	39.1	89 441	24.7	14 430	4.0	116 207	32.1
Manual workers	477 462	26.0	258 125	54.1	75 419	15.8	20 509	4.3	123 409	25.8
Other	181 327	9.9	88 965	49.1	30 558	16.9	7667	4.2	54 137	29.9
Missing	19 808	1.1	6579	33.2	5995	30.3	751	3.8	6483	32.7

*Reclassified NYU-ED algorithm categories classified ED visits based on modified NYU-ED algorithm, and recategorised the inappropriate or unclassifiable group to appropriate group.

ED, emergency department; NYU-ED, New York University Emergency Department.

Table 2

Top 10 diagnoses among emergency department visits by reclassified NYU-ED algorithm categories*

ICD-9-CM code	ICD-10 code	Diagnosis	Number
Appropriate ED visit, n=266 937
786.5	R07.9	Chest pain, unspecified	23 822
486	J18.9	Pneumonia, organism unspecified	20 377
578.9	K92.2	Haemorrhage of gastrointestinal tract, unspecified	11 825
485	J18.0	Bronchopneumonia, organism unspecified	10 457
592.9	N20.9	Urinary calculus, unspecified	9501
708	L50.0	Allergic urticaria	8463
38.9	A41.9	Unspecified septicaemia	6952
493.9	J45.909	Asthma, unspecified	6810
780.2	R55	Syncope and collapse	6657
785.1	R00.2	Palpitations	6346
Inappropriate ED visit, n=815 931
780.6	R50.9	Fever and other physiologic disturbances of temperature regulation	87 943
789	R10.9	Abdominal pain, unspecified site	85 962
558.9	K52.89	Other and unspecified non-infectious gastroenteritis and colitis	80 431
465.9	J06.9	Acute upper respiratory infections of unspecified site	54 732
780.4	R42	Dizziness and giddiness	52 514
599	N39.0	Urinary tract infection, site not specified	28 509
784	R51	Headache	28 139
466	J20.9	Acute bronchitis	24 784
462	J02.9	Acute pharyngitis	24 375
463	J03.90	Acute tonsillitis	24 068
Unclassified ED visit, n=259 326
650	O80	Normal delivery	12 936
585	N18.9	Chronic kidney disease	10 994
386.1	H81.399	Peripheral vertigo, unspecified	7283
571.5	K74.60	Cirrhosis of the liver without mention of alcohol	5852
788.2	R33.9	Retention of urine, unspecified	5426
465	J06.0	Acute laryngopharyngitis	4755
155	C22.8	Malignant neoplasm of the liver, primary	4341
386.9	H81.90	Unspecified vertiginous syndromes and labyrinthine disorders	4007
9.1	A09	Colitis, enteritis, and gastroenteritis of presumed infectious origin	3267
162.9	C34.90	Malignant neoplasm of bronchus and lung, unspecified	3215

*The reclassified NYU-ED algorithm categories classified ED visits based on modified NYU-ED algorithm and recategorised the inappropriate or unclassifiable group to the appropriate group.

ED, emergency department; NYU-ED, New York University Emergency Department.

Precision and sensitivity test of the appropriateness of ED classifications

The AUC was 0.85 (95% CI 0.847 to 0.852) (online supplementary figure 1) for the modified Billings NYU-ED algorithm and adjudication by the process-based and outcome-based reclassification of A-EDU. When the summation probabilities of the EDCNPA and EDCNNPA category thresholds were changed from p≥0.5 to p≥0.75, frequent ED users and 2009 pandemic influenza effects were eliminated, the trend in the A-EDU classification showed no substantial changes.

Multivariate analyses

The GEE analysis (table 3) shows that visits made by men were significantly less likely to be reclassified as appropriate than those made by women (adjusted OR 0.96; 95% CI 0.95 to 0.98; p<0.001). Visits made by relatively older patients were significantly more likely to be reclassified as appropriate than those made by patients under 18 years old (adjusted OR 5.32; 95% CI 5.20 to 5.44; p<0.001). Visits made by patients with more comorbidities (CCI>1) were more likely to be reclassified as being appropriate compared with those made by the reference group (CCI≤1) (adjusted OR 1.83; 95% CI 1.79 to 1.86; p<0.001). Visits made by patients living in rural areas were significantly more likely to be reclassified as appropriate compared with those made by the reference urban group (adjusted OR 1.31; 95% CI 1.29 to 1.33; p<0.001). Furthermore, visits made in areas with adequate ED resources were significantly more likely to be reclassified as appropriate compared with those made in areas with inadequate ED resources (adjusted OR 1.07; 95% CI 1.06 to 1.09; p<0.001). There was a significantly higher likelihood of visits made by patients in the highest income group being reclassified as appropriate ED compared with visits made by patients in the lowest income group (adjusted OR 1.14; 95% CI 1.12 to 1.15; p<0.001). The results of the GEE analysis investigating the reclassification of unclassified to appropriate ED visits were similar.

Table 3

Factors influencing the recategorisation from inappropriate or unclassifiable ED visit groups to the appropriate ED visit group by GEE analysis

	Inappropriate ED visit group			Unclassifiable ED visit group
	aOR	95% CI	P value	aOR	95% CI	P value
Sex
Female	1	Reference	–	1	Reference	–
Male	0.96	0.95 to 0.98	<0.001	0.68	0.67 to 0.70	<0.001
Age (years)
<18	1	Reference	–	1	Reference	–
18–64	2.37	2.33 to 2.42	<0.001	3.37	3.23 to 3.52	<0.001
≥65	5.32	5.20 to 5.44	<0.001	4.55	4.34 to 4.76	<0.001
Charlson Comorbidity Index (CCI)
CCI≤1	1	Reference	–	1	Reference	–
CCI>1	1.83	1.79 to 1.86	<0.001	1.82	1.77 to 1.87	<0.001
Place of residence
Urban	1	Reference	–	1	Reference	–
Suburban	1.18	1.16 to 1.19	<0.001	1.15	1.12 to 1.19	<0.001
Rural	1.31	1.29 to 1.33	<0.001	1.27	1.24 to 1.31	<0.001
Place of resources
Adequate area	1.07	1.06 to 1.09	<0.001	0.96	0.93 to 0.99	0.007
Deprivation area	1	Reference	–	1	Reference	–
Income level
Quintile 1	1	Reference	–	1	Reference	–
Quintile 2	1.17	1.16 to 1.19	<0.001	1.12	1.08 to 1.16	<0.001
Quintile 2	1.04	1.02 to 1.05	<0.001	1	0.97 to 1.04	0.8
Quintile 4	1.1	1.09 to 1.12	<0.001	1.04	1.00 to 1.07	0.057
Quintile 5	1.14	1.12 to 1.15	<0.001	1.02	0.98 to 1.05	0.32
Occupation
Dependents of the insured individuals	1	Reference	–	1	Reference	–
Civil servants, teachers, military personnel and veterans	0.97	0.95 to 1.00	0.071	0.95	0.91 to 1.00	0.041
Nonmanual workers and professionals	0.96	0.94 to 0.97	<0.001	0.92	0.89 to 0.95	<0.001
Manual workers	1.26	1.24 to 1.28	<0.001	1.02	0.98 to 1.05	0.37
Other	1.3	1.27 to 1.33	<0.001	1.16	1.10 to 1.21	<0.001

aOR, adjusted OR; ED, emergency department; GEE, generalised estimating equation.

Discussion

A-EDU is a crucial quality as well as efficiency issues for emergency care delivery. To fill the research gap, this 7-year retrospective observational study combined the implicit modified Billings NYU-ED algorithm as a professional standard with further reclassified inappropriate and unclassifiable group ED visits by process-based and outcome-based explicit criteria as a surrogate of prudent patient standards to estimate appropriateness rate of ED visits. The results show that the AUC Score is 0.85, indicating that the new measure had good performance regarding the classification accuracy. In addition, the A-EDU rate based on the new measure nearly doubled (63.1%) compared with that (33.5%) based solely on the professional algorithm. Therefore, patients’ perspectives are as important as professional perspectives, if not more important, when determining the A-EDU.

The application of extensively used Billings NYU-ED algorithm can determine the optimised care setting based on the procedures performed and ED resources. However, many technical concerns and potential limitations remain. For example, when determining exclusions, chief complaint misclassifications and mapping the chief complaints in the ED to the discharge diagnoses create errors because some ED visits may have more than one diagnosis, a list of resources in the primary care setting is lacking31 and the percentage of patients belonging to the unclassifiable group (approximately 11%–16% in the NYU-ED study) is too high.7 The original Billings NYU-ED algorithm was published in the late 1990s. The percentage of unclassifiable visits increased from 12% in 2006 to 19% in 2009.32 To minimise unclassifiable bias, we used the modified Billings NYU-ED algorithm, updating the algorithm with ICD-9 codes added since 2001.17 In addition, it can improve the face and construct validity of the A-EDU measures, as indicated in the Methods section.

Liberati et al suggest that this ‘algorithm for assessing reasons and alternatives of inappropriate use’ may be adjusted for disease intensity, complexity and severity.26 We used the implicit NYU-ED algorithm to classify cases based on percent probabilities regarding urgency, reflecting the real-world potential uncertainty and variation. This stage was combined with further reclassification based on explicit procedure-based and outcome-based parameters, allowing researchers to use the clinical judgement of the ED physicians who saw the patient to determine the likely intensity of the condition presented during the visit19 without oversimplification resulting from the reliance on explicit refusal of care criteria, triage criteria or denial of payment based on the chief complaint or discharge diagnosis to determine who needed emergency care.22 The combination of the implicit modified Billings NYU-ED criteria with the process and outcome criteria in this study may accurately classify local ED visits according to actual processes and medical resource utilisation, making it possible to extend the external validity of the modified Billings NYU-ED algorithm to our local setting. For example, in the setting of Taiwan, the conditions related to perinatal medical conditions were the major reasons for reclassification of the unclassifiable group as appropriate.

Patients make emergency care-seeking decisions based on their perception of their symptoms, and the majority of ED visits are not unnecessary.22 Most researchers agree that it should not be the responsibility of patients to decide correctly whether a condition is an emergency medical condition or where to seek professional care.33 In response to these concerns, the PLS has been applied to most health plans in the USA. This study sought to find a common definition of appropriate ED visits by considering urgency, severity, complexity, intensity and patient need in ED visit. However, it is impossible to determine the rationale of a layperson in seeking emergency medical care based on analyses of claim data, nor is it possible to have a definite PLS. Our universal NHI scheme, with no need for referrals, no limitations and a low financial barrier to ED visits, allows the emergency medical condition decisions made by laypersons to be explored. These population-based data can provide a new approach to understanding the acute care delivery system from both the professional and patient viewpoints. Our data showed that the percentage of ED visits deemed appropriate varied from one-third, based on professional perspectives, to about two-thirds, based on the prudent patients’ perspectives. The gap between these two viewpoints may reflect the threat to external validity of the NYU-ED criteria outside the New York setting.

When ED visit studies used explicit criteria such as inter-hospital transfers, diagnostic tests or treatments performed and rates of subsequent hospitalisation or mortality, similar findings of approximately two-thirds of the visits classified as appropriate were obtained.18 From the patient perspective, approximately 80% self-report their reason for visiting the ED as a potential serious or urgent situation.34 On the provider side, most countries facing ED overcrowding have compulsory regulations, such as the Emergency Medical Treatment and Active Labour Act, that require ED physicians to provide appropriate medical services such as screening and the stabilisation and reasonable transfer of anyone seeking treatment, without the right of refusing aid. Meanwhile, providers also need to argue for PLS to ensure that payers cannot deny reimbursement based on final diagnoses or non-urgent symptoms. In this study, most of those initially classified in an inappropriate group and further reclassified as appropriate were due to symptom-based diagnoses. This reflects the fact that hospital-based emergency care is characterised by the treatment of specialty-driven by symptoms that require simultaneous therapeutic and diagnostic interventions to stabilise patients.35 Previous study showed that 96% of the ED visits had multiple modified NYU-ED codes, and 82% of the visits had multiple CPT codes.28 This may explain why using only prospective triage refusal criteria or retrospective primary diagnosis or chief complaint criteria cannot accurately reflect the real-world complexity of emergency care. The combination of the professional criteria with further reclassified inappropriate and unclassifiable group ED visits by process-based and outcome-based criteria as a surrogate of prudent patient standards may feasibly allow the use of an administrative claim dataset for the regular monitoring of the appropriate use of EDs. Our data showed that visits made by relatively older participants, those living in rural areas and those with more comorbidities and visits made on weekdays tended more likely to be considered appropriate ED visits. However, the visits in areas with adequate emergency care resources and those involving patients with higher incomes were more likely to be reclassified from inappropriate to appropriate on the basis of the diagnostic procedure or treatment used, which may reflect possible inequality issues.

Strengths and limitations

The long-term observation of ED use patterns from the perspectives of professionals and patients was a strength of this study. First, this study was based on a national representative population-based random samples with nearly two million ED visit observations. Second, the AHRQ methodology was adopted to refine the measurement of the A-EDU and was able to predict medical appropriateness accurately. Third, implicit and explicit criteria were combined with professional, patient and social contexts. However, this study also had limitations. First, NHI Database was collected for routine administrative purposes with natural attrition due to participant migration and death. Second, we cannot further reclassify ‘after hours’ periods of ED visits due to data limitations. Third, research showed that the primary ED diagnosis may not reflect the actual utilisation for multiple ED diagnoses.

Conclusion

This combined methodology refined the modified Billings NYU-ED algorithm, making appropriate ED use classification feasible. According to our data, the percentage of ED visits deemed appropriate varied from one-third, based on professional perspectives, to about two-thirds, based on the prudent patients’ perspectives, which suggests that process-based and outcome-based criteria may be used as supplementary measures to professional standards in determining the A-EDU.