Persistent frequent emergency department users with chronic conditions: A population-based cohort study.

BACKGROUND: Frequent emergency department users are patients cumulating at least four visits per year. Few studies have focused on persistent frequent users, who maintain their frequent user status for multiple consecutive years. This study targets an adult population with chronic conditions, and its aims are: 1) to estimate the prevalence of persistent frequent ED use; 2) to identify factors associated with persistent frequent ED use (frequent use for three consecutive years) and compare their importance with those associated with occasional frequent ED use (frequent use during the year following the index date); and 3) to compare characteristics of "persistent frequent users" to "occasional frequent users" and to "users other than persistent frequent users".
METHODS: This is a retrospective cohort study using Quebec administrative databases. All adult patients who visited the emergency department in 2012, diagnosed with chronic conditions, and living in non-remote areas were included. Patients who died in the three years following their index date were excluded. The main outcome was persistent frequent use (≥4 visits per year during three consecutive years). Potential predictors included sociodemographic characteristics, physical and mental comorbidities, and prior healthcare utilization. Odds ratios were computed using multivariable logistic regression.
RESULTS: Out of 297,182 patients who visited ED at least once in 2012, 3,357 (1.10%) were persistent frequent users. Their main characteristics included poor socioeconomic status, mental and physical comorbidity, and substance abuse. Those characteristics were also present for occasional frequent users, although with higher percentages for the persistent user group. The number of previous visits to the emergency department was the most important factor in the regression model. The occasional frequent users' attrition rate was higher between the first and second year of follow-up than between the second and third year.
CONCLUSIONS: Persistent frequent users are a subpopulation of frequent users with whom they share characteristics, such as physical and mental comorbidities, though the former are poorer and younger. More research is needed in order to better understand what factors can contribute to persistent frequent use.

Introduction

Frequent emergency department (ED) users constitute a small number of ED users, but account for a disproportionately large number of total ED visits [1]. Definition of frequent users varies, though the most common definitions include having more than three or four visits during a 12-month period [2, 3]. As each type of population has its own characteristics, those definitions are context dependent and therefore may vary. For instance, timely palliative care reduces number of ED visits near the end of life because of specific care given to patients [4, 5] while patients with asthma may require more ED visits, as they are more prone to exacerbations. [6, 7]. Even though, ED visits are not necessarily evitable, for instance regarding older adults [8]. Frequent ED users often receive non-optimal and fragmented care in EDs [9], due to their complex healthcare needs. Furthermore, they have higher hospital admissions and outpatient visits along with higher mortality rate [3]. In addition to higher healthcare costs, health outcomes associated with frequent ED use are non-optimal, in contrast to timely interventions from more appropriate health resources, such as in primary care [10]. A significant proportion of frequent ED users are patients with vulnerability factors, such as poor mental health [11], socio-economic precarity [12], or chronic conditions [1, 13, 14].

Among frequent users, a subgroup of persistent users keeps on visiting EDs frequently over a multiple-year period [1, 15–18]. Definitions of persistent frequent ED use vary in the number of visits per year (from more than three visits to more than five visits per year) and in the considered period (from a two-year period to a five-year period). Although their prevalence ranges from 1 to 20%, they can account for more than 60% of the total visit volume [2]. Factors associated with persistent frequent use are physical disorders, mental health disorders, substance abuse, previous number of ED visits, and being a frequent ED user the previous year [2]. Many studies have examined frequent ED use, but few have explored persistent frequent ED use. Besides, few studies have explored persistent frequent use considering chronic conditions.

The study targets an adult population with chronic conditions, and its aims are 1) to estimate the prevalence of persistent frequent ED use; 2) to identify factors associated with persistent frequent ED use (frequent use for three consecutive years) and compare their importance with those associated with occasional frequent ED use (frequent use during the year following the index date); and 3) to compare characteristics of “persistent frequent users” to “occasional frequent users” and to “users other than persistent frequent users” (which include non-frequent users and occasional frequent users).

Materials and methods

This research was completed in accordance with the TRIPOD guidelines (see the Table in S1 Table) [19].

Design and data sources

The provincial health insurance board of the Quebec Province (Régie de l’assurance maladie du Québec or RAMQ) is the provincial organism in charge of universal healthcare services for all of Quebec residents. We conducted a population-based retrospective cohort study using its health databases:

The patient demographic register, which contains information about the sex, date of birth, date of death, and place of residence of the patient;

The physician reimbursement claim register, which contains information about medical services provided by a fee-for-service physician in Quebec: date of service, place of service (emergency, medical clinic, etc.), physician specialty, diagnosis (International Classification of Diseases, ninth revision or ICD-9), and the medical act procedure performed by the physician;

The hospital register (MED-ECHO), which contains information about the reasons for hospitalization (main diagnosis and up to 25 secondary diagnoses coded in ICD-10), dates of admission and release from hospital, and all medical procedures performed during the hospitalization.

Patient information from those databases were linked using unique encrypted identifiers.

Study population

The study population included all adults (≥18 years old) living in the province of Quebec, with at least one ED visit during the inclusion period, i.e. between the 1st of January 2012 and the 31st of December 2012, and diagnosed with one or more chronic conditions. In this study, we considered the Canadian Institute for Health Information definitions for ambulatory care sensitive conditions (see S1 and S2 Tables) [20]: asthma, chronic obstructive pulmonary disease or COPD, congestive heart failure or CHF, coronary heart disease or CHD, diabetes, epilepsy, and high blood pressure or HBP. Diagnoses were considered during a hospitalization or during two physician visits in the two-year period before the index date. The index date was randomly assigned as one ED visit among all ED visits occurring during the inclusion period [21].

There were two exclusion criteria (Fig 1). First, ED use can be different between urban and remote areas since remote residents tend to use it as an alternative to walk-in clinics as there are fewer primary care alternatives [22, 23]. Thus, patients living in municipalities with fewer than 10,000 inhabitants with weak or no metropolitan influence zone (remote areas, i.e. the percentage of resident employed labour force who commute to work in urban areas is less than 5%) were excluded from the study population. However, patients living in municipalities with fewer than 10,000 inhabitants with high or moderate metropolitan influence were included. Secondly, patients who died during the three years after their index date were excluded as they tend to require specialized healthcare (i.e. patients at the end of life [24, 25]). Besides, that last exclusion helps reduce immortal time bias [26]. Each patient had a follow-up time of three years.

Fig 1

Flowchart for the cohort selection.

Outcome and independent variables

The outcome was defined as being a persistent frequent user (binary variable, yes/no). In this study, frequent ED use was defined as having four or more ED visits in the year following their index date, while persistent frequent ED use was defined as frequent ED use during three consecutive years. To emphasize the difference between persistent frequent ED use and frequent ED use, we define a frequent ED user as “occasional frequent ED user”. Occasional frequent ED users and persistent frequent ED users are two mutually exclusive categories, while “users other than persistent frequent users” exclude persistent frequent users but include occasional frequent users.

Independent variables considered were: sex, age category (18-34, 35-54, 55-64, 65-74, 75-84, and ≥85), the type of residential area (categorical variable: metropolitan: ≥100,000 inhabitants; small town: 10,000–100,000 inhabitants; rural: <10,000 inhabitants with high or moderate metropolitan influence), material and social deprivation indices [27], public prescription drug insurance plan status (PPDIP, see below for the different statuses), diagnosis for each chronic condition (yes/no), diagnosis of depression or psychosis (yes/no), diagnosis of substance or drug abuse (yes/no), diagnosis of dementia (yes/no), having at least one hospitalization episode in the two years before the index date (yes/no), the number of ED visits during the year before the index date (≤1, 2, 3, 4, ≥5), and the combined comorbidity index of Charlson and Elixhauser [28]. This index was modified to exclude the considered chronic conditions and was constructed using the following comorbidities: cardiac arrhythmia, any tumor without metastasis, peripheral vascular disorders, neurological disorders, cerebrovascular disease, renal disease, metastatic cancer, fluid and electrolyte disorders, liver disease, rheumatoid arthritis/collagen vascular disease, coagulopathy, weight loss, paralysis, and HIV/AIDS. The reported diagnoses in MED-ECHO (one diagnosis) or in the physician claims records (at least two diagnoses) were used to identify each condition during a two-year period before the index date.

Regarding PPDIP status, it is subdivided according to four different statuses: “not admissible to PPDIP” (individuals with a private insurance plan), “admissible to PPDIP and age ≥65 years with guaranteed income supplement” (GIS), “admissible to PPDIP and being a recipient of last-resort financial assistance” (LRFA), or “regular recipient of PPDIP”.

Statistical analysis

First, we reported the prevalence of persistent frequent ED use and the associated 99.9% confidence interval. Second, we used multivariable logistic regression to identify characteristics associated with persistent frequent ED use, since the outcome is a binary variable (“persistent frequent ED users” versus “users other than persistent frequent ED users”), and those associated with occasional frequent ED use (“occasional frequent ED users” versus “users other than occasional frequent ED users and persistent frequent ED users”). We reported odds ratios and associated 99.9% confidence intervals. Furthermore, given the small prevalence of the outcome, we used Firth’s correction for logistic regression to reduce potential bias in the parameter estimations [29]. The models controlled for sex and age category. For others independent variables, automatic variable selection was implemented using backward selection, which consists in starting with a full model containing all variables, then deleting one at a time based on its statistical significance (Wald test) [30]. We used a split-sample approach in our models: we defined development and validation samples using a temporal split (at July 1st, 2012, Fig 1), as it is considered a stronger approach for developing and validating prognostic models [31]. Variable selection was performed on the development sample (first 50% of the cohort, n = 150,209), while odds ratios were obtained on the validation sample (remaining 50%, n = 146,973). Third, to compare characteristics between persistent frequent users and occasional frequent users or users other than persistent frequent users, chi-square tests of independence were used. Given the large sample size, statistical significance level was set at α = 0.001. All analyses were performed with SAS version 9.4.

Ethics approval

The research ethics board of the Centre intégré universitaire de santé et de services sociaux de l’Estrie–Centre hospitalier universitaire de Sherbrooke approved this study. All data used in this study were fully anonymized.

Results

Out of 297,182 patients who met the eligibility criteria, 1.1% (confidence interval 1.0-1.2%) were considered persistent frequent users (Table 1). Out of the 17,981 frequent users who were followed during their first year, 6,132 were still frequent users after two years and 3,357 after three years. Those users represented respectively 34.1% and 18.7% of the frequent users. The latter were thus characterized as persistent frequent users, as they maintained their status over three consecutive years. Furthermore, they used 9.5%, 9.2% and 8.9% of total ED visits during their three years of follow-up (Fig 2). Their number of ED visits per year ranged from 4 to 59 (median of 7 visits).

Table 1

Number of frequent users each year and relative to the first year.

Prevalence for persistent frequent use is in italic.

	First year	Second year	Third year
Frequent users each year
Total (n)	17,981	20,700	22,387
Percentage relative to cohort	6.0	7.0	7.5
Frequent users who remain frequent users relative to first year
Total (n)	17,981	6,132	3,357
Percentage relative to cohort	6.0	2.1	1.1
Percentage relative to first year frequent users	100	34.1	18.7

Fig 2

Percentages of non-frequent (gray), occasional frequent (blue), and persistent frequent (orange) users relative to a) the number of ED visits (three bottom rows) and b) the number of ED users (three top rows) for each year of follow-up (Y1, Y2 and Y3).

Odds ratios and model fit criteria for frequent use are presented in Table 2, based on the validation sample (n = 146,973). Two models are presented: one for occasional frequent use and one for persistent frequent use, both compared to the entire validation sample. Significant associated variables selected in the development sample (n = 150,209) were age, PPDIP admissibility, presence of COPD, CHD and diabetes, number of previous ED visits, comorbidity index, and diagnosis of depression or drug abuse. Besides those variables, occasional frequent ED use was associated with presence of asthma, coronary heart disease, and diabetes, social and material deprivation indices, and type of residential area. Being a recipient of LRFA and the previous number of ED visits were associated with the two largest odds ratios for persistent frequent ED use (respectively 2.8 and 39.1 for at 5 ED visits or more). We also evaluated our models on the full sample (without splitting) and we obtained similar results.

Table 2

Odds ratios (99.9% confidence interval) and model fit criteria from multiple logistic regression for occasional frequent users and persistent frequent users, based on the validation sample (n = 146,973).

Variable	Occasional frequent usen = 10,510	Persistent frequent usen = 1,859
Sex
Male	Reference
Female	1.02 (0.94–1.09)	1.14 (0.96–1.36)
Age category
18–34	1.42 (1.21–1.67)	1.57 (1.11–2.20)
35–54	1.11 (0.98–1.25)	1.20 (0.93–1.56)
55–64	Reference
65–74	0.97 (0.85–1.11)	0.96 (0.69–1.33)
75–84	1.10 (0.96–1.27)	1.04 (0.74–1.46)
≥85	1.14 (0.96–1.36)	0.76 (0.49–1.17)
PPDIP admissibility
Regular	1.11 (0.99–1.24)	1.09 (0.82–1.46)
≥65 years with GIS	1.36 (1.18–1.56)	1.65 (1.17–2.34)
Not admissible	Reference
Recipients of LRFA	1.76 (1.55–2)	2.73 (2.09–3.59)
Asthma	1.29 (1.16–1.43)	-
Chronic obstructive pulmonary disease	1.46 (1.33–1.59)	1.82 (1.52–2.17)
Coronary heart disease	1.11 (1.02–1.21)	1.17 (0.97–1.41)
Diabetes	1.2 (1.11–1.29)	1.37 (1.16–1.62)
Number of previous ED visits
≤1	Reference
2	2.53 (2.28–2.79)	3.75 (2.82–4.98)
3	3.78 (3.36–4.26)	7.26 (5.41–9.69)
4	5.63 (4.86–6.49)	12.58 (9.20–17.09)
≥5	12.47 (11.11–14)	37.07 (29.33–47.07)
Comorbidity index
0	Reference
1–2	1.32 (1.21–1.44)	1.40 (1.14–1.73)
3–4	1.48 (1.31–1.66)	1.38 (1.06–1.79)
≥5	1.39 (1.24–1.56)	1.28 (0.99–1.64)
Social deprivation
1	Reference
2	1.00 (0.87–1.14)	-
3	1.07 (0.94–1.22)	-
4	1.17 (1.03–1.33)	-
5	1.23 (1.08–1.4)	-
Material deprivation
1	Reference
2	1.04 (0.90–1.2)	-
3	1.10 (0.96–1.26)	-
4	1.14 (1.00–1.31)	-
5	1.16 (1.02–1.33)	-
Residential area
Metropolitan	Reference
Small town	1.16 (1.05–1.28)	-
Rural	1.29 (1.16–1.42)	-
Depression	1.30 (1.18–1.43)	1.34 (1.10–1.62)
Drug abuse	1.55 (1.29–1.85)	1.60 (1.20–2.13)
Area under the curve	0.76	0.89
R2	7%	28%
BIC	65,093	14,885

-: the variable was not selected during the variable selection process.

There were slightly more women in the persistent frequent user group than in the occasional frequent user group (59% and 56% respectively, Table 3). Persistent frequent users were also younger than occasional frequent users (higher proportion in the 35-54 category and lower proportions especially in the ≥85 category). Furthermore, there was a larger proportion of recipients of LRFA for persistent frequent users (31% versus 15% for occasional frequent users) and a reduced proportion of other PPDIP statuses. This indicates low socioeconomic standing, which relates to the distribution of those users in the higher material and social deprivation indices. However, repartition in the residential areas was similar. Except for high blood pressure, persistent frequent users had higher chronic condition prevalence than the occasional frequent users had. Persistent frequent users also had higher comorbidity indices. Furthermore, an important proportion of persistent frequent users were already frequent users in the year before their index dates since more than 60% had four or more previous ED visits. In comparison, this percentage was 28% for occasional frequent users. Drug and alcohol abuse, depression and psychoses were also more prevalent in the persistent frequent user group, whereas dementia was slightly less prevalent. Overall, the differences mentioned in this paragraph were also noticeable between users other than persistent frequent users, and occasional frequent users and persistent frequent users. For instance, chronic obstructive pulmonary disease prevalence increased from 14% to 25% and 37% for each type of users, respectively.

Table 3

Descriptive statistics for the cohort, users other than persistent frequent users, occasional frequent users, and persistent frequent users.

Variable	Total	Users other than persistent frequent users	Occasional frequent users	Persistent frequent users
Total	297,182 (100)	293,825 (100)	14,624	3,357 (100)
Sex				a, b
Female	158,881 (53.5)	156,896 (53.4)	8,171 (55.9)	1,985 (59.1)
Male	138,301 (46.5)	136,929 (46.6)	6,453 (44.1)	1,372 (40.9)
Age				a, b
18–34	17,640 (5.9)	17,301 (5.9)	1,155 (7.9)	339 (10.1)
35–54	58,072 (19.5)	57,224 (19.5)	2,806 (19.2)	848 (25.3)
55–64	64,372 (21.7)	63,757 (21.7)	2,640 (18.1)	615 (18.3)
65–74	73,591 (24.8)	72,904 (24.8)	3,305 (22.6)	687 (20.5)
75–84	61,041 (20.5)	60,370 (20.5)	3,285 (22.5)	671 (20.0)
≥85	22,466 (7.6)	22,269 (7.6)	1,433 (9.8.)	197 (5.9)
PPDIP admissibility				a, b
Regular	109,034 (36.7)	108,216 (36.8)	4,677 (32.0)	818 (24.4)
≥65 years with GIS	77,638 (26.1)	76,632 (26.1)	4,679 (32.0)	1,006 (30.0)
Not admissible	85,621 (28.8)	85,116 (29.0)	3,024 (20.7)	505 (15.0)
Recipients of LRFA	24,889 (8.4)	23,861 (8.1)	2,244 (15.3)	1,028 (30.6)
Residential area				b
Metropolitan	196,791 (66.2)	194,737 (66.3)	9,083 (62.1)	2,054 (61.2)
Small town	45,605 (15.3)	44,988 (15.3)	2,552 (17.5)	617 (18.4)
Rural	54,786 (18.4)	54,100 (18.4)	2,989 (20.4)	686 (20.4)
Coronary heart disease	75,564 (25.4)	74,347 (25.3)	4,714 (32.2)	1,217 (36.3) a, b
Asthma	34,291 (11.5)	33,465 (11.4)	2,369 (16.2)	826 (24.6) a, b
Chronic obstructive pulmonary disease	43,307 (14.6)	42,051 (14.3)	3,619 (24.7)	1,256 (37.4) a. b
Congestive heart failure	17,505 (5.9)	17,083 (5.8)	1,592 (10.9)	422 (12.6) a, b
Diabetes	96,983 (32.6)	95,620 (32.5)	5,270 (36.0)	1,363 (40.6) a, b
Epilepsy	8,600 (2.9)	8,361 (2.8)	709 (4.8)	239 (7.1) a, b
High blood pressure	163,132 (54.9)	161,234 (54.9)	8,314 (56.9)	1,898 (56.5)
Number of ED visits (1 year before the index date)				a, b
≤1	221,524 (74.5)	221,021 (75.2)	6,124 (41.9)	503 (15.0)
2	36,457 (12.3)	36,079 (12.3)	2,542 (17.4)	378 (11.3)
3	17,830 (6.0)	17,414 (5.9)	1,876 (12.8)	416 (12.4)
4	8,931 (3.0)	8,578 (2.9)	1,248 (8.5)	353 (10.5)
≥5	12,440 (4.2)	10,733 (3.7)	2,834 (19.4)	1,707 (50.8)
Previous hospitalization in the last two years	135,257 (45.5)	132,682 (45.2)	9,293 (63.5)	2,575 (76.7) a, b
Material deprivation				a, b
Missing	21,969 (7.4)	21,654 (7.4)	1,313 (9.0)	315 (9.4)
1	42,397 (14.3)	42,088 (14.3)	1,620 (11.1)	309 (9.2)
2	51,146 (17.2)	50,716 (17.3)	2,219 (15.2)	430 (12.8)
3	55,375 (18.6)	54,797 (18.6)	2,619 (17.9)	578 (17.2)
4	61,847 (20.8)	61,107 (20.8)	3,215 (22.0)	740 (22.0)
5	64,448 (21.7)	63,463 (21.6)	3,638 (24.9)	985 (29.3)
Social deprivation				a, b
Missing	21,969 (7.4)	21,654 (7.4)	1,313 (9.0)	315 (9.4)
1	46,570 (15.7)	46,181 (15.7)	1,883 (12.9)	389 (11.6)
2	49,885 (16.8)	49,483 (16.8)	2,123 (14.5)	402 (12.0)
3	55,220 (18.6)	54,632 (18.6)	2,555 (17.5)	588 (17.5)
4	58,378 (19.6)	57,723 (19.6)	2,928 (20.0)	655 (19.5)
5	65,160 (21.9)	64,152 (21.8)	3,822 (26.1)	1,008 (30.0)
Comorbidity index				a, b
0	179,430 (60.4)	178,329 (60.7)	6,424 (43.9)	1,101 (32.8)
1–2	66,097 (22.2)	65,132 (22.2)	3,868 (26.4)	965 (28.7)
3–4	24,240 (8.2)	23,673 (8.1)	2,007 (13.7)	567 (16.9)
≥5	27,415 (9.2)	26,691 (9.1)	2,325 (15.9)	724 (21.6)
Alcohol abuse	8,644 (2.9)	8,191 (2.8)	1,023 (7.0)	453 (13.5) a, b
Depression	36,601 (12.3)	35,473 (12.1)	3,098 (21.2)	1,128 (33.6) a, b
Drug abuse	5,817 (2.0)	5,323 (1.8)	885 (6.1)	494 (14.7) a, b
Psychoses	11,498 (3.9)	11,013 (3.7)	1,266 (8.7)	485 (14.4) a, b
Dementia	12,189 (4.1)	11,960 (4.1)	1,023 (7.0)	229 (6.8) b

Percentages in parentheses are relative to the column total.

achi-square test of independence significant between persistent and occasional frequent users.

bchi-square test of independence significant between persistent frequent users and other users than persistent frequent users.

Discussion

To the best of our knowledge, this work is the first to focus on persistent frequent ED users with chronic conditions. In this study, persistent frequent users represented 1.1% of the cohort, adding up to 9% of total ED visits each year. This prevalence is consistent with other studies about persistent frequent users in children and nonelderly adults [32-34]. Those studies used the same definition (≥4 ED visits during three consecutive years). Hudon et al. (2017) reported a higher prevalence of 2.6%, but their threshold for defining persistent frequent use was lower (≥3 ED visits during three consecutive years) and they focused on a diabetic population [35].

The variables associated with persistent frequent use found in this study were all reported by previous studies [32, 33, 35, 36], while some studied both occasional frequent use and persistent frequent use [32, 33, 36]. In those same studies, depending on the studied population, some specific diagnoses were also associated with persistent frequent use, such as COPD diagnosis. Besides, a few studies mentioned race [32, 33] and deprivation indices [32, 35, 36] as significant, though those latter were not necessarily the same indices as ours. Race was not available in our databases and deprivation indices were not significant in our analyses of persistent frequent use. Andren and Rosenqvist (1987) mentioned that patient’s loneliness and good rating of how they have been received at the ED increased the risk of returning to the ED in the next year in a two-year follow-up study [15]. Those variables had been collected during interviews and were not available in our databases. Thus, it would be relevant to include self-reported variables in a future work, complementary to administrative data.

Understanding reasons that may lead occasional frequent use to persistent frequent use is not trivial, as one type was not so different from the other one. In our study, they both were patients with a high comorbidity burden, diagnosed with depression and drug abuse, and with a history of ED visits. Regarding occasional frequent users with chronic conditions, Hudon et al. (2019) highlighted those same characteristics [21]. However, there were differences in our results. For instance, the social deprivation index, diagnosis of diabetes, or the type of residential area were not included in the persistent frequent use model, although they were in the occasional frequent use one. Furthermore, persistent frequent users were younger and had a heavier ED history (60% had more than four ED visits while this proportion was 27% for occasional frequent users). Finally, odds ratios for PPDIP status (being a recipient of LRFA) and number of previous ED visits were larger for persistent frequent use than for occasional frequent use, while other odds ratios were comparable in the two cases. We are not aware of other studies comparing those two variables between persistent frequent and occasional frequent users, but some reported Medicaid, an American federal insurance for patients with low-income amongst others [37], as associated with persistent frequent use [32, 38].

Previous use of ED turned out to be the most important associated factor for all the models, though its impact was stronger in the case of persistent frequent use (larger odds ratios) than in the case of occasional frequent use. The importance of previous ED use has been stated in previous studies, in the cases of occasional frequent use and persistent frequent use [2, 6, 17, 39, 40]. In particular, when studied in the baseline year, it has been reported as the strongest predictor of persistent frequent ED use in the subsequent year [32]. This may be explained by the fact that many occasional frequent ED users will not keep on visiting ED frequently and most of them will not have as many previous ED visits as already established persistent frequent ED users. Thus, previous use of ED has greater importance when it comes to studying persistent frequent ED use.

Many of the persistent frequent users in this study were already frequent users in the year before their index dates. They may have been frequent users for an even longer time than studied here. More precisely, frequent use attrition (i.e. proportion of frequent users who do not maintain their status the following year) was higher after the first year, compared to the second year (Table 1). Relative to the first-year frequent users, 34.0% maintained their status over the next year and 18.7% over the next two years. Two other studies reported this rate of decline slowing after the first year, with similar rates [15, 18], though they did not focus on a population with chronic conditions. In particular, Mandelberg et al. (2000) found that frequent users in their first year had a probability of 37.9% to maintain their status for another consecutive year [18]. This probability increased to 56.1% after two years of frequent use and to 78.7% after five years of frequent use. This suggests a “core” group in persistent frequent users.

Targeting persistent frequent users for specific interventions (such as case management [10]) may be even more relevant than targeting occasional frequent users. However, identifying them may also be more challenging. Firstly, administrative and medical data usually do not include self-perceived variables. Secondly, the high imbalance of class (persistent frequent users represent 1.1% of our cohort) means that traditional statistical models may not perform well. Since the majority of cases are not persistent frequent users, most of the statistical models will mainly use information from those cases, resulting in a suboptimal use of the information. One study showed that when the class of interest amounts to 1% of the observations, logistic regression has limited power compared to more advanced techniques, such as random forests [41]. Some adaptations exist for imbalanced data, but none has been applied to frequent ED users yet [42]. For instance, artificially balanced datasets or cost-sensitive methods, which have been used in medical sciences [43, 44], could be of interest for increasing the classification performances.

Limitations

We did not include self-perceived variables, such as physical pain or mental distress. Those variables were not available in our databases and are associated with occasional frequent use [45, 46], though they have not been studied with persistent frequent use. We also did not have access to financial status or education at the individual level. Proxy variables such as social and material deprivation and PPDIP were used instead. Lastly, our study investigated ED users with chronic conditions (defined as 1 hospitalization or 2 physician visits related to a chronic condition in the two year period before the index date, though we did not investigate the reasons for ED visits during the follow-up period), thus limiting its generalization to all ED users.

Conclusions

This paper focuses on persistent frequent ED users with chronic conditions. It highlights the fact that those users, who are a special case of frequent ED users over three consecutive years (1.1% of the total cohort), share similarities with occasional frequent users such as physical and mental comorbidities, though with higher rates. However, they are younger and poorer than occasional frequent users. Those characteristics would make them priority targets for specialized interventions. More studies are needed in order to fully characterize persistent frequent use and to understand what factors can transform occasional frequent use into persistent frequent use, especially using other databases than administrative ones or specialized statistical methods for imbalanced data.

TRIPOD checklist for reporting cohort studies.

(DOCX)

Click here for additional data file.

International classification of diseases for diagnoses used in this study.

(DOCX)

Click here for additional data file.

7 Aug 2019

PONE-D-19-18906

Persistent frequent emergency department users with chronic ambulatory care sensitive conditions: a population-based cohort study

PLOS ONE

Dear Mr Chiu,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The methodology of this study presents several shortcomings and there are relevant points in the manuscript that need further clarification:

The criteria for inclusion and exclusion of patients are questionable. Please, reconsider it according to the comments of the reviewers.

Besides, I have serious doubts about the use of ACSCs in this study. The authors selected patients with “at least one ED visit during the inclusion period, i.e. between the 1st of January 2012 and the 31st of December 2012, and diagnosed with an ACSC “. It would seem logical to assume that all the ED visits were categorized into ACSC and non-ACSC; afterward, the patients with an ACSC visit would have been followed for a three year period. However, I am not sure that this was the case. There is not any mention in the manuscript of the diagnoses or reasons for visiting the ED, but antecedents of chronic diseases instead. The concept of ACSCs is referred to conditions for which the adequate use of primary care services could avoid admissions or ED visits. Consequently, it is not appropriate to regard as ACSC the visits due to other reasons in patients whose chronic pathologies are adequately controlled in primary care or elsewhere.

In the statistical analyses it is not clear the reason for splitting the sample into training and validation ones.

Also, the comparison of the results of only two groups (“persistent & frequent” vs. “frequent but not persistent”) seems insufficient. A better option will be to include also a third group of “no frequent users” .

Follow all the valuable comments of the reviewers.

We would appreciate receiving your revised manuscript by Sep 21 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Juan F. Orueta, MD, PhD

Academic Editor

PLOS ONE

Journal Requirements:

1.    When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for including your ethics statement:

"The research ethics board of the CIUSSS Estrie - CHUS approved this study.".

i) Please amend your current ethics statement to include the full name of the ethics committee/institutional review board(s) that approved your specific study.

ii) Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research.

3. Thank you for submitting the above manuscript to PLOS ONE. We note you have stated the following in the data availability statement:

"The datasets generated and/or analysed during the current study are not publicly available due to individual privacy. The CAI (Commission d'accès à l'information duQuébec) does not authorize data sharing outside of the research team."

Please note that PLOS journals require authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. PLOS journals will not consider manuscripts for which authors will not share data because of personal interests, such as patents, commercial interests or potential future publications. Your current statement in not in line with our data availability policy, which can be found at the following link: https://journals.plos.org/plosone/s/data-availability

Please update your Data Availability Statement to indicate whether you will be able to make your data available at the time of acceptance and provide details of where the underlying dataset can be found. Please be aware that without such confirmation we won't be able to consider your manuscript for publication.

Please also note that we only require you to provide the minimal dataset, to consist of the data set used to reach the conclusions drawn in the manuscript with related metadata and methods, and any additional data required to replicate the reported study findings in their entirety. We do not require that you submit the entire data set if only a portion of the data were used in the reported study.

Thank you for your attention to this request.

4. In ethics statement in the manuscript and in the online submission form, please provide additional information about the patient records used in your retrospective study. Specifically, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information."

5. Thank you for stating the following in the Competing Interests section:

Alain Vanasse has received grants for unrelated research from AstraZeneca Canada Inc. The authors declare no other competing interests.

We note that you received funding from a commercial source:AstraZeneca Canada Inc

Please provide an amended Competing Interests Statement that explicitly states this commercial funder, along with any other relevant declarations relating to employment, consultancy, patents, products in development, marketed products, etc.

Within this Competing Interests Statement, please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your amended Competing Interests Statement within your cover letter. We will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

6. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

7. Please amend either the abstract on the online submission form (via Edit Submission) or the abstract in the manuscript so that they are identical.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for the opportunity to read this manuscript, which concerns itself with a) estimating the prevalence of persistent frequent ED use, 2) identifying factors associated with this use and 3) comparing characteristics of frequent and non-frequent users. While I have offered comments to strengthen the paper should these authors resubmit to PLOS-One or elsewhere, I am not sure that it can make a contribution to this literature for reasons I detail in my comments. Ultimately, the fact that the study is described as a population based cohort study but the outcome variable was operationalized for only a subset of cases (and the fact that this is not clear in the manuscript) detract from the paper and preclude me from being able to recommend it for publication.

I offer comments by section.

Introduction

My initial comments were relative to the number of and appropriateness of the references in the introduction section. I would like to have seen more references (and potentially more explication) for the first set of assertions in the paper (e.g. the entire first paragraph). For example, even the second sentence (“Definition of frequent users varies depending on the context, though the most common definitions include having more than three or four visits during a 12-month period”) only includes one citation, the pertinence of which is not immediately appreciable. The premises in this paragraph form the bulk of their argument and feel insufficiently motivated (referenced) and under also under-nuanced. Is this true for cancer patients, patients with cancer related pain, the frail elderly, etc.? I am, in general, against language that suggests that patients “over-use” emergency care, when the situations of these patients’ lives is often such that it does represent the best care option for them at the time they choose it. Similarly, what is meant by ‘vulnerable’ patients on line 60? More detail and nuanced description of patterns of emergency care use are needed overall in the introduction. Although they have cited Billings (2013), it feels as through they have missed the subtleties of this piece.

As an other example, the exclusion from the cohort of patients who had died within 3 years of their index visit strengthened my opinion that the intro section could do a better job with the nuances of what constitutes ‘frequent and persistent use’ as in some cases (terminally ill patients) there may well be another definitions. I had also initially commented that looking deeper into this literature would also provide guidance for these investigators about whether it is in fact true that “no study exploring persistent frequent use with ACSC has been published.”

However – as I read further in the manuscript and identified the issue of ‘outcome cases’ vs ‘reference cases’, and realized that the analyses in the paper are actually relative to ‘persistent and frequent users’ vs ‘frequent users’, I felt that the intro should also be revised in light of this focus. The reader assumes that the study is about ‘high users’ vs ‘non high users’ when in in fact is it about ‘persistent high users’ vs ‘episodic high users’. The literature review provided does little to shed light on hypothesis that may relate to ‘those with frequent use’ vs ‘those with persistent frequent use’. In fact, it is a bit misleading to imply that this is a ‘population based cohort study’ when in fact less than 10% of the sample is utilized for the primary analytic aim of the study.

Materials & Methods Section

Consistent with my comment above – I felt that the method section should make it clearer early on what cases comprise the actual analytic sample in this study. The reader does not know until Table 3. Figure 1 should be similarly revised to reflect this.

Other comments:

Is the RAMQ datasource actually exhaustive? i.e. is it literally all adult patients in the province? Please detail, for American readers this is going to be a difficult concept to grasp. Does it contain all the variables used, or is it matched with (an)other data source? It wasn’t entirely clear to me if the RAMQ and the RAMQ administrative data are the same. If not, was it possible to match all cases in the cohort across the databases used? Cases lost (if any) during this process should be detailed in Fig. 1.

As cases were excluded for rural residential status with no/low metropolitan influence (but cases with rural status with high metropolitan influence were included), this variable should be more clearly defined.

The splitting of the testing and validation samples should be described in more detail, this is a somewhat unusual technique and deserves further explication. What is the rationale for this approach over others? Does it have any implications if there are temporal dependencies in the data?

Most importantly, and weaved throughout all my comments: In the section ‘Outcome and Dependent variables’, it should be made clearer that the binary outcome was ‘persistent & frequent users’ compared with the reference category ‘frequent but not persistent’. I was somewhat surprised when I reached the results section to see that this was the case. It is natural, given the language used and overall framing of this study, for the reader to assume that the reference category in analyses is going to be ‘non frequent (non persistent) user’ (e.g. the rest of the sample). This should also be addressed in figure 1, as the reader cannot tell (easily) from either the text or the figure what N was actually used in analyses (one does not see it until Table 3). It is difficult to tell if everyone who was not in the ‘persistent’ category were by default included in the other category? There must certainly have been cases for whom the index visit was the only visit? Is it fair to categorize them as ‘frequent but not persistent’? This is quite confusing until one reaches Table 3.

I was going to comment on the applicability of logistic regression with such a low frequency of cases on the outcome compared to the reference category, until I realized what was going on with the reference category (there is in fact a reasonable distribution of cases to controls in the analytic sample). Unless I am misunderstanding, these analyses really only include the approximately 13,000 cases categorized as either ‘persistent frequent’ or ‘frequent but not persistent’. This has to be made clearer throughout the manuscript. The rationale for not using the other (more obvious) reference category should be presented and as I said above, the introduction section should be made consistent with this approach.

However, the paper still requires more detail on the backward selection technique used – did it result in only sex and age category as control variables, as is implied? This is confusing as Table 2 presents the odds ratios for many predictors in the multivariable model.

Results

As the sample was split for training and validation it should be clarified in the Tables what sample(s) were used for which analyses. It is not stated what the N’s are that resulted from this process (size of each subsample), and as I highlight below, it remains somewhat unclear throughout the rest of the paper when the split samples were used, and what size the split samples were with respect to the total, frequent users, and persistent users. The fact that this study rests on approx. 13,000 cases is fine, but it should not be represented in the paper as being based on the full sample.

Reviewer #2: The manuscript by Chiu YM et al. analyze in an adult population characterized by an ambulatory care sensitive conditions (ACSC) the prevalence of and the factors eventually associated to the condition of persistent frequent users in the Emergency Department. The manuscript is clearly written, data are well collected and the conclusions are well supported by novel and interesting results.

My main methodological concern is related to some criteria used for the selection of the experimental sample. In particular previous studies (see in particular PLOS ONE |2016 Dec 14;11(12):e0165939. doi: 10.1371/journal.pone.0165939. eCollection 2016.) have shown that older patients present clinical and social characteristics related to the definition of “elderly frail frequent users”. This data do not seem to be confirmed in the present study. Is it possible that the exclusion of patients affected by dementia and of patients died in the three years following their index may have caused a bias in the interpretation of data considering that dementia is strictly correlated with the old age? The authors need to discuss this issue.

Furthermore the authors state that previous use of ED turned out to be the most important factor for all methods in encouraging the transition from occasional to persistent frequent users. My question is why does this happen? Which are the mechanisms linking the previous ED use to the transition in persistent frequent user?

Can the authors provide some data about the hospital admission of the occasional and persistent frequent user? It is not clear to the reviewer the data “Previous hospitalization” in Tab. 3, please clarify this issue and eventually discuss these data.

Page 15, line 235: What do the authors mean for “...heavier ED history.” ?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

25 Oct 2019

October 10th, 2019

PLOS ONE

Dear Dr Orueta,

We would like to thank the editors and reviewers for their comments on our manuscript, “Persistent frequent emergency department users with chronic ambulatory care sensitive conditions: a population-based cohort study”, which offered us the opportunity to considerably strengthen it. You will find attached the revised version. Moreover, you will find detailed responses to each of the comments in this letter.

We hope the revisions will be to your satisfaction and we look forward to hearing back from you.

Best regards,

Yohann M. Chiu, Ph.D.

Corresponding Author

Université de Sherbrooke

3001, 12e Avenue Nord

Sherbrooke, QC J1H 5N4

Phone: 819-346-1110, #70538

Email: yohann.chiu@usherbrooke.ca

Editor’s comments

The criteria for inclusion and exclusion of patients are questionable. Please, reconsider it according to the comments of the reviewers.

Response: We have updated the text according to the comments of the reviewer (see below for detailed answers).

Besides, I have serious doubts about the use of ACSCs in this study. The authors selected patients with “at least one ED visit during the inclusion period, i.e. between the 1st of January 2012 and the 31st of December 2012, and diagnosed with an ACSC “. It would seem logical to assume that all the ED visits were categorized into ACSC and non-ACSC; afterward, the patients with an ACSC visit would have been followed for a three year period. However, I am not sure that this was the case. There is not any mention in the manuscript of the diagnoses or reasons for visiting the ED, but antecedents of chronic diseases instead. The concept of ACSCs is referred to conditions for which the adequate use of primary care services could avoid admissions or ED visits. Consequently, it is not appropriate to regard as ACSC the visits due to other reasons in patients whose chronic pathologies are adequately controlled in primary care or elsewhere.

Response: Thank you for pointing out this issue, we added it to the limits. ACSCs were an inclusion criterion for the cohort at the beginning of the project, which studies ED users diagnosed with ACSC. In particular, it predates our study of persistent frequent ED users. Therefore, our databases included only patients diagnosed with ACSC (1 hospitalization or 2 physician visits related to an ACSC in the two year period before the index date), regardless of the reasons for their ED visits during their follow-up period in the present study. The aim of this study was thus to examine persistent frequent use in a cohort which was already established as ACSC diagnosed ED users.

In the statistical analyses it is not clear the reason for splitting the sample into training and validation ones.

Response: The reason for splitting the cohort into training and validation samples was to avoid some statistical issues such as overfitting and to allow for estimations that are more robust. Moons et al. (2015) established guidelines for transparent reporting of a multivariable prediction model – part of the EQUATOR network – and recommended sample splitting as part of a robust design. Split sample validation is valid for prognostic model, which is why we used it only for the objective 2). It can be random or temporal; we chose the latter as it is considered a stronger approach in this context (intermediate between internal and external validation). It can lead to under or overestimation when there are strong temporal dependencies, however we also evaluated the occasional and persistent frequent ED use models 1) on the whole sample, 2) with a random split, and reached the same conclusions. We have added some details about the procedure (p. 11).

Also, the comparison of the results of only two groups (“persistent & frequent” vs. “frequent but not persistent”) seems insufficient. A better option will be to include also a third group of “no frequent users”.

Response: We thank you for this relevant suggestion. We updated the objectives and the results with the group “users other than persistent frequent users”, the complementary population to persistent frequent users (our primary interest).

Journal Requirements

Thank you for submitting the above manuscript to PLOS ONE. We note you have stated the following in the data availability statement:

"The datasets generated and/or analysed during the current study are not publicly available due to individual privacy. The CAI (Commission d'accès à l'information duQuébec) does not authorize data sharing outside of the research team."

Please note that PLOS journals require authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. PLOS journals will not consider manuscripts for which authors will not share data because of personal interests, such as patents, commercial interests or potential future publications. Your current statement in not in line with our data availability policy, which can be found at the following link: https://journals.plos.org/plosone/s/data-availability

Please update your Data Availability Statement to indicate whether you will be able to make your data available at the time of acceptance and provide details of where the underlying dataset can be found. Please be aware that without such confirmation we won't be able to consider your manuscript for publication.

Please also note that we only require you to provide the minimal dataset, to consist of the data set used to reach the conclusions drawn in the manuscript with related metadata and methods, and any additional data required to replicate the reported study findings in their entirety. We do not require that you submit the entire data set if only a portion of the data were used in the reported study.

Response: We agree that data availability is an important issue in health research. We also agree that personal and commercial interests should not interfere with data availability. However, our research team is bound by legal reasons to not divulge any part of the data. The Commission de l’accès à l’information du Québec (CAI) is the provincial organism that allows researchers to access health databases. It is also responsible for ensuring their privacy as those databases contain sensitive patient information and it does not legally allow for making any part of them public. Therefore, we are not able to do so.

Reviewer #1

Thank you for the opportunity to read this manuscript, which concerns itself with a) estimating the prevalence of persistent frequent ED use, 2) identifying factors associated with this use and 3) comparing characteristics of frequent and non-frequent users. While I have offered comments to strengthen the paper should these authors resubmit to PLOS-One or elsewhere, I am not sure that it can make a contribution to this literature for reasons I detail in my comments. Ultimately, the fact that the study is described as a population based cohort study but the outcome variable was operationalized for only a subset of cases (and the fact that this is not clear in the manuscript) detract from the paper and preclude me from being able to recommend it for publication. I offer comments by section.

Response: We thank the reviewer for its comments; they helped us improve the manuscript. Please see below for our detailed answers.

Introduction. My initial comments were relative to the number of and appropriateness of the references in the introduction section. I would like to have seen more references (and potentially more explication) for the first set of assertions in the paper (e.g. the entire first paragraph). For example, even the second sentence (“Definition of frequent users varies depending on the context, though the most common definitions include having more than three or four visits during a 12-month period”) only includes one citation, the pertinence of which is not immediately appreciable. The premises in this paragraph form the bulk of their argument and feel insufficiently motivated (referenced) and under also under-nuanced. Is this true for cancer patients, patients with cancer related pain, the frail elderly, etc.? I am, in general, against language that suggests that patients “over-use” emergency care, when the situations of these patients’ lives is often such that it does represent the best care option for them at the time they choose it. Similarly, what is meant by ‘vulnerable’ patients on line 60? More detail and nuanced description of patterns of emergency care use are needed overall in the introduction. Although they have cited Billings (2013), it feels as through they have missed the subtleties of this piece.

Response: We agree that frequent ED use is a complex and diverse situation. Regarding the second sentence, we cited Krieg et al. (2016) because it is a recent scoping review, therefore a straightforward way to assess the definition of frequent ED user. We have added some references about the end of life and about patients with asthma, along with more details and nuances in the introduction (p. 4-5).

As another example, the exclusion from the cohort of patients who had died within 3 years of their index visit strengthened my opinion that the intro section could do a better job with the nuances of what constitutes ‘frequent and persistent use’ as in some cases (terminally ill patients) there may well be another definitions. I had also initially commented that looking deeper into this literature would also provide guidance for these investigators about whether it is in fact true that “no study exploring persistent frequent use with ACSC has been published.” However – as I read further in the manuscript and identified the issue of ‘outcome cases’ vs ‘reference cases’, and realized that the analyses in the paper are actually relative to ‘persistent and frequent users’ vs ‘frequent users’, I felt that the intro should also be revised in light of this focus. The reader assumes that the study is about ‘high users’ vs ‘non high users’ when in in fact is it about ‘persistent high users’ vs ‘episodic high users’. The literature review provided does little to shed light on hypothesis that may relate to ‘those with frequent use’ vs ‘those with persistent frequent use’. In fact, it is a bit misleading to imply that this is a ‘population based cohort study’ when in fact less than 10% of the sample is utilized for the primary analytic aim of the study.

Response: This is a population-based cohort study as the whole population was indeed included for the estimations. In particular, in the logistic regression models, the validation sample was used to estimate the odds ratios of being a “persistent frequent user” (versus “all other users”, therefore including the whole sample). We have split the population as a way to obtain estimations that are statistically robust (there are more details about the splitting strategy below, that have also been added to the text). The reviewer was right that the objectives 1) and 2) were about “persistent frequent users”, more precisely for 2) the case references for the odds ratios were “all other users other than persistent frequent users”. The objective 3) was about “persistent frequent users” versus “occasional frequent users” and “all users”. Regarding 3), a category “all others than persistent frequent users” has been added to the results. We added details in the methodology (p. 8-10).

Materials & Methods Section. Consistent with my comment above – I felt that the method section should make it clearer early on what cases comprise the actual analytic sample in this study. The reader does not know until Table 3. Figure 1 should be similarly revised to reflect this.

Response: This is a relevant comment on clarity. We have updated the text (p. 8-9) and Figure 1 (p. 7) accordingly.

Other comments: Is the RAMQ data source actually exhaustive? i.e. is it literally all adult patients in the province? Please detail, for American readers this is going to be a difficult concept to grasp. Does it contain all the variables used, or is it matched with (an)other data source? It wasn’t entirely clear to me if the RAMQ and the RAMQ administrative data are the same. If not, was it possible to match all cases in the cohort across the databases used? Cases lost (if any) during this process should be detailed in Fig. 1.

Response: The Régie de l’assurance maladie du Québec (RAMQ) is the provincial organism in charge of universal healthcare services; it therefore manages information about all residents living in the province of Québec. We used exhaustive health databases owned by the RAMQ in this study: 1) patient demographic register, which provides information on sex, date of birth, date of death and history of place of residence; 2) physician reimbursement claim register, which includes data on all medical services provided by a fee-for-service physician in Quebec: date of service, place of service (emergency, medical clinic, etc.), physician specialty, diagnosis (ICD-9), medical act procedure performed by the physician and the associated cost; and 3) the hospital register (MED-ECHO), which contains reasons for hospitalisation, that is, main diagnosis and up to 25 secondary diagnoses (ICD-10), dates of admission and release from hospital, and all medical procedures performed. We have added details in the text (p.6).

As cases were excluded for rural residential status with no/low metropolitan influence (but cases with rural status with high metropolitan influence were included), this variable should be more clearly defined.

Response: We agree with this comment. We thus have updated the text with the definition for remote areas and mentioned that the type of residential area was a categorical variable (p. 7-8).

The splitting of the testing and validation samples should be described in more detail, this is a somewhat unusual technique and deserves further explication. What is the rationale for this approach over others? Does it have any implications if there are temporal dependencies in the data?

Response: The reason for splitting the cohort into training and validation samples was to avoid some statistical issues such as overfitting and to allow for estimations that are more robust. Moons et al. (2015) established guidelines for transparent reporting of a multivariable prediction model – part of the EQUATOR network – and recommended sample splitting as part of a robust design. Split sample validation is valid for prognostic model, which is why we used it only for the objective 2). It can be random or temporal; we chose the latter as it is considered a stronger approach in this context (intermediate between internal and external validation), as mentioned in the same guidelines. It can lead to under or overestimation when there are strong temporal dependencies, however we also evaluated the occasional and persistent frequent ED use models 1) on the whole sample, 2) with a random split, and reached the same conclusions. We have added details about the procedure (p. 9-10).

Most importantly, and weaved throughout all my comments: In the section ‘Outcome and Dependent variables’, it should be made clearer that the binary outcome was ‘persistent & frequent users’ compared with the reference category ‘frequent but not persistent’. I was somewhat surprised when I reached the results section to see that this was the case. It is natural, given the language used and overall framing of this study, for the reader to assume that the reference category in analyses is going to be ‘non frequent (non persistent) user’ (e.g. the rest of the sample). This should also be addressed in figure 1, as the reader cannot tell (easily) from either the text or the figure what N was actually used in analyses (one does not see it until Table 3). It is difficult to tell if everyone who was not in the ‘persistent’ category were by default included in the other category? There must certainly have been cases for whom the index visit was the only visit? Is it fair to categorize them as ‘frequent but not persistent’? This is quite confusing until one reaches Table 3.

Response: Our objectives were 1) to estimate the prevalence of persistent frequent ED use; 2) to identify factors associated with persistent frequent ED use; and 3) to compare characteristics of persistent frequent users to occasional frequent users. Regarding objective 2), the evaluated category is “persistent frequent use” and the reference category is, as you suggested, “all others than persistent frequent use” (e.g. the rest of the sample, in which occasional frequent ED users and non frequent users are included). We realize it was not clear; we thus have updated the text (p. 5, 8 10) and Figure 1. Regarding objective 3), we wanted to evaluate any significant difference between occasional and persistent frequent users. This is a relevant comment and we updated objective 3) and results with “to compare characteristics of persistent frequent users to occasional frequent users and to users other than persistent frequent users” (p. 5, 13-15).

I was going to comment on the applicability of logistic regression with such a low frequency of cases on the outcome compared to the reference category, until I realized what was going on with the reference category (there is in fact a reasonable distribution of cases to controls in the analytic sample). Unless I am misunderstanding, these analyses really only include the approximately 13,000 cases categorized as either ‘persistent frequent’ or ‘frequent but not persistent’. This has to be made clearer throughout the manuscript. The rationale for not using the other (more obvious) reference category should be presented and as I said above, the introduction section should be made consistent with this approach.

Response: The reviewer is right as in our regression analyses, the reference category for “persistent frequent users” was “all others than persistent frequent users”. We agree that this was not clear and, as mentioned in our previous response, we have updated the text accordingly.

However, the paper still requires more detail on the backward selection technique used – did it result in only sex and age category as control variables, as is implied? This is confusing as Table 2 presents the odds ratios for many predictors in the multivariable model.

Response: The backward selection technique is an automated technique used for selecting independent variables in regression models, based on statistical criterion, and for developing parsimonious models. It starts from a full model (all independent variables included), then eliminates the non significant variable one at a time until all variables that remain are significant. We “forced” age and sex into the models as control variables, meaning we forced the selection process to keep them. We have updated the methods accordingly (p. 9-10).

Results. As the sample was split for training and validation it should be clarified in the Tables what sample(s) were used for which analyses. It is not stated what the N’s are that resulted from this process (size of each subsample), and as I highlight below, it remains somewhat unclear throughout the rest of the paper when the split samples were used, and what size the split samples were with respect to the total, frequent users, and persistent users. The fact that this study rests on approx. 13,000 cases is fine, but it should not be represented in the paper as being based on the full sample.

Response: We used the development sample for selecting the significant variables in the logistic regressions (n = 143,879), whereas we used the validation sample to estimate the odd ratios and the associated confidence intervals (n = 141,114). This information was clarified in Table 2, in Figure 1, and in the text (p. 9, 13-14).

Reviewer #2

The manuscript by Chiu YM et al. analyze in an adult population characterized by an ambulatory care sensitive conditions (ACSC) the prevalence of and the factors eventually associated to the condition of persistent frequent users in the Emergency Department. The manuscript is clearly written, data are well collected and the conclusions are well supported by novel and interesting results.

Response: We thank you for this positive feedback.

My main methodological concern is related to some criteria used for the selection of the experimental sample. In particular previous studies (see in particular PLOS ONE |2016 Dec 14;11(12):e0165939. doi: 10.1371/journal.pone.0165939. eCollection 2016.) have shown that older patients present clinical and social characteristics related to the definition of “elderly frail frequent users”. This data do not seem to be confirmed in the present study. Is it possible that the exclusion of patients affected by dementia and of patients died in the three years following their index may have caused a bias in the interpretation of data considering that dementia is strictly correlated with the old age? The authors need to discuss this issue.

Response: Our focus in this study was persistent frequent users (four ED visits per year over three consecutive years), which lead to us to exclude patients who died within three years of their index date. We also excluded patients who were diagnosed with dementia because of their specific characteristics; including them could have influenced our results generalization. However, those excluded patients may indeed have presented characteristics related to “elderly frail frequent users”. Our population being different from the one used by Legramante et al. (2016), this prevent us from drawing further conclusion. There are few papers on persistent frequent users, let alone on “elderly frail persistent frequent users” to compare results with those in the cited paper. However, we thank the reviewer for this comment and have added this issue to the limits (p. 19).

Furthermore the authors state that previous use of ED turned out to be the most important factor for all methods in encouraging the transition from occasional to persistent frequent users. My question is why does this happen? Which are the mechanisms linking the previous ED use to the transition in persistent frequent user?

Response: One of our main results is that the persistent frequent users included in our study were already frequent users in the year before the index date. Studies about frequent ED use also mention that previous ED use is the most important predictive variable. While we do not necessarily assume that this variable encourage the transition from occasional to persistent frequent use, we believe that persistent frequent users may have been holding that status for a longer period than studied here, as explained in p. 17-18 (we also added some details to the text). Because of the high rate of attrition of frequent users after the first year, most of them will not have as many ED visits as (already established) persistent frequent ED users, as seen in Table 3. Thus, this variable is of greater importance when it comes to studying persistent frequent use.

Can the authors provide some data about the hospital admission of the occasional and persistent frequent user? It is not clear to the reviewer the data “Previous hospitalization” in Tab. 3, please clarify this issue and eventually discuss these data.

Response: The variable “previous hospitalization” relates to having at least one hospitalization in the two years before the index date, regardless of the admission reason. Since we did not investigate reasons for hospitalization, we added this to the limits of the study.

Page 15, line 235: What do the authors mean for “...heavier ED history.”?

Response: Persistent frequent users had higher number of ED visits than frequent users (60 and 27 % had more than 4 ED visits respectively). We clarified this in the text (p. 16).

Submitted filename: Chiu_PLOSONE-Responses.docx

Click here for additional data file.

15 Nov 2019

PONE-D-19-18906R1

Persistent frequent emergency department users with chronic ambulatory care sensitive conditions: a population-based cohort study

PLOS ONE

Dear Dr Chiu,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Although the revised manuscript has improved, there are several important concerns that remain inadequately addressed.Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process:

- Most of the mentions of ACSC found in the title, text and tables are misleading. Almost universally, ACSC are referred to preventable hospitalizations or ED visits potentially avoidable. The population of this study are all the patients with any of a group of chronic diseases that visited ER. Such diseases could be considered as ACSC if they would have been the reason for such visits. Consequently, in order to avoid confusion for the readers, the term ACSC should be replaced by “chronic conditions” or other similar.

- As reviewer 2 points out, I do not find a sufficient reason to exclude the patients with dementia. It seems plausible that such patients presented different health care needs to others subjects, but the same reason can be argued in relation to other diseases (for example HBP compared to asthma, COPD, or CHF) or age-groups.

- The authors should also follow the recommendations of reviewer 1 about the statistical model, inclusion of measures of model fit, and presentation of the results.

We would appreciate receiving your revised manuscript by Dec 30 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Juan F. Orueta, MD, PhD

Academic Editor

PLOS ONE

Additional Editor Comments:

Besides, there are some minor errors in the draft. The description of the process to exclude patients presented in text (page 7) does not follow the same order as the one in figure 1; it seems to be a typo in the number and percentage of patients due to death. Also, in discussion (page 16, line 261) it is stated that “deprivation indices were not significant in our analyses”, in contradiction to the results of the study.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I think the paper is improved with the changes. The purpose is clearer and the samples included in the analyses are also clearer. However, I continue to have some questions about the statistical approach and statistical conclusions, and the reporting of the results as well.

There is still an issue in these logistic regressions concerning the rarity of the outcome for the group ‘Persistent frequent users’, which comprise only 1/10 of 1% of the sample. As these models are inferential models intended to produce generalizable inferences about relationships between predictive factors and membership in a rare utilization group I think the models need more attention. Especially given the number of predictive variables (and their levels) considered in these models. (As an example, I wonder how many cases there are in the multidimensional cross-tabs of ‘age >-85 and ‘ has ACSC diabetes’, given all the other predictors. One might assume that this multidimensional cell size is quite small.) I think that in order for the results from these models to be credible some additional sensitivity analyses are warranted. I recommend considering Firths’ penalized logistic regression (Puhr R, Heinze G, Nold M, Lusa L, Geroldinger A. Firth‘s logistic regression with rare events – accurate effect estimates and predictions? Statistics in Medicine 2017.).

Additionally I strongly recommend the variable and model screening processes be reported as well as some measures of model fit, at a minimum an R2 measure, and possibly also ROC/AUC or information criteria measures for model fit and precision in order to allow readers to gauge for themselves how these models fit, and potentially how incrementally important the identified predictors are relative to a baseline model.

Reporting issues:

the statistical analysis section states that the first result is ‘prevalence of persistent frequent ED use’, however Table 1 does not contain that result (rather it reports ‘frequent users’). Is this a typo in the table?

Pg 11, line 207-208 should specify if the results presented in Table 2 represent the results of 2 separate models comparing 1) occasional frequent users to the entire sample and 2) comparing Persistent frequent users to the entire sample (vs comparing these two groups to each other) because it is still confusing. Additionally, the N’s should be reported in this table.

What are the cells in Table 2 that contain a ‘-‘? Were those coefficients not estimable?

Reviewer #2: The revised version of manuscript by Chiu YM et al. responded only partially to our concerns and also to the important concerns of other reviewers and to the Journal Requirements. In particular I continue to be convinced that not consider older people with chronic diseases such as dementia who, in my opinion, often could be considered as ambulatory care sensitive, may cause a bias in the full interpretation of the problem.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

16 Jan 2020

Dear Dr Orueta,

We would like to thank you again, along with the reviewers for their second round of comments on our manuscript. We believe that the comments allowed us to improve the quality of our manuscript. Please find attached the revised version and the detailed responses to each of the comments.

We hope the revisions will be to your satisfaction and we look forward to hearing back from you.

Best regards,

Yohann M. Chiu, Ph.D.

Corresponding Author

Université de Sherbrooke

3001, 12e Avenue Nord

Sherbrooke, QC J1H 5N4

Phone: 819-346-1110, #70538

Email: yohann.chiu@usherbrooke.ca

Editor’s comments

Most of the mentions of ACSC found in the title, text and tables are misleading. Almost universally, ACSC are referred to preventable hospitalizations or ED visits potentially avoidable. The population of this study are all the patients with any of a group of chronic diseases that visited ER. Such diseases could be considered as ACSC if they would have been the reason for such visits. Consequently, in order to avoid confusion for the readers, the term ACSC should be replaced by “chronic conditions” or other similar.

Response: We agree; since we did not investigate the reason for ED visit, we have updated the text with “chronic conditions” instead of “ambulatory care sensitive conditions”.

As reviewer 2 points out, I do not find a sufficient reason to exclude the patients with dementia. It seems plausible that such patients presented different health care needs to others subjects, but the same reason can be argued in relation to other diseases (for example HBP compared to asthma, COPD, or CHF) or age-groups.

Response: We have run our analyses including this time patients with dementia, although this does not change the results or the interpretations. We have updated the manuscript with those latest results.

The authors should also follow the recommendations of reviewer 1 about the statistical model, inclusion of measures of model fit, and presentation of the results.

Response: We have updated the methods and the results according to the comments of the reviewer 1 (see below for detailed answers).

Besides, there are some minor errors in the draft. The description of the process to exclude patients presented in text (page 7) does not follow the same order as the one in figure 1; it seems to be a typo in the number and percentage of patients due to death. Also, in discussion (page 16, line 261) it is stated that “deprivation indices were not significant in our analyses”, in contradiction to the results of the study.

Response: The text for the description of the sample selection now follows Figure 1 (Lines 123-138); thank you for pointing out that. Regarding deprivation indices, it is true that they were significant in the occasional frequent use model, but they were not in the persistent frequent use model. We mentioned this difference in the discussion.

Reviewer #1

I think the paper is improved with the changes. The purpose is clearer and the samples included in the analyses are also clearer. However, I continue to have some questions about the statistical approach and statistical conclusions, and the reporting of the results as well.

There is still an issue in these logistic regressions concerning the rarity of the outcome for the group ‘Persistent frequent users’, which comprise only 1/10 of 1% of the sample. As these models are inferential models intended to produce generalizable inferences about relationships between predictive factors and membership in a rare utilization group I think the models need more attention. Especially given the number of predictive variables (and their levels) considered in these models. (As an example, I wonder how many cases there are in the multidimensional cross-tabs of ‘age >-85 and ‘has ACSC diabetes’, given all the other predictors. One might assume that this multidimensional cell size is quite small.) I think that in order for the results from these models to be credible some additional sensitivity analyses are warranted. I recommend considering Firths’ penalized logistic regression (Puhr R, Heinze G, Nold M, Lusa L, Geroldinger A. Firth‘s logistic regression with rare events – accurate effect estimates and predictions? Statistics in Medicine 2017.).

Response: We thank the reviewer for this pertinent suggestion. It is true that, given the small prevalence of persistent frequent use, specialized models should be used. We have run the analyses using Firth’s penalized logistic regression and have modified the methods and results sections accordingly. However, the results do not change in regards to numeric values or interpretations (though the updated results in the manuscript are slightly different as they now include patients with dementia, as suggested by another reviewer).

Additionally I strongly recommend the variable and model screening processes be reported as well as some measures of model fit, at a minimum an R2 measure, and possibly also ROC/AUC or information criteria measures for model fit and precision in order to allow readers to gauge for themselves how these models fit, and potentially how incrementally important the identified predictors are relative to a baseline model.

Response: Area under the curve, R2, and BIC have been added to the analysis (Table 2). However, since the goal is to evaluate explicative factors for persistent frequent use and not to compare different models, we did not compare our models to a baseline model. Line 229

Reporting issues:

The statistical analysis section states that the first result is ‘prevalence of persistent frequent ED use’, however Table 1 does not contain that result (rather it reports ‘frequent users’). Is this a typo in the table?

Response: We agree that it was not clear; we reported the prevalence of persistent frequent ED use in Table 1 but did not mention it. We clarified it in the table. Lines 207-208

Pg 11, line 207-208 should specify if the results presented in Table 2 represent the results of 2 separate models comparing 1) occasional Frequent users to the entire sample and 2) comparing Persistent frequent users to the entire sample (vs comparing these two groups to each other) because it is still confusing. Additionally, the N’s should be reported in this table.

Response: Table 2 indeed presents results for two regression models: 1) occasional frequent users versus the entire sample and 2) persistent frequent users versus the entire sample. We have clarified this in the text and added the sample sizes in the table. Lines 215-217

What are the cells in Table 2 that contain a ‘-‘? Were those coefficients not estimable?

Response: Table 2 contains results for two models: 1) occasional frequent use and 2) persistent frequent use. Since both models went through an automatic variable selection process, the explicative variables included were not necessarily the same for both; in particular, there were more variables for the first model. The dash ‘-‘ means that a variable was not selected therefore not estimated for the persistent frequent use model. We clarified this point in a footnote of Table 2. Line 229

Reviewer #2

The revised version of manuscript by Chiu YM et al. responded only partially to our concerns and also to the important concerns of other reviewers and to the Journal Requirements. In particular I continue to be convinced that not consider older people with chronic diseases such as dementia who, in my opinion, often could be considered as ambulatory care sensitive, may cause a bias in the full interpretation of the problem.

Response: Thank you for pointing out the possibility of a bias in the interpretations when excluding patients with dementia. We thus have run our analyses including them (n=12,189, 4.1 % of the total sample). Results and interpretations remain the same, though the material deprivation index is now significant in the logistic models. The sample selection, results and discussion are now updated. Lines 137; 155; 252

Submitted filename: Chiu_PLOSONE-Responses.docx

Click here for additional data file.

29 Jan 2020

Persistent frequent emergency department users with chronic conditions: a population-based cohort study

PONE-D-19-18906R2

Dear Dr. Chiu,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Juan F. Orueta, MD, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Marianna LaNoue

3 Feb 2020

PONE-D-19-18906R2

Persistent frequent emergency department users with chronic conditions: a population‑based cohort study

Dear Dr. Chiu:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Juan F. Orueta

Academic Editor

PLOS ONE