Fulfillment, burnout and resilience in emergency medicine-Correlations and effects on patient and provider outcomes.

BACKGROUND: Healthcare provider wellness have been reported to correlate with patient care outcomes. It is not understood whether synergistic effects may exist between them.
OBJECTIVE: We aim to investigate three provider wellness markers and determine their associations with provider self-reported medical errors and intent-to-leave outcomes among Emergency Department (ED) providers.
DESIGN: This is a multi-center retrospective study.
METHOD: Three wellness domains include professional fulfillment (PF), burnout (BO), and personal resilience (PR). Two outcomes measured as provider self-reported medical errors and provider intent-to-leave. Correlations between wellness markers and outcomes were analyzed. When adjusted for other confounders (provider demographics, provider experience, and operational environment), a multivariate logistic regression analysis was performed to further determine the interactions among these three domains on provider wellness affecting patient and provider related outcomes.
RESULTS: Total 242 surveys were collected from providers at 16 different EDs. The median score of PF were 2.83 among physicians and 2.67 among APPs, BO were 1.00 (physicians) and 0.95 (APPs), and PR were 0.88 (physicians) and 0.81 (APPs). The median scores of self-reported medical errors were 1.50 (physicians) and 0.95 (APPs), and intent-to-leave were 1.00 (physicians and APPs). High correlations occurred among PF, BO, and PR. When analyzed together, high PF, low BO, and high PR functioned as a protective effect on provider intent-to-leave (adjusted odds ratios = 0.09, 95% CI 0.03-0.30).
CONCLUSION: High correlations occurred among three provider wellness markers with no significant difference between physicians and APPs. Providers with high PR, low BO, and high PR tended to be more stable in their jobs.

Introduction

Healthcare provider wellness has been recognized as a factor that plays an important role in patient-centered care in recent years [1-3]. Previous studies have shown that physician wellness was positively associated with patient satisfaction, decreased medical errors, and improved patient clinical outcomes [1, 4–6]. Professional fulfillment (PF) is defined as happiness or meaningfulness, self-worth, self-efficacy and satisfaction at work [7]. Studies focusing on PF and professional satisfaction have demonstrated that physicians with high PF or satisfaction are associated with higher patient satisfaction [8, 9]. Whereas, ones with lower PF correlated with increased medical errors and higher professional burnout (BO),—an emotional exhaustion, depersonalizations, and reduced feelings of personal accomplishment [4, 10]. On the other hand, personal resilience (PR)–the capacity to respond to stress in a healthy way–has been studied using sleep-related impairment, provider anxiety and depression measures [11-13]. Providers with higher PR tended to be more sustainable in the health care workforce [11]. Promoting PR could potentially decrease provider BO and increase the quality of care while reducing medical errors [12, 14–16].

In terms of these provider wellness domains, previous studies have focused on specialties of family or internal medicine with few reported in the field of Emergency Medicine (EM). Given a common belief that EM providers have higher rates of stress, anxiety, and burnout [17, 18], it is important to investigate wellness conditions using similar tools amongst Emergency Department (ED) providers. Furthermore, provider wellness markers may have moderating effects on each other or interactions among different domains that could affect outcome measures. One domain functioning as a moderator can further affect other domains functioned as the mediators, so that the mediators and moderators would interact in the same model [19]. Therefore, it is necessary to determine whether provider wellness markers have internal correlation with potential interactions that could positively or negatively affect outcomes.

At present, interactions among healthcare provider wellness domains have rarely been reported, especially regarding their association with either patient care outcomes (i.e. medical errors) or provider related outcomes (i.e. job retention). We hypothesized that the unique environment in which ED providers practice differentiates them from other specialties, wellness conditions differ from other specialties, and both patient safety and provider job security may be unique to that environment. Therefore, to better understand ED provider wellness conditions and their relationship to patient and provider outcomes, we aim to: 1) measure ED provider wellness domains, specifically PF, BO, and PR; 2) determine the correlation of these wellness markers to patient-related medical errors and provider intent-to-leave; and 3) further investigate interactions among all three wellness domains and its association with the study outcomes.

Methods

Study design

This was a secondary data analysis of a previous quality improvement project focusing on healthcare provider wellness within Integrative Emergency Services (IES) group, an Emergency Medicine (EM) group providing EM coverage of 16 hospital EDs mainly in the State of Texas, USA. This project was intended to include all ED providers within IES group. Therefore, the survey was sent to all ED providers including physicians and Advanced Practice Providers (APP). APP includes physician assistant (PA) and nurse practitioner (NP). Data were collected prospectively online using Qualtrics Survey Software (Provo, UT) from January to March 2018, via the Stanford Wellness Survey, created by the WellMD team at Stanford University. The survey questionnaires were provided by the Stanford WellMd Center with contractual agreement and permission to use data in secondary analysis. Due to the nature of secondary data analysis with deidentified personal information, this study was waived for approval by the local Institutional Review Board (IRB).

Provider wellness survey

The study provider wellness survey was reported previously in the literature and was a closed survey. Survey was sent to all ED providers via email with a link to the survey website. The first two pages of the survey included the general instruction of survey completion and informed consent. Survey was not proceeded if providers declined to participate. Survey website were active, and providers were allowed to enter multiple times during the entire study period. The participation of this project was totally volunteer with no incentive provided regardless of the participations. While answering the survey, providers had opportunity to review their response using the back button and were able to change their response before the final submission. We identified unique participant based on the general information and email/IP address that individual participant provided. We allowed multiple entrances of the survey during the study period. If duplicated surveys were found, the latest one was used for the final analysis.

Study setting and participants

This study enrolled healthcare providers from 16 different hospital EDs across the state of Texas, USA. Among all 16 EDs, 2 EDs have extremely high annual volume (>100,000/year), 5 EDs have moderate to high annual volume (60,000–100,000/year), while the other 9 EDs have low to moderate annual volume (<60,000/year). We included surveys from qualified ED healthcare providers who agreed to participate in this study. We excluded surveys: 1) from providers who declined to participate; 2) empty surveys; 3) incomplete surveys (<10% of completions); and 4) duplicate surveys.

Provider wellness measurements

Three areas of healthcare provider wellness were measured including professional fulfillment (PF), burnout (BO), and personal resilience (PR). PF and BO were measured using a 16-point professional fulfillment index (PFI) questionnaire with good reliability [4]. PFI includes measurement within three domains (professional fulfillment, emotional exhaustion, and interpersonal disengagement) covering two distinct areas (PF and BO). PR measures sleep-related impairment, provider anxiety, and provider depression. Sleep-related impairment was measured using an 8-item questionnaire, a short version of the Pittsburgh Sleep Quality Index and portion of the Patient Reported Outcomes Measurement Information System (PROMIS) tool [4, 13, 20]. Provider anxiety and depression measurement included a 4-item questionnaire which was also derived from the PROMIS tool [4, 21]. Each item was scored on a five-point Likert scale (0 to 4). The overall score of each area was calculated by averaging the total item scores. High scores indicated providers had high PF, high BO, or low PR (defined as high sleep-related impairment, high anxiety, and high depression levels). These provider wellness measures have all been previously reported and validated [4, 20, 21].

Outcome measurements

We measured two outcomes (patient-related and provider-related). Provider self-reported medical errors were considered patient-related outcome measures. We used 4-item questionnaire tool that was previously reported in the literature for medical error measurements [4]. Briefly, medical errors were classified as errors resulting in patient harm (include any adverse events occurred linked to the medical errors), wrong medication (incorrected medications including names, doses, and type of administrations, that had administered to patients, or had not given to the patients but recognized by others (pharmacists, nurses, or other providers), or wrong lab test (any incorrected lab tests initially ordered in the computerized physician order entry (CPOE) system regardless of the completions), etc. that occurred both recently (e.g., the previous week) and within providers’ working lifetime on a six-point Likert scale ranging 0–5. The overall score was calculated by averaging the total item scores. Provider intent-to-leave was considered provider-related outcome measurement. This was a one-item question asking the providers’ likelihood of leaving the institution within two years using a 5-point Likert scale ranging from 0 (None) to 4 (definitely), that was also reported in the literature. The investigators decided a priori using Delphi technique that scores of <1 for intent-to-leave and <2 for self-reported medical errors were considered “low” scores [22, 23].

Study variables

Provider demographics included sex, race, ethnicity, and age. Other variables included volume of primary practice ED, years in practice at the practice site, and years since completion of training. We also surveyed whether providers lived with a significant other and/or children.

Data analysis

We intended to measure ED provider wellness markers in three domains: professional fulfillment, burnout, and personal resilience. We used Cronbach’s alpha (α) to determine internal consistency of professional fulfillment, burnout, and personal resilience measures. An α>0.8 was considered good reliability and α>0.7 was considered adequate reliability. Skewness and kurtosis were used to determine whether wellness markers were normally distributed. |Skewness|<0.5 was mildly to normally distributed, 1>|Skewness|≥0.5 was moderately skewed, and |Skewness|≥1 was highly skewed. Kurtosis>3 was considered data less normally distributed. Furthermore, we investigated the associations between provider wellness markers and outcomes. We initially used correlation co-efficiency (r) with |r|≥0.5 indicating strong correlations, 0.5>|r|≥0.3 indicating moderate correlations, and 0.3>|r|≥0.1 indicating weak correlations. Secondly, we performed a multivariate logistic regression to determine the association between different provider wellness markers and study outcomes (self-reported medical errors, and provider intent-to-leave). After completed appropriate diagnostic checks to identify the outliers and determine the collinearity, all available independent variables were initially included in the regression model. Variables including provider demographics and practice environment, in addition to measured wellness markers, were analyzed as potential independent risk predictors of patient-related and provider-related outcomes. The backward stepwise variable selection was applied to obtain the final regression model. Significance levels for entry and to stay were set at 0.1 to avoid exclusion of potential candidate variables. The final regression model was determined by sequentially excluding individual variables with a p-value > 0.05 until all regression coefficients were significant. The overall model performance was tested using the Hosmer–Lemeshow goodness-of-fit test. Finally, interaction analysis was performed to determine whether these three wellness markers have interactions which affect study outcomes. To quantify such interactions, investigators use previously reported cut off of >3 for high “professional fulfillment” and 1.33 for high “burnout” scores [4]. Low PR was defined as above 1.25 based on the upper quartile scores of the entire cohort. All analyses were performed using Stata v14.2 (College Station, Texas).

Results

From January to March 2018, a total of 382 surveys were sent out to all ED providers. We received 326 surveys with over 85% (326/382) of response rate. A final completion rate of 89% (289/326) was yielded after the exclusion of 37 providers who declined to participate. Providers worked at 16 different hospital EDs, included full-licensed ED physicians and APPs (including physician assistants and nurse practitioners). Among all received surveys, we also excluded 47 invalid surveys (31 empty surveys and 16 surveys with <10% of completions) with 242 surveys placed in a final analysis (see detail in Fig 1).

Fig 1

Study flow diagram.

Table 1 shows the general characteristics of the study participants. Our study included 146 ED physicians and 96 APPs. Males were predominant among ED physician and females were predominant among APP participants. Most participants were White, non-Hispanic, and practicing at moderate to high volume EDs. Over half of the participants were out of training less than 10 years and have been practicing at designated ED less than 5 years (see Table 1).

Table 1

General characteristics of study participants.

	Physician (N = 146)	APP (N = 96)
Gender---n (%)
Male	101 (69)	34 (35)
Female	43 (29)	62 (65)
Unknown	2 (1.4)
Age---n (%)
<30 years old	2 (1.4)	9 (9.4)
30–39 years old	75 (51)	44 (46)
40–49 years old	37 (25)	23 (24)
50–59 years old	19 (13)	18 (19)
≥60 years old	10 (6.9)	2 (2.1)
Unknown	3 (2.1)
Race---n (%)
White	108 (74)	75 (78)
Black or African American	5 (3.4)	4 (4.2)
Asian	28 (19)	13 (14)
Others*	5 (3.4)	4 (4.2)
Ethnicity---n (%)
Hispanic/Latino	4 (2.7)	4 (4.2)
Not Hispanic/Latino	142 (97)	91 (95)
Unknown		1 (1.0)
Participant Primary Hospital ED Size---n (%)
Low ED Annual Volume (<60,000/year)	18 (12)	18 (19)
Moderate ED Annual Volume (60,000–100,000/year)	59 (40)	44 (46)
High ED Annual Volume (>100,000/year)	66 (45)	30 (31)
Unknown	3 (2.1)	4 (4.2)
Practice Years out from Training---n (%)
<5 years	51 (35)	38 (40)
5–10 years	44 (30)	31 (32)
11–15 years	20 (14)	13 (14)
16–20 years	10 (6.9)	9 (9.4)
>20 years	21 (14)	4 (4.2)
Unknown		1 (1.0)
Practice Years at Primary Participant ED---n (%)
0–5 years	96 (66)	62 (65)
6–10 years	28 (19)	24 (25)
11–15 years	9 (6.2)	9 (9.4)
>15 years	13 (8.9)	1 (1.0)
Relationship---n (%)
Not living with a significant other	15 (10)	25 (26)
Living with a significant other	129 (88)	70 (73)
Not living with dependent children	51 (35)	42 (44)
Living with dependent children	94 (64)	53 (55)

Other* includes American Indian or Alaska Native, Native Hawaiian or Pacific Islander, or Unknown.

Provider wellness markers including PF, BO, and PR were measured and descriptive analyses are reported in Table 2. Data from most wellness markers were mildly skewed. Therefore, both mean with standard deviation (SD) and median with interquartile range (IQR) were reported. Internal consistency was measured among different wellness markers using Cronbach’s α, all showed good internal consistency (see detail in Table 2). No significant differences in terms of PF, BO, and PR were found between physician and APP groups. However, physicians tended to report more medical errors than those of the APP’s (p<0.001).

Table 2

Descriptive analysis of healthcare provider wellness survey and outcome measurements.

Measurement	Physicians	APPs	P value
Healthcare Provider Wellness Measurements
Professional Fulfillment (PF)
Mean (SD)	2.71(0.78)	2.64(0.77)	0.47
Median (IQR)	2.83(2.17,3.33)	2.67 (2.17,3.08)	0.54
Skewness/Kurtosis/Cronbach α	-0.39/2.76/0.91	-0.60/3.43/0.89
Burnout (BO)
Mean (SD)	1.06(0.69)	1.05(0.72)	0.88
Median (IQR)	1.00(0.60,1.40)	0.95 (0.50, 1.50)	0.87
Skewness/Kurtosis/Cronbach α	0.95/4.24/0.94	0.56/2.90/0.92
Personal Resilience (PR)
Mean (SD)	0.96(0.51)	0.93(0.60)	0.68
Median (IQR)	0.88(0.56,1.25)	0.81(0.44,1.31)	0.36
Skewness/Kurtosis/Cronbach α	0.80/3.25/0.90	0.87/3.49/0.92
Outcome Measurements
Patient-related (Self-reported Medical Errors)
Mean (SD)	1.63(0.86)	1.08(0.68)	<0.001
Median (IQR)	1.50(1.00,2.25)	1.00(0.50,1.50)	<0.001
Skewness/Kurtosis	0.25/2.46	0.37/2.90
Provider-related (Intent-to-leave)
Mean (SD)	0.82(1.00)	0.96(1.01)	0.31
Median (IQR)	1.00(0,1.00)	1.00(0,2.00)	0.22
Skewness/Kurtosis	1.27/4.24	0.96/3.38

Table 3 shows correlations between different wellness markers. Moderate to high correlations were found in PF, BO, and PR regardless of types of ED providers. Regarding outcome measurements, provider wellness markers seem to have no correlation to self-reported medical errors among physicians but showed weak to moderate correlation with APPs. Additionally, better correlations were found between wellness markers and provider intent-to-leave in physicians than in APPs.

Table 3

Correlations between different healthcare provider wellness markers and outcomes.

	Professional Fulfillment	Burnout	Personal Resilience	Medical Errors
Physicians
Professional Fulfillment	X
Burnout	-0.61	X
Personal Resilience	-0.48	0.59	X
Self-reported Medical Errors	-0.07	0.09	0.06	X
Intent-to-Leave	-0.50	0.44	0.35	0.05
APPs
Professional Fulfillment	X
Burnout	-0.58	X
Personal Resilience	-0.49	0.55	X
Self-reported Medical Errors	-0.27	0.44	0.32	X
Intent-to-Leave	-0.36	0.43	0.19	0.14

To further determine associations between different wellness markers and outcomes, a multivariate logistic regression analysis was performed to analyze the risk of wellness markers affecting either patient-related or provider-related outcomes. Table 4 shows no such effects of wellness markers on provider self-reported medical errors, consistent with no correlation shown in Table 3. When provider wellness markers were analyzed to determine the association with provider intent-to-leave, high PF and low BO seemed to independently correlate with low provider intent-to-leave scores. Hosmer–Lemeshow goodness-of-fit test showed no statistically significant difference (p>0.05) indicating data fit the model well. When all three wellness markers were analyzed together, individual markers seemed to have fewer protective effects on provider intent-to-leave. Table 5 demonstrates the significant protective effects when high PF, low BO, and high PR were combined (p<0.001).

Table 4

Different wellness markers affecting patient and provider related outcomes.

	Adjusted Odds Ratio	95% Confidence Interval	P value
No Wellness Markers Affecting Provider Self-Reported Medical Errors
Provider
ED Physicians	Reference	Reference	Reference
ED APPs	0.34	0.21–0.56	<0.001
Two Wellness Markers Affecting Provider Intent-to-Leave Outcomes
Professional Fulfillment (PF)
Low	Reference	Reference	Reference
High	0.22	0.09–0.57	0.002
Burnout (BO)
High	Reference	Reference	Reference
Low	0.39	0.19–0.76	0.006

Table 5

Interactions among different wellness markers affecting provider intent-to-leave outcomes.

	Adjusted Odds Ratio	95% Confidence Interval	P value
Interactions among PF, BO, and PR
Low PF (↓), High BO (↑), Low PR (↓) (↓PF↑BO↓PR)	Reference	Reference	Reference
↓PF↑BO↑PR	1.49	0.43–5.19	0.535
↓PF↓BO↓PR	0.73	0.20–2.69	0.641
↑PF↑BO↓PR	6.58	0.17–251.8	0.311
↑PF↓BO↑PR	0.09	0.03–0.30	<0.001

Discussion

In this study, we found that ED physicians and APPs had similar professional fulfillment, burnout, and personal resilience scores. Moderate-to-high correlations were also found among these wellness domains in both physician and APP groups. They correlated better with provider intent-to-leave than self-reported medical errors. When all three wellness domains were analyzed together, it was noted that high professional fulfillment, low burnout, and high personal resilience tended to have a protective effect related to intent to leave current position. Our study findings link wellness domains to patient and provider outcomes and these findings provide important information to help future ED provider wellness programs improve quality and patient-centered care.

Clinicians, in general, tend to have higher burnout than other professions [24]. It is postulated that ED providers in particular should be investigated as a specialty differing from others due to the nature of rotating shift schedules and stressful work environment. ED providers face challenges of high patient volume and varying severity of illness, often high, which may contribute to levels of stress. Given similar working conditions, this might result in similar burnout levels between ED physicians and APPs. However, due to different responsibility of physicians and APPs, their self-reported medical errors might be different. In study EDs, physicians play supervising roles on APPs and answer any questions that APPs may ask though physicians might not see every APPs’ patients during their shift. The challenges of working rotating shift schedules (mornings, evenings, and nights) may also affect ED provider personal resilience. We found that older providers who tended to work fewer night shifts (due to group option to reduce or eliminate nights based on age) had higher personal resilience (low score) when compared to providers from younger groups (see S1 Table).

When each wellness domain was measured, it showed high internal consistencies similar to previous studies [4, 25, 26]. Since there are no similar report in the literature, we provide initial wellness measurements using the average scores in each domain and further divided scores into higher and lower categories. Our findings to determine wellness domain cutoffs were based on a previous study [4]. Since the previous study did not focus on Emergency Medicine (EM) wellness measurements (included only <1% of EM healthcare providers) [4], such cutoffs might not be accurate for ED providers. Therefore, correlations and interactions among different wellness domains and its association with patient/provider centered outcomes might be more important than simply determining cutoff levels of provider wellness.

While literature reports of physician wellness have shown correlation with medical errors [27-29], our findings showed no correlations between provider burnout and medical errors. Medical errors which occur in healthcare systems usually happen due to a chain of failures, the actions of a single provider may not fully account for this, as they only represent a portion of the medical error chain. Like the Swiss cheese model, the occurrence of medical errors usually happens due to a failure of the whole healthcare system, as opposed to the individual [30]. In addition, these findings may indicate an offset effect when providers have opposite conditions in different domains, under reporting, or decreased recollection of those events.

When intent-to-leave was measured, however, poor physician wellbeing correlated better with high intent-to-leave, consistent with previous reports [31, 32]. Intent-to-leave might be more controllable by the provider themselves; therefore, poor physician wellbeing might play an important role in provider willingness to remain at current job. Additionally, we did find synergistic effects occurred regarding provider intent-to-leave. There was significant protection if providers ranked high in professional fulfillment, low in burnout, and high in personal resilience (Table 5). Wellness domains are not isolated within any given provider. The potential exists that provider wellness conditions at different domains have either offset or synergistic effects. Therefore, these domains should not be analyzed in isolation.

This study has its limitations. Firstly, provider wellness includes numerous areas and this study only chose three domains for provider wellness measurements, which are limited. More provider wellness domains should be studied in the future. Secondly, at present, all the provider wellness measurements in this study were self-reported, subjective, and perhaps less accurate. Adding physiological or biomarkers to future wellness investigations, in addition to surveys, may produce valuable information. Thirdly, study outcome measurements were limited to provider intent-to-leave and self-reported medical errors, which again, is subjective. Fourthly, provider wellness conditions are multi-factorial, by only analyzing provider demographics, years of practice, and relations to significant others and children in this study, similar to other studies, examines only a portion of factors effecting wellness. Other confounders, which could potentially affect study results, were not analyzed in this study. Lastly, although this is the largest study to date investigating these wellness markers specific to ED providers our study sample was limited. The average of EM physicians in US is male, 40–50 age, with nearly 10 years of practicing in ED with moderate volume after the residency graduation [33]. However, our providers tend to be younger with less years of practice in the ED, which has some differences in comparison to national average. Therefore, our finding might lack of generalizability and require external validations. A large-scaled prospective multicenter study is warranted in the future to 1) measure provider wellness using both survey and biological markers; and 2) better and more accurately determine wellness conditions and its associations to patient/provider quality healthcare.

Conclusion

High correlations occurred among three different provider wellness markers in ED healthcare providers with no significant difference between physicians and advanced practice providers. Providers with higher professional fulfillment, lower burnout, and higher personal resilience tended to report lower likelihood to leave their current jobs.

Personal resilience from healthcare providers of different age groups.

(DOCX)

Click here for additional data file.

19 Jun 2020

PONE-D-20-11949

Fulfillment, Burnout and Resilience in Emergency Medicine– Correlations and Effects on Patient and Provider Outcomes

PLOS ONE

Dear Dr. Wang,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 03 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Andrew Carl Miller

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

3. Please include your tables as part of your main manuscript and remove the individual files. Please note that supplementary tables should be uploaded as separate "supporting information" files.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I think that this is a good article. However, I think that introduction should make emphasys on the description of the terms taht the study will evaluate.

Moreover there are different studies related with burnout and empathy in emergency professionals that could be interesting to be referred.

Reviewer #2: Thank you for the opportunity to review this timely and well-written article. More information about the original survey instrument and technique is required. Detailed feedback is as follows:

- Was a sample size calculation performed? If so, please provide.

- Please describe the informed consent process. Where were the participants told the length of time of the survey, which data were stored and where and for how long, who the investigator was, and the purpose of the study?

- If any personal information was collected or stored, describe what mechanisms were used to protect unauthorized access.

- State how the survey was developed, including whether the usability and technical functionality of the electronic questionnaire had been tested before fielding the questionnaire. If it was a previously validated/published survey, then provide the reference.

- Was the survey open or closed? An “open survey” is a survey open for each visitor of a site, while a closed survey is only open to a sample which the investigator knows (password-protected survey).

- How/where was the survey announced or advertised?

- Was the survey web-based or through email? If it is an e-mail survey, were the responses entered manually into a database, or was there an automatic method for capturing responses?

- Were any incentives offered (eg, monetary, prizes, or non-monetary incentives such as an offer to provide the survey results)?

- To prevent biases items can be randomized or alternated. Was this done?

- Did the survey make use of adaptive questioning?

- It is technically possible to do consistency or completeness checks before the questionnaire is submitted. Was this done, and if “yes”, how (usually JAVAScript)? An alternative is to check for completeness after the questionnaire has been submitted (and highlight mandatory items). If this has been done, it should be reported.

- State whether respondents were able to review and change their answers (eg, through a Back button or a Review step which displays a summary of the responses and asks the respondents if they are correct).

- Please define how you determined a unique visitor. There are different techniques available, based on IP addresses or cookies or both.

- What was the view rate? Calculation requires counting unique visitors to the first page of the survey, divided by the number of unique site visitors (not page views!). It is not unusual to have view rates of less than 0.1 % if the survey is voluntary.

- Line 162: Please clarify how response rate was determined. Is this what is typically called the participation rate, or is it the completion rate?

- What was the completion rate? The number of people submitting the last questionnaire page, divided by the number of people who agreed to participate (or submitted the first survey page). This is only relevant if there is a separate “informed consent” page or if the survey goes over several pages. This is a measure for attrition. Note that “completion” can involve leaving questionnaire items blank. This is not a measure for how completely questionnaires were filled in. (If you need a measure for this, use the word “completeness rate”.)

- Registration - In “closed” (non-open) surveys, users need to login first and it is easier to prevent duplicate entries from the same user. Describe how this was done. For example, was the survey never displayed a second time once the user had filled it in, or was the username stored together with the survey results and later eliminated? If the latter, which entries were kept for analysis (eg, the first entry or the most recent)?

- Line 92: please define advanced practice providers as some readers may be unfamiliar with this term.

- Statistical correction - Indicate whether any methods such as weighting of items or propensity scores have been used to adjust for the non-representative sample; if so, please describe the methods.

- How were duplicate surveys identified? Was there an IP check, log file analysis, etc.?

- Line 122: please provide the threshold/definition of patient harm.

- Line 122: please clearly define ‘wrong medication’. For example, did the wrong medication have to actually be administered? What about a wrong order that his caught and changed, for example after inquiry from pharmacy or nursing staff?

- Line 122: please clearly define ‘wrong order’. Did the wrong order have to be executed? What if it was placed and then deleted before execution?

- Line 164: please clarify what made these surveys ‘invalid’.

- Table 1: The data shows a respondent predominance of early-mid career white males working at mod-high volume centers. Does this match regional or national demographics in EM? Please comment in the discussion how this may influence the observed results.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

17 Jul 2020

AUTHORS RESPONSE TO EDITORS AND REVIEWERS

As requested, we have included the original letter and comments with our point by point response in red colored font.

PONE-D-20-11949

Fulfillment, Burnout and Resilience in Emergency Medicine– Correlations and Effects on Patient and Provider Outcomes

PLOS ONE

Dear Dr. Wang,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 03 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Andrew Carl Miller

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response: Yes, we revised our manuscript with the PLOS ONE style.

2. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

Response: Sorry for the confusion. This is a secondary data analysis study based on the data collected via a quality improvement project. This original project was contracted with the WellMD team at Stanford University. Therefore, the study team of this manuscript does not own the copyright of the data, and unable to authorize the upload of all available data to the public due to legal restrictions. The authors of the present study had no special access privileges in accessing the datasets as well. The consent signed by the clinicians taking the survey would preclude us from sharing the raw deidentified data. (See quotes from privacy statement below) However, data might be available upon request to group authorized personnel on a case by case scenario. The contact info is: Dr. Jyothindran, Director of Physician Wellness, Integrative Emergency Services, email: Revathi.jyothindran@bswhealth.org

Quote from the privacy statement:

The database will be stored on a password-protected, encrypted computer system that has limited access and is located in a locked office in a controlled facility. The staff will also delete the Provider Wellness Survey data from the Qualtrics system. Your personally identifiable data associated with the Provider Wellness Survey will be accessible only to the limited staff from the independent survey administrator that IES has appointed for the purposes of collecting and maintaining the data. Survey data will not be shared with your institution or others. Data will be de-identified by the survey administration staff. In addition, results from small work groups may be combined with those from larger groups to protect confidentiality and provide meaningful results. The de-identified data will also be shared with the national Provider Wellness Academic Consortium for purposes of benchmarking and program evaluation. None of your answers will be connected to you.

If you accept this privacy statement, the survey administrator will use your email address to link your responses across multiple years of planned Provider Wellness Survey administrations and to link these survey data to other data, such as participation data collected by IES for its health promotion programs. These linkages will allow IES to examine the relationship between survey variables and program participation, and plan future health promotion programming. Information regarding department/specialty will be used to compare work/life wellness and risks of different groups, and identify which health promotion programming will be most effective for different physician groups. Data collected from the online Provider Wellness Survey will be used for the following additional purposes, as permitted or required by law, including:

1. to provide norms against which sub-groups can be compared

2. to suggest recommendations for future physician health promotion programming and benefits design within IES

3. to identify the work/life wellness and risks of different physician groups and compare these risks and wellness variables with national data

4. to apply for, and/or justify, funding for programs that will help promote wellness among physicians and families participating in those programs

5. to investigate the relationship among variables of physicians' work/life wellness, lifestyle behaviors, knowledge, attitudes, mental and physical well-being, and quality of care metrics

6. potentially , to conduct research approved or deemed exempt by the IRB and approved by IES

When used for these purposes (1 through 6 above), your survey data will be combined with the data of other respondents and potentially, with data collected from other sources. Data will be analyzed and reported in aggregate to groups and individuals within IES, to certain other organizations (e.g., health plans), and potentially, to other audiences for research and publication purposes (with IRB approval or exemption). No personally identifying information will be included in such reports.

By agreeing to this privacy statement, you agree that your data from your partially or fully completed survey may be gathered, stored in the database, and used for the purposes described above.

How You Can Access, Change, or Delete Your Information. If you answer any of the survey questions, your responses will be recorded instantly and stored in Qualtrics. At the end of the survey administration period, your responses will be included in the database even if you have not finished the survey by clicking “submit.” If you would like your data changed or removed from the database, please email the independent survey administrator Dr. Jyothindran, at rjyothindran@ies.healthcare. After a dataset has been de-identified and delivered to IES, the survey administrator will not be able to remove individual respondents’ data from the shared de-identified dataset. This privacy statement will be updated periodically to reflect any material changes to our privacy policy.

3. Please include your tables as part of your main manuscript and remove the individual files. Please note that supplementary tables should be uploaded as separate "supporting information" files.

Response: Yes, we include all takes to the main manuscript and removed the individual files. In addition, we uploaded our supplementary tables separately.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: I Don't Know

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I think that this is a good article. However, I think that introduction should make emphasis on the description of the terms that the study will evaluate.

Moreover there are different studies related with burnout and empathy in emergency professionals that could be interesting to be referred.

Response: Yes, we revised our introduction and emphasized on the terms that described in this study (e.g. professional fulfillment, burnout, and personal resilience). Since this study mainly focused on three provider wellness domains (PF, BO, and PR), more emphasis on the relationship among these three domains were addressed. To discuss different studies related with burnout and empathy in emergency professions seem to be deviated from this study focus. In addition, provider wellness can be affected by multiple domains including empathy, which has not been addressed in this study. We think this might better fit for placing to the limitation section. We addressed this in our limitation section as the followings:

“This study has its limitations. Firstly, provider wellness includes numerous areas and this study only chose three domains for provider wellness measurements, which are limited. More provider wellness domains should be studied in the future.”

Reviewer #2: Thank you for the opportunity to review this timely and well-written article. More information about the original survey instrument and technique is required. Detailed feedback is as follows:

- Was a sample size calculation performed? If so, please provide.

Response: Integrative Emergency Services (IES) is a physician-owned group mainly providing EM service in the North Texas, USA. We intended to include all providers of our group for this study, therefore, due to epidemiology reason, a sample size calculation was not performed.

- Please describe the informed consent process. Where were the participants told the length of time of the survey, which data were stored and where and for how long, who the investigator was, and the purpose of the study?

Yes. The first page of the survey is the introduction to the survey, including the approximately length of time the survey can be completed. Informed consent was also provided on the second page of the survey, survey was not proceeded if providers declined to participate. The second page details the privacy statement and requires the user to provide consent for 1. Saving email address for future longitudinal projects. 2. Proceeding with the survey. The survey is collected on Qualtrics, but then stored in a database that is password protected and only accessible to The Risk Authority, the original creators of the survey. The information is then deleted off of the Qualtrics system. We revised and added them to the revised manuscript.

Quote from the privacy statement:

The database will be stored on a password-protected, encrypted computer system that has limited access and is located in a locked office in a controlled facility. The staff will also delete the Provider Wellness Survey data from the Qualtrics system. Your personally identifiable data associated with the Provider Wellness Survey will be accessible only to the limited staff from the independent survey administrator that IES has appointed for the purposes of collecting and maintaining the data. Survey data will not be shared with your institution or others. Data will be de-identified by the survey administration staff. In addition, results from small work groups may be combined with those from larger groups to protect confidentiality and provide meaningful results. The de-identified data will also be shared with the national Provider Wellness Academic Consortium for purposes of benchmarking and program evaluation. None of your answers will be connected to you.

If you accept this privacy statement, the survey administrator will use your email address to link your responses across multiple years of planned Provider Wellness Survey administrations and to link these survey data to other data, such as participation data collected by IES for its health promotion programs. These linkages will allow IES to examine the relationship between survey variables and program participation, and plan future health promotion programming. Information regarding department/specialty will be used to compare work/life wellness and risks of different groups, and identify which health promotion programming will be most effective for different physician groups. Data collected from the online Provider Wellness Survey will be used for the following additional purposes, as permitted or required by law, including:

1. to provide norms against which sub-groups can be compared

2. to suggest recommendations for future physician health promotion programming and benefits design within IES

3. to identify the work/life wellness and risks of different physician groups and compare these risks and wellness variables with national data

4. to apply for, and/or justify, funding for programs that will help promote wellness among physicians and families participating in those programs

5. to investigate the relationship among variables of physicians' work/life wellness, lifestyle behaviors, knowledge, attitudes, mental and physical well-being, and quality of care metrics

6. potentially , to conduct research approved or deemed exempt by the IRB and approved by IES

When used for these purposes (1 through 6 above), your survey data will be combined with the data of other respondents and potentially, with data collected from other sources. Data will be analyzed and reported in aggregate to groups and individuals within IES, to certain other organizations (e.g., health plans), and potentially, to other audiences for research and publication purposes (with IRB approval or exemption). No personally identifying information will be included in such reports.

By agreeing to this privacy statement, you agree that your data from your partially or fully completed survey may be gathered, stored in the database, and used for the purposes described above.

How You Can Access, Change, or Delete Your Information. If you answer any of the survey questions, your responses will be recorded instantly and stored in Qualtrics. At the end of the survey administration period, your responses will be included in the database even if you have not finished the survey by clicking “submit.” If you would like your data changed or removed from the database, please email the independent survey administrator Dr. Jyothindran, at rjyothindran@ies.healthcare. After a dataset has been de-identified and delivered to IES, the survey administrator will not be able to remove individual respondents’ data from the shared de-identified dataset. This privacy statement will be updated periodically to reflect any material changes to our privacy policy.

- If any personal information was collected or stored, describe what mechanisms were used to protect unauthorized access.

Response: We use standard mechanisms to protect unauthorized access as the followings: 1) any personal information linked to the data will be stored initially in the files with password protections, only PI can get access to the data; 2) initial data was recoded to generate a master data file, each provider will be assigned to a unique number with the deletion of all personal information; 3) all data will be destroyed 3 years after the completion of this project.

- State how the survey was developed, including whether the usability and technical functionality of the electronic questionnaire had been tested before fielding the questionnaire. If it was a previously validated/published survey, then provide the reference.

Response: Yes. This survey was a previously published survey. We revised and added the reference.

- Was the survey open or closed? An “open survey” is a survey open for each visitor of a site, while a closed survey is only open to a sample which the investigator knows (password-protected survey).

Response: This is a closed survey. Survey was sent to each provide via email. We revised our method in the manuscript.

- How/where was the survey announced or advertised?

Response: We sent the survey to all our group ED providers and encourage provider to participate but not mandatory.

- Was the survey web-based or through email? If it is an e-mail survey, were the responses entered manually into a database, or was there an automatic method for capturing responses?

Response: An email was sent to the providers with a link to the survey web. Therefore, data was automatically captured. We revised and addressed it in the method section.

- Were any incentives offered (eg, monetary, prizes, or non-monetary incentives such as an offer to provide the survey results)?

Response: No incentives was offered regardless of the participations.

- To prevent biases items can be randomized or alternated. Was this done?

Response: No, we sent the same items to all the participants.

- Did the survey make use of adaptive questioning?

Response: No, the survey did not use adaptive questions.

- It is technically possible to do consistency or completeness checks before the questionnaire is submitted. Was this done, and if “yes”, how (usually JAVAScript)? An alternative is to check for completeness after the questionnaire has been submitted (and highlight mandatory items). If this has been done, it should be reported.

Response: Unfortunately, no mandatory items required to be answered before the completion of this survey, therefore, no consistency or completeness checks done before the questionnaire is submitted.

- State whether respondents were able to review and change their answers (eg, through a Back button or a Review step which displays a summary of the responses and asks the respondents if they are correct).

Response: Yes, the respondents were able to review and change their answers before the final submission. We revised and added to the method of the manuscript.

- Please define how you determined a unique visitor. There are different techniques available, based on IP addresses or cookies or both.

Response: we defined our unique participant based on email and IP addresses. In addition, we also defined each participant with their email and IP address linked to their general information (such as practice location, years of practice, age, gender, etc.). We revised and addressed in our method section.

- What was the view rate? Calculation requires counting unique visitors to the first page of the survey, divided by the number of unique site visitors (not page views!). It is not unusual to have view rates of less than 0.1 % if the survey is voluntary.

Response: Unfortunately, we are not able to calculate the view rate of this study.

- Line 162: Please clarify how response rate was determined. Is this what is typically called the participation rate, or is it the completion rate?

Response: We determine the response rate as the following:

Response rate = (the number of providers submitted their survey) / (the number of providers received the initial email). In detail: we sent an initial email to 382 providers, and we received 326 survey reports, therefore, an 85% of response rate was found.

- What was the completion rate? The number of people submitting the last questionnaire page, divided by the number of people who agreed to participate (or submitted the first survey page). This is only relevant if there is a separate “informed consent” page or if the survey goes over several pages. This is a measure for attrition. Note that “completion” can involve leaving questionnaire items blank. This is not a measure for how completely questionnaires were filled in. (If you need a measure for this, use the word “completeness rate”.)

Response: Yes, we had the completion rate of 89% (289/326, we have 289 providers submitted the last questionnaire page with the final submission button and 37 providers declined participating by clicking decline button on the first page). We revised in our result section of the manuscript.

- Registration - In “closed” (non-open) surveys, users need to login first and it is easier to prevent duplicate entries from the same user. Describe how this was done. For example, was the survey never displayed a second time once the user had filled it in, or was the username stored together with the survey results and later eliminated? If the latter, which entries were kept for analysis (eg, the first entry or the most recent)?

Response: We allowed multiple entrances of the survey during the study period. If duplicated surveys were found, we intended to use the later one. However, such issue was not occurred in our study. We revised and addressed in our method section.

- Line 92: please define advanced practice providers as some readers may be unfamiliar with this term.

Response: Yes, we defined advanced practice providers in more detail in the method section of the manuscript.

- Statistical correction - Indicate whether any methods such as weighting of items or propensity scores have been used to adjust for the non-representative sample; if so, please describe the methods.

Response: Unfortunately, we did not use any methods to adjust for the non-representative sample since approximately 10% of sample are considered invalid with over half were empty surveys.

- How were duplicate surveys identified? Was there an IP check, log file analysis, etc.?

Response: Yes, we checked for duplicate surveys by using both IP check with the combination of participants’ general information provided. We revised in our method section.

- Line 122: please provide the threshold/definition of patient harm.

Response: Patient harm defined as any adverse events occurred that directly linked to the medical errors. We revised in our method section.

- Line 122: please clearly define ‘wrong medication’. For example, did the wrong medication have to actually be administered? What about a wrong order that his caught and changed, for example after inquiry from pharmacy or nursing staff?

Response: Yes, we addressed more on the definitions of “wrong medication” and “wrong order” and revised in the method section.

- Line 122: please clearly define ‘wrong order’. Did the wrong order have to be executed? What if it was placed and then deleted before execution?

Response: Yes, we revised and define clearer on “wrong order” in the method section.

- Line 164: please clarify what made these surveys ‘invalid’.

Response: Yes, we revised and clarified the “invalid” survey in the result section.

- Table 1: The data shows a respondent predominance of early-mid career white males working at mod-high volume centers. Does this match regional or national demographics in EM? Please comment in the discussion how this may influence the observed results.

Response: Thanks for reviewer’s valued comment. The average of EM physicians in US is male, 40-50 age, with nearly 10 years of practicing in ED with moderate volume after the residency graduation. However, our providers tend to be younger with less years of practicing in the ED, which has some differences in comparison to national average. Therefore, our finding might lack of generalizability and require external validations. We realized this limitation and addressed more in the discussion.

________________________________________

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

________________________________________

In compliance with data protection regulations, you may request that we remove your personal registration details at any time. (Remove my information/details). Please contact the publication office if you have any questions.

This electronic transmission and any attached files are intended solely for the person or entity to which they are addressed and may contain information that is privileged, confidential or otherwise protected from disclosure under applicable law. Any review, retransmission, dissemination or other use, including taking any action concerning this information by anyone other than the named recipient, is strictly prohibited. If you are not the intended recipient or have received this communication in error, please immediately notify the sender by return email and delete the original message from your system.

Submitted filename: AUTHORS RESPONSE TO EDITORS AND REVIEWERS (R1_7_2020).docx

Click here for additional data file.

6 Oct 2020

Fulfillment, Burnout and Resilience in Emergency Medicine– Correlations and Effects on Patient and Provider Outcomes

PONE-D-20-11949R1

Dear Dr. Wang,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sergio A. Useche, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have addressed all the comments i sent in the previous revision. I think that the paper is better right now. Congratulations

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

9 Oct 2020

PONE-D-20-11949R1

Fulfillment, Burnout and Resilience in Emergency Medicine– Correlations and Effects on Patient and Provider Outcomes

Dear Dr. Wang:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Sergio A. Useche

Academic Editor

PLOS ONE