Development and initial validation of the Japanese healthy work environment assessment tool for critical care settings.

AIM: This study aims to translate the Healthy Work Environment Assessment Tool (HWE-AT) into Japanese and evaluate its validity and reliability. DESIGN AND METHODS: The authors followed the guidelines for scale translation, adaptation, and validation in cross-cultural healthcare research. After translation and back-translation, a series of pilot studies were conducted to assess comprehensibility. Subsequently, an expert panel established the content validity. Content validity was calculated using the content validity index (CVI). Finally, we verified the construct validity and calculated the test-retest reliability.
RESULTS: The updated HWE-AT achieved sufficient comprehensibility after conducting the two pilot tests. Content validity was calculated using the scale-level CVI/average and all the items were 1.00. The content validity indices CFI and RMSEA were 0.918 and 0.082, respectively. Intraclass correlation coefficients for all dimensions ranged from 0.618 to 0.903, indicating acceptable test-retest reliability. Our findings suggest that the Japanese version of the HWE-AT has good validity and reliability.

Introduction

A healthy work environment is essential for providing high-quality care to patients. The work environment includes physical and psychosocial conditions that influence employee motivation, productivity, engagement, and collaboration with other employees [1]. According to the World Health Organization, a healthy workplace is one in which everyone works together to achieve a shared vision for the health and well-being of workers and the surrounding community. It also provides all workforce members with physical, psychological, social, and organizational conditions that protect and promote their health and safety [2]. In fact, some studies have reported that a poor work environment is associated with the provision of low-quality care [3, 4]. In turn, a healthy work environment, such as an environment with higher perceptions of authentic leadership, was associated with lower burnout and higher compassion satisfaction [5]. Effectively promoting a healthy work environment (HWE) has not only prevented burnout among nurses but also decreased medication errors and pressure ulcers [6, 7].

Furthermore, there has been a growing interest in improving the work environment during the COVID-19 pandemic; however, the lack of human and personal protective equipment during the pandemic makes it difficult to maintain a safe work environment, such by as reducing the risk of infection for nurses [8]. Even then, open-ended communication, especially in leadership, and a supportive work environment could increase resilience in workers during and after the pandemic [9, 10]. In addition, by improving the work environment, nurses may be able to maintain a positive outlook even during the temporary crisis of the pandemic [11].

The American Association of Critical-Care Nurses (AACN) recognizes that a healthy work environment (HWE) is integral for nurses to contribute optimally to patient care. Moreover, a healthy work environment is vital to every healthcare team member, and respect and care for others in the healthcare team are critical [12]. To build a healthy workplace, clear standards are required. The AACN has thus developed six standards to achieve an HWE: appropriate staffing, authentic leadership, effective decision making, meaningful recognition, skilled communication, and true collaboration [13]. In 2009, the AACN developed a web-based Healthy Work Environment Assessment Tool (HWE-AT) with 18 questions on these six standards; each item is rated on a five-point Likert scale, ranging from 1 (strongly disagree with the statement) to 5 (strongly agree with the statement) [14]. As for the score cutoff, the HWE-AT score guidelines determine that 1.00–2.99 is “Needs Improvement”, 3.00–3.99 is “Good”, and 4.00–5.00 is “Excellent” [15]. Previous studies have indicated that the HWE-AT has good validity and reliability in a critical care setting [16]. Additionally, this tool was validated not only for nurses but also for other healthcare professionals. Therefore, all professionals in the hospital should be able to use HWE-AT to evaluate HWE.

The AACN’s HWE-AT has been used to assess and improve work environments that motivate healthcare professionals to provide high-quality care, especially in the intensive care unit (ICU) in English-speaking countries [14] However, the Japanese HWE-AT has not been translated officially yet. Thus, this study aims to translate the HWE-AT into Japanese and evaluate its validity and reliability.

Methods

In this study, the Healthy Work Environment Assessment Tool was first translated into Japanese and validated for content validity. Further, it assessed for construct validity, reliability, and internal consistency.

Summary of the Japanese HWE-AT development

Illustrates the steps taken by the authors to develop the Japanese HWE-AT (Fig 1).

Fig 1

Summary of the HWE-AT-J development.

First, the HWE-AT-J was developed through translation and content validation. And then construct validity and reliability were conducted.

Translation process and content validity study

Translation procedure

The translation was initiated after obtaining permission from the AACN. To develop the translation, we followed Sousa’s guidelines [17]. Primarily, the original version of the HWE-AT was translated independently into Japanese by two bilingual nurses. One was a nurse educator who formerly lived in the U.S. as a postdoctoral fellow and had worked as a nurse manager in the ICU in Japan for 25 years. The other nurse has worked in the ICU settings for 16 years and has been registered as a certified nurse in intensive care by the Japanese Nursing Association. The two translated versions were integrated into one Japanese version of the HWE-AT (hereafter referred to as “draft version”), based on the discussion among the translation team. This team consists of a group of bilingual nurses: a nurse educator, another nurse educator working in the United States, two nurse managers, three ICU nurses, and a graduate student in the U.S. who had previously worked as a nurse in Japan. Subsequently, the integrated document was back-translated into English by a professional native English-speaking translator who was unfamiliar with HWE-AT. Subsequently, the original and back-translated versions were compared. Following this process, the translation team finalized a preliminary Japanese HWE-AT based on the draft version.

Pilot testing for clarification of the translated tools

A series of pilot studies were conducted to evaluate the clarity and validity of each item in the preliminary Japanese HWE-AT. We invited participants who were working as ICU nurses via one of the mailing lists of the Japanese Society of Education for Physicians and Trainees in Intensive Care. Forty nurses voluntarily participated in a web-based survey. Each participant was asked to rate the instructions for the scale and each item using a dichotomous answer option (clear or unclear). If more than 20% of participants rated an instruction/item as unclear, we modified it to improve clarity and understanding. We repeated these steps until most participants rated all the items as clear (80% or more rated the items as clear). In case of repeated tests, the same participant was not allowed to answer the survey again. Therefore, a pre-final version of the Japanese HWE-AT was developed.

Content validity testing for expert panel

The content validity of the pre-final Japanese HWE-AT was evaluated by 10 experts who were knowledgeable about this content and experienced in hospital settings. All of these experts have more than ten years of experience as critical care nurses. The expert panel was asked to assess each item of the instrument for content equivalence concerning how it relates to the HWE using the following scale: 1 = “not relevant,” 2 = “unable to assess its relevance,” 3 = “relevant but needs minor alteration,” and 4 = “very relevant and succinct.” We also requested that these 10 experts write any comments necessary about the clarity of the items and provide suggestions and recommendations to improve their formulation.

The item-level content validity index (CVI) was then calculated. The scale was dichotomized by combining answer options 3 and 4 to calculate the scale-level CVI/average (S-CVI/Ave). We then calculated the ratio by dividing the CVI/average by the total number of experts [18]. Subsequently, we developed the Japanese Healthy Work Environment Assessment Tool (HWE-AT-J).

Construct validity testing

Participant selection for construct validity

The authors collected additional data to establish construct validity. The eligible participants were nurses working in ICUs in Japan. A convenience sampling method was used, and the participants were recruited through the mailing lists of the Japanese Society of Intensive Care Medicine and the Japanese Society of Education for Physicians and Trainees in Intensive Care. We asked participants who had already completed the questionnaire to distribute the invitation to other local mailing lists or social networking sites. Data were collected between October and December of 2020. The questionnaire was web-based and anonymous. Each participant was asked to complete the HWE-AT-J and provide demographic characteristics, including gender, years of ICU experience, years of nursing experience, type of hospital, working unit, position, and qualification.

Construct validity testing

Confirmatory factor analysis (CFA) was performed to assess the theoretical connectedness and structural equivalence of the original and translated HWE-AT. CFA is a typical method for assessing the relationship between a factor and an observed variable based on a prior measure.

Reliability testing

Participant selection for reliability. We recruited participants using the same method utilized to evaluate construct validity. We sent messages to those who provided consent to be contacted for the test and retest surveys, collecting basic data characteristics and e-mail addresses.

We evaluated the reliability of the HWE-AT-J using the test-retest method. The participants were asked to complete the HWE-J. Two weeks after they answered the HWE-J (test), we asked participants to complete the HWE-AT-J again (retest). The reliability of the Japanese translated instrument was determined using Cronbach’s alpha as a measure of internal consistency of the items within the six domains of the HWE-AT-J. To estimate Cronbach’s alpha, we used a larger of the two cohorts to avoid statistical errors.

Sample size calculation for validity and reliability testing

Adequate statistical power contributes to observing authentic relationships in a dataset [19]. Minimum sample sizes in absolute numbers were the first rule of thumb, suggesting that any N > 200 offers adequate statistical power for data analysis [20]. Therefore, we set the sample size of the confirmatory factor analysis (CFA) to 200 and recruited participants. We also used Kaiser-Meyer-Olkin test to confirm that the sample size was appropriate [21].

For the reliability test, as an indicator of test-retest reliability, we used the two-way random-effects model of the intraclass correlation coefficient (ICC). We calculated the sample size of the ICC statistics based on the formula recommended by Zou [22] and set the null hypothesis as 0.6, alternative hypothesis = 0.8, alpha = 0.5, and test power = 0.8. Based on these numbers, 49 patients were required. Therefore, we selected 50 participants.

Data analysis

Participant characteristics were expressed as percentages and numbers, medians, and interquartile ranges (IQR) for non-normally distributed data or means and standard deviations (SD) for normally distributed data. Construct validity was evaluated using the Confirmatory Factor Analysis (CFA). We did not perform exploratory factor analysis (EFA) because as far as we know the HWE—AT was not developed using EFA [23]. Moreover, the purpose of this study was to consider structural equivalence between the Japanese version and the original version. Therefore, we chose to perform CFA. Under the CFA, comparative fit index (CFI), and root-mean-square error of approximation (RMSEA), two frequently used models, were adopted in this study [24]. CFI ranges between 0 and 1 and is generally appropriate at values of 0.90 [25]. RMSEA needs a cut-off value to be changed depending on the sample size, and in our study, we set the cut-off value to 0.1 [26]. We evaluated ceiling and floor effects for each question item. Additionally, we assessed the ceiling and floor effects corresponding to the percentage of respondents who obtained minimum (1) or maximum (5) for each question. Above 15% of respondents either minimum or maximum indicated there was a problem with validity [27].

We used the two-way random-effects model of the intraclass correlation coefficient (ICC) to measure test-retest reliability. ICCs < 0.5 indicate poor reliability; ICCs between 0.5 and 0.75 indicate moderate reliability; ICCs between 0.75 and 0.9 indicate good reliability; and ICCs > 0.9 indicate excellent reliability [28]. We also calculated Cronbach’s alpha as an internal consistency measure to ensure that the subscale questions measured similar concepts. Cronbach’s α of 0.7 to 0.8 is generally considered good [29].

Ethical considerations

Ethical approval for the research protocol was granted by the ethical review board of the Kanazawa Medical University Hospital, Ishikawa, Japan, (approval ID H275). For participant consent, an instruction sheet was attached on the web and a check box was provided so that participants could indicate their willingness to participate. The consent form had to be viewed and checked to be able to respond. In addition, contact information was attached so that participants could request verbal explanations or written consent, and this was handled on an individual basis.

Results

Translation and content validity

Pilot testing

In the first test, five items in the HWE-AT (1, 2, 6, 7, and 10) were rated as unclear by equal to or greater than 20% of participants. Based on these results, we considered ways to clarify expressions and revise the meaning of the items to improve comprehension. In particular, we tailored the item description, noting the U.S.-Japan differences in the scope of nursing practice and the name designations of nurse managers. Regarding the different position titles between the U.S. and Japan, we received permission from the AACN to adapt the names appropriately to the Japanese medical hierarchy. After the revision, we conducted a second pilot test with 40 nurses who did not participate previous survey. Consequently, this test showed that all items were clearly described (maximum lack of clarity rate of less than 20%).

Content validity testing

The expert panel evaluated the comprehensibility between the items of the original HWE-AT and the pre-final Japanese HWE-AT. Experts suggested a more appropriate expression of Japanese terms that were either not explicit or misleading. For content validity, S-CVI/Ave was 1.00. After establishing content validity, we called the HWE-AT translated into Japanese “HWE-AT-J.” as shown in S1 Table.

Characteristics of the participants for construct validity testing

The total number of participants was 202, and there were no missing values for any questionnaire item. We also evaluated the measures of sampling adequacy (MSA) using the Kaiser-Meyer-Olkin test for each item, and the results indicate that sample size was sufficient for factor analysis (meritorious:0.87~ marvelous:0.95), as shown in S2 Table.

Participant characteristics are shown in Table 1. Approximately 34% of the participants were female, and the largest number of participants had 10 to 15 years of ICU experience (n = 65). Nearly half of the participants worked in the university hospital settings (n = 92), and a majority worked in an ICU (n = 132). Among the participants, 61 were certified nurses, the largest number.

Table 1

Participant characteristics for construct validity and reliability testing.

Characteristic	Construct Validity Testing n = 202	cReliability Testing n = 50
	n (%)	n (%)
Gender
Male	134 (66)	35 (70)
Female (%)	68 (34)	15(30)
Years of ICU experience
<5	41 (20)	8 (16)
5–9	61 (30)	19 (38)
10–15	65 (32)	16 (32)
16–20	26 (13)	6 (12)
>20	9 (5)	1 (2)
Years of Nursing experience
<5	14 (7)	-
5–9	48 (24)	-
10–15	58 (29)	-
16–20	44 (21.5)	-
>20	38 (18.5)	-
Working unit
aICU	132 (65.5)	38 (76)
Emergency ICU	38 (19)	7 (14)
bCCU	27 (13)	5 (10)
ICU/CCU	1 (0.5)	0 (0)
Pediatric ICU	2 (1)	0 (0)
Stroke Care Unit	1 (0.5)	0 (0)
Surgical ICU	1 (0.5)	0 (0)
Hospital facilities
University hospital	92 (46)	29 (58)
Public hospital	35 (17)	7 (14)
National hospital	12 (6)	13 (26)
Private hospital	63 (31)	1 (2)
Position
Staff	147 (73)	31 (62)
Administrator	55 (27)	19 (38)
Qualification
Registered Nurse	99 (49)	31 (62)
Nurse Practitioner	3 (1)	0(0)
Certified Nurse Specialist	22 (11)	3 (6)
Certified nurse	61 (30)	15 (30)
Others	17 (8)	1 (2)

Note.

aICU = Intensive Care Unit

bCCU = Cardiac Care Unit

cReliability Testing = It does not collect information on years of nursing experience in reliability testing.

Ceiling and floor effect

The authors determined that the ceiling effect was observed when 0% to 14% of respondents rated “5”. The floor effect was determined when 7% to 40% of respondents rated as “0”. We found Q3, Q 4, Q7, Q8, Q10, Q14, Q16, Q17, and Q 18 had floor effects.

The mean scores and standard deviations are shown in Table 2. The indices of the CFA are listed in Table 3. The fit indices obtained were 0.918 for CFI and 0.082 for RMSEA.

Table 2

Means, standard deviations for each questionnaire and factor loading.

	Mean	a SD	bFactor loading
Skilled communication
Q.1: Maintain frequent communication	3.28	0.921	0.63
Q.6: Input seeking for decision-making	2.94	0.941	0.73
Q.14: Staff members let people know when they’ve done a good job	2.87	1.019	0.66
True collaboration
Q.2: Actions match words	3.18	0.929	0.77
Q.10: Enough staff to maintain patient safety	2.51	1.004	0.67
Q.15: Motivating opportunities for personal growth	3.02	0.972	0.82
Effective decision-making
Q.7: Consistent use of data-driven, logical decision-making process	2.75	0.951	0.80
Q.11: Right mix of nurses and other staff to ensure optimal outcomes	3.17	0.931	0.69
Q.16: Staff have positive relationship with nurse leaders	3.44	0.908	0.74
Appropriate staffing
Q.3: Zero tolerance for disrespect and abuse	3.10	1.067	0.78
Q.8: Right departments, professions, groups are involved	2.97	1.004	0.82
Q.12: Support services level allows nurses and staff to focus on care	3.05	0.971	0.68
Meaningful recognition
Q.4: Staff involved in decision-making	2.50	1.052	0.75
Q.9: Patient’s perspective is considered in important decisions	3.05	0.976	0.78
Q.17: Nurse leaders understand dynamics at point of care	2.94	1.016	0.78
Authentic leadership
Q.5: Able to influence policies, procedures, and bureaucracy	3.16	0.844	0.67
Q.13: Formal recognition system makes staff feel valued	3.01	0.980	0.82
Q.18: Nurse leaders play role in making key decisions	3.01	1.005	0.71

Note

aSD = Standard Deviations

Note

bFactor loading is standardized.

Table 3

CFA fit indices.

CFA fit indices
Factor	Indices
χ2	233
df	12
	p<0.01
bCFI	0.918
cRMSEA	0.082

Note

aCFA = Confirmatory Factor Analysis

Note

bCFI = comparative fit index

Note

cRMSEA = root-mean-square error of approximation.

Reliability testing

Participant characteristics

The total number of participants was 50, and there were no missing values for any questionnaire item. As shown in Table 1, the participants were about 30% female, and the largest number of participants had 5 to 9 years of ICU experience (n = 19). Fifty-eight percent of the participants worked in a university hospital setting (n = 29). Seventy-six percent of them worked in medical-surgical ICUs (n = 38). Among the participants, 15 were certified nurses, the largest number.

Cronbach’s α

We used the same data when we tested the construct validity to estimate Cronbach’s alpha (n = 202). As indicated in Table 4, the HWE-AT-J showed adequate reliability, as estimated by Cronbach’s α, for all domains: 0.607–0.811.

Table 4

Reliability testing: Cronbach’s α.

	Scale mean that items are deleted	Variance of frequencies that an item is deleted	Corrected items Total Correlation	Cronbach’s alpha for the case when an item is deleted
Domain Skilled Communications (Over all Cronbach’s α:0.707)
Q1	5.81	2.823	0.507	0.637
Q6	6.15	2.585	0.585	0.541
Q14	6.21	2.6.6	0.486	0.669
Domain True Collaboration (Over all Cronbach’s α:0.754)
Q2	5.53	2.847	0.621	0.629
Q10	6.20	2.966	0.488	0.780
Q15	5.69	2.662	0.647	0.594
Domain Effective Decision Making (Over all Cronbach’s α:0.786)
Q7	6.60	2.658	0.611	0.728
Q11	6.19	2.751	0.595	0.744
Q16	5.92	2.640	0.674	0.659
Domain Appropriate Staffing (Over all Cronbach’s α:0.799)
Q3	6.01	2.980	0.677	0.690
Q8	6.15	3.145	0.690	0.677
Q12	6.07	3.577	0.570	0.790
Domain Meaningful Recognition (Over all Cronbach’s α:0.811)
Q4	5.99	3.174	0.653	0.750
Q9	5.44	3.411	0.656	0.746
Q17	5.54	3.234	0.674	0.726
Domain Authentic Leadership (Over all Cronbach’s α:0.778)
Q5	6.03	3.024	0.622	0.697
Q13	6.17	2.661	0.655	0.654
Q18	6.17	2.781	0.574	0.748

Intraclass correlation coefficients

The ICCs are presented in Table 5. The ICCs for all dimensions ranged from 0.618 to 0.903. Therefore, it was shown to have moderate reliability.

Table 5

Intraclass correlation coefficient.

	aICC	95% CI	F test value	P value
Q1	0.762	0.580–0.865	4.156	<0.01
Q2	0.618	0.331–0.783	2.631	<0.01
Q3	0.811	0.666–0.893	5.208	<0.01
Q4	0.762	0.581–0.865	4.167	<0.01
Q5	0.756	0.573–0.861	4.146	<0.01
Q6	0.77	0.595–0.870	4.316	<0.01
Q7	0.787	0.622–0.88	4.638	<0.01
Q8	0.812	0.669–0.894	5.255	<0.01
Q9	0.83	0.700–0.904	5.785	<0.01
Q10	0.854	0.733–0.919	7.443	<0.01
Q11	0.854	0.744–0.917	7.002	<0.01
Q12	0.769	0.596–0.869	4.358	<0.01
Q13	0.71	0.487–0.836	3.401	<0.01
Q14	0.82	0.683–0.897	5.519	<0.01
Q15	0.771	0.596–0.870	4.316	<0.01
Q16	0.653	0.386–0.804	2.87	<0.01
Q17	0.903	0.829–0.945	10.699	<0.01
Q18	0.73	0.521–0.848	3.664	<0.01

Note. ICC = Intraclass Correlation Coefficient.

Discussion

The authors demonstrated the comprehensibility, validity, and reliability of the HWE-AT-J. We used Sousa’s translation guidelines [17] since we believe that the quality of the data obtained from the translated scale depends on the accuracy of the translation. We paid particular attention to the differences between the U.S. and Japanese healthcare systems and hierarchies within nursing positions. Differences in job titles between the two countries were particularly problematic. For example, the word “leader” in the U.S. indicates managers, certified nurse leaders, and advanced registered nurse practitioners. However, in Japan, it implies nurses in charge of a shift (equivalent to “charge nurses” in the U.S.). Therefore, we defined and included the titles of Japanese nurse leaders in hospitals, such as “chief nurses,” “certified nurses,” and “nurse managers.” A Certified nurse (CN) is those who received at least 600 hours of training in a special field and who have often served as leaders of a unit because of their specialized knowledge and expertise. Also, a Certified Nurse Specialist (CNS) has completed a master’s degree in a specific field. One research team member who is a registered nurse and had previously worked in an ICU in the U.S. helped clarify these differences.

Floor effects were observed in the following items (Q3, Q4, Q7, Q8, Q10, Q14, Q16, Q17 and Q18). As consistent with the original scale, a five-point Likert scale was used to measure the items. The result reflects the current situation of Japanese nursing practice that is hierarchical in nature. Therefore, the floor effect does not hamper the generalization of the scale adequacy.

The overall CVI score was excellent. Using web-based surveys as a data collection method, we obtained responses from ICU nurses with considerable variability in years of experience and roles. As a result, we believe that the data show variability in experience with diverse perceptions. In terms of validation, the model fit was moderate and considered acceptable. CFI and RMSEA were 0.918 and 0.082, respectively. A CFI value of 0.90 or higher is deemed acceptable, and a cut-off score of RMSEA is 0.1 [25, 26]. Therefore, our fit model was good. The factor loadings for each factor of HWE-AT-J showed that all items were acceptable, given the recommended cut-off value of 0.4 [30]. In a study of healthcare workers at a children’s hospital in the U.S., the range of the HWE-AT’s factor loading was 0.63–0.82 [23]. In other studies, the factor loading ranged from 0.485 to 0.824 [16]; therefore, we considered the HWE-AT-J factor loading more acceptable. Also, the result of sample size by using Kaiser-Meyer-Olkin test was appropriate [21].

Reliability was considered moderate and acceptable. There are no clear cut-off scores for ICC criteria, but it is generally considered that 0.61–0.80 is substantial [31]. In our study, the range was 0.618–0.903, which we believe is appropriate. Cronbach’s alpha coefficients ranged from 0.620 to 0.907, with some internal consistency. In the AACN HWE-AT study, Cronbach’s alpha was 0.97, indicating high internal consistency [30]. However, in previous studies, they ranged from 0.75–to0.86 [29], which showed lower internal consistency when compared to our survey. Therefore, we determined that reliability was acceptable.

Limitation

First, we generally followed Sousa’s guidelines, but we could not fully follow its back-translation step due to limited human resources. Specifically, the Sousa approach recommends comparing two back translations; however, we could only conduct a single back translation. Second, we used a convenience sampling method to recruit participants for CFA, which may have led to selection bias. However, the characteristics of the participants varied in terms of their ICU experience, the proportion of participants working in a university hospital setting, and the proportion of staff to administrators. Thus, we determined that the sampling method did not significantly affect our findings.

Clinical implications/Research implication

The HWE-AT-J provides a way to determine the health of the work environment in healthcare facilities (e.g., units, departments, hospitals). The tool will help identify and evaluate current standards. The HWE-AT score guidelines determine that 1.00–2.99 is “Needs Improvement”, 3.00–3.99 is “Good”, and 4.00–5.00 is “Excellent” [15]. If a unit falls below the standard, issues can be identified, and steps can be taken to resolve them. Moreover, AACN had previously surveyed HWE on a five-point scale between 2006 and 2018 [32]. Using the HWE—AT-J, we will be able to evaluate the current state in each unit.

The HWE-AT-J enables researchers to measure the health of nursing units’ work environments. Future studies can measure the health of the work environment in relation to patient care quality and its associated factors for a stratified sample of ICU nurses in Japan.

Conclusion

The Japanese version of the HWE-AT has good comprehensibility, validity, and reliability. The HWE-AT-J was the first Japanese translation of the HWE-AT, which showed promising preliminary results in creating a healthy environment in Japanese ICUs.

Japanese version healthy work environment assessment tool.

(DOCX)

Click here for additional data file.

Measures of sampling adequacy.

(DOCX)

Click here for additional data file.

(XLSX)

Click here for additional data file.

(XLSX)

Click here for additional data file.

11 Feb 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Fatih Özden, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. During your revisions, please note that a simple title correction is required: Remove "PLOS ONE Dear Editor-in-Chief". Please ensure this is updated in the manuscript file and the online submission information.

3. Please provide additional details regarding participant consent. In the Methods section, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

4. Thank you for stating the following financial disclosure:

“The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

At this time, please address the following queries:

a)        Please clarify the sources of funding (financial or material support) for your study. List the grants or organizations that supported your study, including funding received from your institution.

b)        State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

c)        If any authors received a salary from any of your funders, please state which authors and which funders.

d)        If you did not receive any funding for this study, please state: “The authors received no specific funding for this work.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

5. Thank you for stating the following in the Acknowledgments Section of your manuscript:

“The authors thank the Clinical Research Support Office of the Medical Research Institute at Kanazawa Medical University for their excellent support in this study.”

We note that you have provided additional information within the Acknowledgements Section that is not currently declared in your Funding Statement. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

“The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

6. Please ensure that you refer to Figure 1 in your text as, if accepted, production will need this reference to link the reader to the figure.

Additional Editor Comments:

Dear Authors,

Thanks for your submission. After the careful consideration regarding the comments of two reviewers, my decision is a major revision. Please carefully read and fulfill all the comments. Also, provide a point to point response file.

King Regards

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for asking me to review this article. The results are very interesting, the article is clearly written and organized, and I believe the article will be of interest to the readers.

There are some minor fixes to consider.

Introduction

On page 5, line 89, a few more recent studies examining the impact of the work environment on patient and employee health/outcomes should be included.

The Covid-19 pandemic has been an important factor affecting intensive care environments, workforce planning, rapid decision making, communication and collaboration. This is not mentioned at all in the article. Briefly, the effect of the pandemic on the intensive care work environment can be mentioned.

Background

Page 6, line 99. Does this measurement tool (HWE-AT) have a cutoff score? How is the score obtained from the scale interpreted? It should be explained briefly.

Methods/ Content validity

What technique was content validation done (Davis?)

Page 8, line 141. It is the expert opinion experienced in the hospital setting? Or were those with intensive care experience selected?

Participant Selection for Construct Validity

Is the delivery of care in intensive care units in Japan cascaded according to the severity of the patient condition? Which level (1st, 2nd or 3rd level) of the nurses participating in the study work in the intensive care unit?

Results

Pilot testing

Were 40 nurses participating in the study included in the sample of the study at this stage? It must be disclosed.

Sample size

Page 10, line 184. Has the sample size been confirmed by the Kaiser-Meyer-Olkin test as to whether the sample size is sufficient for factor analysis?

Before the discussion, the characteristics of the final form should be given briefly. It is as if any item was removed from the Japanese scale and the sub-factor structure was preserved.

Clinical Implications/Research Implication

Page 17, lines 301-302. “standard” should be explained, whether a unit falls below the standard will be evaluated according to a written/valid procedure or according to the score obtained from the scale!

Figure 1

“Testeing” should be fixed

Reviewer #2: #1

Since HWE-AT-j is an evaluation method derived from the original HWE-AT, it is necessary to discuss the comparison with the original HWE-AT (distribution of the sample, factor loading for each question, etc.).

#2

The author started the analysis with confirmatory factor analysis. Please explain why you did not consider factor structure, factor loading, etc. in the exploratory factor analysis.

#3

The reader cannot evaluate the ceiling effect and floor effect from table2. Please make it a general table (mean, SD).

#4

In table 5, Cronbach's increases when Q10 is deleted. Please explain this in the discussion. Is it the same trend in the original paper?

#5

(line 241) There is no mention of this in table2. Please correct it.

#6

Please list χ2 and df in table3. The researcher is free to use any index to some extent, but χ2 is usually necessary.

#7

What is the reason why the factor loading in table3 is not standardized?　In other papers, standardized factor loading is used.

#8

In Japan, what is the difference between a certified nurse and a specialist?　What kind of license is it?　Please describe.

#9

The percentages for staff and administration in table 4 are blank. Please correct it.

#10

Why don't you combine the characteristics in table 1 and 4 into the same table? This is a suggestion.

#11

Please attach the Japanese version of the questionnaire (HWE) prepared by the author to the supporting information.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: Review comments.docx

Click here for additional data file.

4 Apr 2022

Response to Editor and Reviewer

Journal Requirements:

Comment

#1

Please ensure that your manuscript meets PLOS ONE's style

requirements, including those for file naming. The PLOS ONE style

templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf

and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response

Thank you for pointing this out. We have made the corrections as per PLOS ONE’s style requirements.

Comment

#2

Please provide additional details regarding participant consent. In the Methods section, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was

waived by the ethics committee, please include this information.

Response

Thank you for your comment. We added these sentences.

------Revised Manuscript------

P12, Line221

For participant consent, an instruction sheet was attached on the web and a check box was provided so that participants could indicate their willingness to participate. The consent form had to be viewed and checked to be able to respond. In addition, contact information was attached so that participants could request verbal explanations or written consent, and this was handled on an individual basis.

Comment

#3

Thank you for stating the following financial disclosure:

“The funders had no role in study design, data collection and analysis,

decision to publish, or preparation of the manuscript.”

Response

Thank you for your comment. The authors received no specific funding for this work.

We apologize for any misleading representations. In addition, we got to advice about English expressions, so we added that part.

------Revised Manuscript------

These sentences were deleted.

P28, Line382

The authors thank the Clinical Research Support Office of the Medical Research Institute (institution name blinded for peer review) for their excellent support in this study.

These sentences were added.

P28, Line384

We would like to thank Dr. James Britton and the University of Miami School of Nursing and Health Studies for the editorial support.

Comment

#4

Please ensure that you refer to Figure 1 in your text as, if accepted, production will need this reference to link the reader to the figure.

Response

Thank you for your comment. We added these sentences.

------Revised Manuscript------

P6, Line 106

Reviewer #1

Thank you for asking me to review this article. The results are very interesting, the article is clearly written and organized, and I believe the article will be of interest to the readers. There are some minor fixes to consider.

Comment

#1

Introduction

On page 5, line 89, a few more recent studies examining the impact of the work environment on patient and employee health/outcomes should be included.

Response

Thank you for your comment. We have added a few more recent studies related to the impacts of work environment on patients and employees.

------Revised Manuscript------

P4, Line 67

In turn, a healthy work environment, such as an environment with higher perceptions of authentic leadership, was associated with lower burnout and higher compassion satisfaction [5]. Effectively promoting a healthy work environment (HWE) has not only prevented burnout among nurses but also decreased medication errors and pressure ulcers [6,7].

Comment

#2

Introduction

The Covid-19 pandemic has been an important factor affecting intensive care environments, workforce planning, rapid decision making, communication and collaboration. This is not mentioned at all in the article. Briefly, the effect of the pandemic on the intensive care work environment can be mentioned.

Response

Thank you for your comment. We have added material about the effect of the COVID-19 pandemic on work environment in intensive care units.

------Revised Manuscript------

P4, Line 72

Furthermore, there has been a growing interest in improving the work environment during the COVID-19 pandemic; however, the lack of human and personal protective equipment during the pandemic makes it difficult to maintain a safe work environment, such by as reducing the risk of infection for nurses [8]. Even then, open-ended communication, especially in leadership, and a supportive work environment could increase resilience in workers during and after the pandemic [9, 10] In addition, by improving the work environment, nurses may be able to maintain a positive outlook even during the temporary crisis of the pandemic [11].

Comment

#3

Background

Page 6, line 99. Does this measurement tool (HWE-AT) have a cutoff score?

How is the score obtained from the scale interpreted? It should be explained briefly.

Response

Thank you for your comment. We added a statement that the scale is a five point Likert scale and discussed the cutoff score of each question.

------Revised Manuscript------

P5, Line88

each item is rated on a five-point Likert scale, ranging from 1 (strongly disagree with the statement) to 5 (strongly agree with the statement) [14]. As for the score cutoff, the HWE-AT score guidelines determine that 1.00-2.99 is “Needs Improvement”, 3.00-3.99 is “Good”, and 4.00-5.00 is “Excellent” [15].

Comment

#4

Methods/ Content validity

What technique was content validation done (Davis?) Page 8, line 141. It is the expert opinion experienced in the hospital setting? Or were those with intensive care experienceselected?

Response

Thank you for your comment. We added the following sentence.

------Revised Manuscript------

P8, Line140

All of these experts have more than ten years of experience as critical care nurses.

Comment

#5

Participant Selection for Construct Validity

Is the delivery of care in intensive care units in Japan cascaded according to the severity of the patient condition? Which level (1st, 2nd or 3rd level) of the nurses participating in the study work in the intensive care unit?

Response

Thank you for your comment. There is no criteria dividing the ICU by severity of illness in Japan.

Comment

#6

Results

Pilot testing

Were 40 nurses participating in the study included in the sample of the study at this stage? It must be disclosed.

Response

Thank you for your comment. Forty nurses who participated in the study were not include in the sample of the study at this stage. We added the following sentence.

------Revised Manuscript------

P8, Line135

In case of repeated tests, the same participant was not allowed to answer the survey again.

Comment

#7

Sample size

Page 10, line 184. Has the sample size been confirmed by the Kaiser- Meyer-Olkin test as to whether the sample size is sufficient for factor analysis? Before the discussion, the characteristics of the final form should be given briefly. It is as if any item was removed from the Japanese scale and the sub-factor structure was preserved.

Response

Thank you for your comment. We determined the sample size using the Kaiser-Meyer-Olkin test. For this analysis, the sample size was sufficient. We added the following sentence and supplemental file 1.

------Revised Manuscript------

P10, Line 190

We also used Kaiser-Meyer-Olkin test to confirm that the sample size was appropriate [21].

P14, Line 252

We also evaluated the measures of sampling adequacy (MSA) using the Kaiser-Meyer-Olkin test for each item, and the results indicate that sample size was sufficient for factor analysis (meritorious:0.87~ marvelous:0.95), as shown in S1 Table.

P26, Line 345

Also, the result of sample size by using Kaiser-Meyer-Olkin test was appropriate [21].

Comment

#8

Clinical Implications/Research Implication

Page 17, lines 301-302. “standard” should be explained, whether a unit

falls below the standard will be evaluated according to a written/valid

procedure or according to the score obtained from the scale!

Response

Thank you for your comment. There are criteria based on the scoring guideline; thus we added the following sentences.

------Revised Manuscript------

P27, Line367

The HWE-AT score guidelines determine that 1.00-2.99 is “Needs Improvement”, 3.00-3.99 is “Good”, and 4.00-5.00 is “Excellent” [15]. If a unit falls below the standard, issues can be identified, and steps can be taken to resolve them. Moreover, AACN had previously surveyed HWE on a five-point scale between 2006 and 2018 [32]. Using the HWE - AT-J, we will be able to evaluate the current state in each unit.

Comment

#9

Figure 1

“Testeing” should be fixed

Response

P7, Line 106

Thank you for your comment. We fixed Fig 1.

Reviewer #2

Comment

#1

Since HWE-AT-j is an evaluation method derived from the original HWE-AT,

it is necessary to discuss the comparison with the original HWE-AT (distribution of the sample, factor loading for each question, etc.).

Response

Thank you for your useful comment, we discussed the comparison with the original HWE-AT.

Comment

#2

The author started the analysis with confirmatory factor analysis.

Please explain why you did not consider factor structure, factor loading,etc. in the exploratory factor analysis.

Response

Thank you for your reasonable advice.

We considered performing the EFA again; however, we decided not to perform this analysis because of the following reasons:

1. The original the HWE-AT was not developed using EFA.

2. The purpose of this study was to assess construct validity and reliability of the HWE-AT-J. Revising the original HWE-AT was not within the purpose of this study.

3. Thus far, no studies performed EFA on the HWE-AT in any language. Also, we determined that our study’s purpose will be achieved using only CFA.

We added the reasons we did not perform EFA in the method section.

------Revised Manuscript------

P11, Line201

We did not perform exploratory factor analysis (EFA) because as far as we know the HWE - AT was not developed using EFA [23]. Moreover, the purpose of this study was to consider structural equivalence between the Japanese version and the original version. Therefore, we chose to perform CFA.

Comment

#3

The reader cannot evaluate the ceiling effect and floor effect from

table2. Please make it a general table (mean, SD).

Response

Thank you for your useful comment. We revised table 2 according to your comment to make it easier to understand for the readers. We deleted the ceiling and floor effects as a part of the result and discussion, and revised Table 4. We also added new sentences.

------Revised Manuscript------

Revised table2

・P18, Line282

These sentences were deleted.

・P17, Line266

The authors checked for ceiling and floor effects; when SD was added or subtracted from the mean value, none of the items exceeded the maximum value of 5 or were below the minimum value of 1, as shown in Table 2. Therefore, the results showed no significant bias in the distribution of responses to each questionnaire item.

・P26,Line343

Additionally, there is no ceiling or floor effect; thus, the scale is considered acceptable.

These sentences were added.

・P17, Line270

The authors determined that the ceiling effect was observed when 0% to 14% of respondents rated “5”. The floor effect was determined when 7% to 40% of respondents rated as “0”. We found Q3, Q 4, Q7, Q8, Q10, Q14, Q16, Q17, and Q 18 had floor effects.

・P25, Line329

Floor effects were observed in the following items (Q3, Q4, Q7, Q8, Q10, Q14, Q16, Q17 and Q18). As consistent with the original scale, a five-point Likert scale was used to measure the items. The result reflects the current situation of Japanese nursing practice that is hierarchical in nature. Therefore, the floor effect does not hamper the generalization of the scale adequacy.

Comment

#4

In table 5, Cronbach's increases when Q10 is deleted. Please explain

this in the discussion. Is it the same trend in the original paper?

Response

Thank you for your critical comment.

The original paper[1] does not show internal consistency in detail, and we do not think it is comparable.Our study purpose was to use the same specification method as the original study, and to maintain consistencey with the original HWE-AT study, we did not remove any specific questions.

1. Connor JA, Ziniel SI, Porter C, Doherty D, Moonan M, Dwyer P, et al. Interprofessional use and validation of the AACN Healthy Work Environment Assessment Tool. Am J Crit Care. 2018;27(5): 363–371. https://doi.org/10.4037/ajcc2018179.

Comment

#5

(line 241) There is no mention of this in table2. Please correct it.

Response

Thank you for your comment. We mentioned Table 2 in text.

------Revised Manuscript------

These sentences were deleted.

・P17, Line275

The mean scores, standard deviations, and factor loadings are presented in Table 2.

These sentences were added.

・P17, Line276

The mean scores and standard deviations are shown in Table2.

Comment

#6

Please list Χ2 and df in table3. The researcher is free to use any index

to some extent, but Χ2 is usually necessary.

Response

Thank you for your comment. We added X2 and df values in Table 3.

------Revised Manuscript------

Revised table3

P20, Line286

Comment

#7

What is the reason why the factor loading in table3 is not standardized?

In other papers, standardized factor loading is used.

Response

Thank you for pointing that out. We changed the standardized factor loading as table2.

------Revised Manuscript------

Revised table2

・P18, Line282

Comment

#8

In Japan, what is the difference between a certified nurse and a specialist?

What kind of license is it? Please describe.

Response

Thank you for asking this question. In Japan, a CN has received at least 600 hours of training in a specific field, and a CNS has completed a master's degree in a specific field.

------Revised Manuscript------

P24, Line323

A Certified nurse (CN) is those who received at least 600 hours of training in a special

field and who have often served as leaders of a unit because of their specialized

knowledge and expertise. Also, a Certified Nurse Specialist (CNS)has completed a

master’s degree in a specific field.

Comment

#9

The percentages for staff and administration in table 4 are blank.

Please correct it.

Response

Thank you very much for letting us know. We revised the tables and combined the information in Table 4 with Table 1.

------Revised Manuscript------

Revised table1

・P14, Line261

Comment

#10

Why don't you combine the characteristics in table 1 and 4 into the same

table? This is a suggestion.

Response

Thank you for your comment. We combined Tables 1 and 4, and renumbered the tables accordingly.

------Revised Manuscript------

Revised table1

・P14, Line 261

#11

Please attach the Japanese version of the questionnaire (HWE) prepared

by the author to the supporting information.

Response

Thank you for your comment. We attached the Japanese version of HWE as a supplemental file 2.

------Revised Manuscript------

S2 Table

Reference

Red articles were added after first submission

1. Massoudi DAH, Hamdi DSSA. The consequence of work environment on employees productivity. IOSR JBM. 2017;19(1): 35–42. https://doi.org/10.9790/487X-1901033542.

2.Thing R, et al. WHO Healthy Workplace Framework and Model Synthesis Report; 2010. Available:

https://apps.who.int/iris/bitstream/handle/10665/113144/9789241500241_eng.pdf.

3.McHugh MD, Rochman MF, Sloane DM, Berg RA, Mancini ME, Nadkarni VM, et al. Better nurse staffing and nurse work environments associated with increased survival of in-hospital cardiac arrest patients. Med Care. 2016;54(1): 74–80. https://doi.org/10.1097/MLR.0000000000000456.

4.Olds DM, Aiken LH, Cimiotti JP, Lake ET. Association of nurse work environment and safety climate on patient mortality: A cross-sectional study. Int J Nurs Stud. 2017;74: 155–161. https://doi.org/10.1016/j.ijnurstu.2017.06.004.

5. Kelly L, Todd M. Compassion fatigue and the healthy work environment. AACN Adv. Crit. Care. 2017; 28(4):351–274. https://doi.org/10.4037/aacnacc2017283.

6. Manojlovich, M. De Cicco, B. Healthy work environment, nurse-physician communication, and patients’ outcome. Am Crit Care .2007; 16(6): 536–543.

7. Manojlovich, M, Antonakos, C L, Ronis, D L. Intensive care units, communication between nurses and physicians, and patients' outcomes. Am. J. Crit. Care. 2009; 18(1): 21–30. https://doi.org/10.4037/ajcc2009353

8. Cohen J, Rodgers, Y M. Contributing factors to personal protective equipment s hortages during the COVID-19 pandemic. Prev. Med. 2020; 141. 1–24. https://doi.org/10.1016/j.ypmed.2020.106263

9. Chatzittofis A, Constantinidou A, Artemiadis A, Michailidou K, et al. The role of perceived organizational support in mental health of healthcare workers during the COVID-19 pandemic: A cross-sectional study. Front. Psychiatry. 2021; 12. https://doi.org/1–6. 10.3389/fpsyt.2021.707293.

10. Shah M, Roggenkamp M, Ferrer L, Burger V, et al. Mental health and COVID-19: The psychological implications of a pandemic for nurses. Clin. J. Oncol. Nurs. 2021; 25(1). 69–75. https://doi.org/10.1188/21.CJON.69-75.

11. Sezgin D, Dost A, Esin M N. Experiences and perceptions of Turkish intensive care nurses providing care to Covid-19 patients: A qualitative study. Int. Nurs. Rev. 2021; 1–13. https://doi.org/10.1111/inr.12740.

12.Munro CL, Hope AA. Healthy work environment: Resolutions for 2020. Am J Crit Care. 2020;29(1): 4–6. https://doi.org/10.4037/ajcc2020940.

13. American Association of Critical Care Nurses. AACN Standards for establishing and sustaining healthy work environments: A Journey to excellence. American Association of Critical Care Nurses; 2005. https://doi.org/10.4037/ajcc2005.14.3.187.

14. American Association of Critical Care Nurses. AACN Healthy Work Environment Assessment Tool; 2009. Available: https://www.aacn.org/nursing-excellence/healthy-work-environments/aacn-healthy-work-environment-assessment-tool.

15. American Association of Critical are Nurses. AACN Healthy Work Environment

Assessment. Team Assessment Results. Available: https://www.aacn.org/WD/HWE/Docs/HWESampleAssessment.pdf

16.Huddleston P, Gray J. Measuring nurse leaders’ and direct care nurses’ perceptions of a healthy work environment in an acute care setting, part 1: A pilot study. J Nurs Adm. 2016;46(7–8): 373–378. https://doi.org/10.1097/NNA.0000000000000361.

17. Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: A clear and user-friendly guideline. J Eval Clin Pract. 2011;17(2): 268–274. https://doi.org/10.1111/j.1365-2753.2010.01434.x.

18.Yusoff MSB. ABC of content validation and content validity index calculation. Educ Med J. 2019;11(2): 49–54. https://doi.org/10.21315/eimj2019.11.2.6.

19.Kyriazos TA. Applied psychometrics: Sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology. 2018;9: 2201–2230. https://doi.org/10.4236/psych.2018.98126.

20.Hoe SL. Issues and procedures in adopting structural equation modeling technique. J Appl Quant Methods. 2008;3: 76–83. Available:

https://ink.library.smu.edu.sg/sis_research/5168.

21.Kaiser, Rise. Little Jiffy Mark IV. Educational and Psychological Measurement.

1974; 34, 111-117. https://doi.org/10.1177/001316447403400115.

22.Zou GY. Sample size formulas for estimating intraclass correlation coefficients with precision and assurance. Stat Med. 2012;31(29): 3972–3981. https://doi.org/10.1002/sim.5466.

23. Connor JA, Ziniel SI, Porter C, Doherty D, Moonan M, Dwyer P, et al. Interprofessional use and validation of the AACN Healthy Work Environment Assessment Tool. Am J Crit Care. 2018;27(5): 363–371.

24.McDonald RP, Ho MH. Principles and practice in reporting structural equation analyses. Psychol Methods. 2002;7: 64–82. https://doi.org/10.1037/1082-989X.7.1.64.

25.Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model. 1999;6(1): 1–55. https://doi.org/10.1080/10705519909540118.

26.Kenny DA, Kaniskan B, McCoach DB. The performance of RMSEA in models with small degrees of freedom. Sociol Methods Res. 2015;44(3): 486–507. https://doi.org/10.1177/0049124114543236.

27. McHorney, C. A., Tarlov, A. R. Individual-patient monitoring

in clinical practice: Are available health status surveys adequate?

Qual Life Res. 1995; 4(4): 293–307. https://doi.org/10.1007/

BF015 93882.

28. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation

coefficients for reliability research. J Chiropr Med. 2016;15(2): 155–163. https://doi.org/10.1016/j.jcm.2016.02.012.

29.Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16: 297–334. Available: https://link.springer.com/content/pdf/10.1007/BF02310555.pdf. DOI: 10.1007/BF02310555.

30. Maskey R, Fei J, Nguyen HO. Use of exploratory factor analysis in maritime research. Asian J Ship Logist. 2018;34(2): 91–111. https://doi.org/10.1016/j.ajsl.2018.06.006.

31. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33: 159–174. https://doi.org/10.2307/2529310.

32. Beth U, Connie B, Linda C,et al.Crinical care nurse work environment 2018: Findings and implications.2019; 39(2), 67-84. https://doi.org/10.4037/ccn2019605.

mp

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

25 Apr 2022

Development and Initial Validation of the Japanese Healthy Work Environment Assessment Tool for Critical Care Settings

PONE-D-21-39634R1

Dear Dr. Kitayama,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Fatih Özden, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

4 May 2022

PONE-D-21-39634R1

Development and Initial Validation of the Japanese Healthy Work Environment Assessment Tool for Critical Care Settings

Dear Dr. Kitayama:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Fatih Özden

Academic Editor

PLOS ONE