Psychological Distress in Healthcare Workers During the Covid-19 Pandemic: Development and Validation of the Italian Version of the Healthcare Workers Emergency Distress Questionnaire (HEDQ).

Objective: The COVID-19 pandemic has presented unprecedented challenges for health systems as it has proven to be an extraordinary emergency. Exposure to such chronic stress can have detrimental effects on the psychophysical well-being of healthcare workers, with possible manifestation of stress-related symptoms. The present study aimed to develop and validate the Healthcare Workers Emergency Distress Questionnaire (HEDQ), a self-report screening questionnaire with the purpose of identifying the healthcare workers at risk for psychological and moral distress, and PTSD-related symptoms during emergencies. Method: The HEDQ was administered to 250 healthcare workers of the AULSS 3 Serenissima and 47 citizens from the general population in Venice (Italy) during the health emergency period (from April to June 2020). In the second administration (from August to September 2020), data were collected from 101 of the 250 healthcare workers who had participated in the first administration. To test for the convergent validity, the Depression Anxiety Stress Scale (DASS-21) was also administered.
Results: Exploratory and hierarchical confirmatory factor analyses validated the 21-item structure of the questionnaire. Internal consistency, and factorial and convergent validity were good. Moreover, the HEDQ discriminated between those who worked in COVID-19 hospital units and those who did not (including the general population), showing good known-group validity. The two-month temporal stability of the questionnaire was excellent. The HEDQ scale scores significantly decreased from the first to the second administration, thus supporting that the tool is a measure of acute stress in healthcare workers. Conclusions: Our results provide support for the use of the HEDQ as a brief, multidimensional measure of emergency-related stress reactions in healthcare workers. Consequently, the HEDQ can be considered a useful instrument supporting clinical activity to identify those who may be more easily affected by stress reactions in the event of atypically high levels of risk exposure during crises.

1. Introduction

The recent COVID-19 pandemic and its rapid global spread have presented unprecedented challenges for health systems as it has proven to be an unexpected and extraordinary emergency. First, exposure to chronic stress, such as the prolongation of the COVID-19 health emergency, may have a detrimental impact on the psychophysical well-being of front-line health workers, with possible manifestation of stress-related symptoms such as sleep disorders, chronic fatigue and somatic symptoms (headache, muscle tension, etc.), lack of concentration and memory deficits, behavioral alterations, difficulties in interpersonal relationships, worries and negative emotions caused by frustration, isolation and discrimination, reactive anxiety and depression (Albott et al., 2020; Benedek et al., 2007; Chen et al., 2020; Kang et al., 2020; Lai et al., 2020; Lee et al., 2007; MHPSS, I.R., 2012; Preti et al., 2020; Rajkumar, 2020).

Moreover, implementation of dysfunctional stress compensation strategies, such as the underestimation of the risk perception, is a common mechanism of avoidance and unhealthy behaviors (e.g., increasing alcohol, tobacco, or drugs consumption). Such strategies are mostly observed in quarantined hospital staff, as these workers are more likely to report symptoms of acute stress disorders (Bai et al., 2004; Brooks et al., 2020; Liu et al., 2012; Marjanovic et al., 2007; Wu et al., 2008, 2009).

Furthermore, the COVID-19 health emergency has amplified the perception of uncertainty and burden in hospital settings (Ibar et al., 2021), thus causing some healthcare workers to develop symptoms of burnout (Liberati et al., 2021). This is composed of three key dimensions – exhaustion, cynicism, and inefficacy – and may have long-ranging negative implications for workplace morale, healthcare professionals’ well-being, quality of care, and healthcare costs (Maslach et al., 2001; Noseworthy et al., 2017).

Other significant risk factors for healthcare workers' psychological distress are exposure to high rates of death – known as vicarious traumatization – and the perception of delivering inappropriate or insufficient care, which is a core part of moral distress (Burghi et al., 2014). This is defined as stress or worry experienced by healthcare workers when they know the right thing to do, but, for several reasons (e.g., institutional constraints or fear), they do not or cannot pursue the ethically correct action (e.g., Wilkinson, 1987). Such condition was found to be a crucial factor in threatening the physical and psychological well-being of health professionals, thus reducing the quality of patients care, too (Giannetta et al., 2020).

During the COVID-19 pandemic, the level of moral distress experienced by front-line healthcare workers has significantly increased; in fact, sudden changes in the work environment, in terms of resource reduction and work overload, have magnified all of those moral issues that were already a source of daily concern for them (Williams et al., 2020). Silverman et al. (2021) identified themes and sub-themes representative of major causes of moral distress in nurses caring COVID-19 patients. These included (a) lack of knowledge and uncertainty regarding how to treat a new illness; (b) being overwhelmed by the depth and breadth of the COVID-19 illness; (c) fear of exposure to the virus leading to suboptimal care; (d) adopting a team model of nursing care that caused intra-professional tensions and miscommunications; (e) policies to reduce viral transmission (visitation policy and PPE policy) that prevented nurses to assume their caring role; (f) practicing within crisis standards of care; and (g) dealing with medical resource scarcity.

Therefore, in order to prevent adverse outcomes for patients, healthcare workers and work environment, it is of crucial importance to promptly identify staff who are facing moral distress (Miljeteig et al., 2020), as also pointed out by the literature about past health emergencies (Smith et al., 2018; Ulrich, 2014).

All of the abovementioned risk factors have been demonstrated to affect a high number of healthcare workers during a state of emergency, with immediate repercussions on the management of the health emergency as well as on the individual’s health and well-being. Consequently, the World Health Organization (WHO) and the Italian Superior Health Institute (Istituto Superiore di Sanità; ISS), through their guidelines and related scientific literature, are stressing the importance of providing psychological aid resources and support for distress management to help those who affected by this emergency on a daily basis (De Mei et al., 2020; INAIL & Consiglio Nazionale Ordine Psicologi (CNOP), 2020; Kang et al., 2020; Liu et al., 2020; Rajkumar, 2020; World Health Organization [WHO], 2020).

1.1. Contextual factors and the current study

Based on the above premises, we created an ad hoc questionnaire, called the Healthcare Workers Emergency Distress Questionnaire (HEDQ), in order to guide clinical practice and identify which psychological interventions are the most effective in supporting the psychological well-being of health workers. Specifically, the development of a new tool arises from the need to accurately detect acute stress and its several manifestations (physical, cognitive, emotional, behavioral, and social symptomatology) experienced by healthcare workers during an unexpected and extraordinary emergency, such as the COVID-19 worldwide pandemic. The HEDQ, adapted from valid and reliable instruments, investigates experienced psychological distress in terms of healthcare workers' thoughts, emotions, and behaviors in everyday and working life when dealing with an emergency. Moreover, the questionnaire was designed to take into account the level of personal exposure to the pandemic situation and the presence of self-protective factors in working and daily life (perception of control, self-efficacy, family support, coping, sense of trust in the working team etc.). These elements have been demonstrated to be crucial in promoting resilience in populations affected by mass trauma (Hobfoll et al., 2007).

2. Materials and Methods

2.1 Sample and data collection procedures

To highlight the differences in the type and severity of psychological distress experienced by hospital healthcare workers and the community, data were collected from the healthcare workers of AULSS 3 Serenissima and the community living in Venice (Italy) during the early stages of the pandemic (from April to June 2020) through an internet-based self-report survey. The participation was on a voluntary basis, and participants could withdraw their consent and exit the study at any time without any penalization.

Multiple recruitment strategies were implemented. Specifically, participants from the general population were recruited using advertisements on social media and social networks. Advertisements included the contact details of the researchers conducting the survey, the description of the study purposes, and inclusion criteria for participating (i.e., Italian citizenship, age > 18 years, and not be a healthcare worker). For participants belonging to the population of healthcare workers, information regarding the survey was emailed to CEOs and departmental directors of frontline areas (e.g., emergency medicine, critical care, respiratory medicine, general medicine, infectious diseases, and hospital aged care) belonging to all the hospital facilities of AULSS 3 Serenissima.

The Scientific Ethics Committee for Clinical Trials approved the data collection protocol. All respondents consented, prior to beginning the survey, to the processing and communication of their personal data, within the limits, for the study purposes and for the duration specified by current laws (Legislative Decree 196/2003 and subsequent amendments and additions and EU GDPR 679/2016).

The sample of 297 adults included 250 (84.1%) healthcare workers and 47 (15.8%) citizens from the general population. Participants were 71.1% female and 27.6% male, and 1.3% did not identify themselves as either female or male. Respondents were aged 20 to 68 years (M = 45.2 years, SD = 10.9). The healthcare workers sample was further divided into "COVID unit" (N = 168) and "not COVID unit" (N = 82) sub-samples. The “COVID unit” sub-sample was composed of 48 men (age range: 26-61, M = 44 years, SD = 10.9), 120 women (age range: 24-64, M = 46.1 years, SD = 10.32) and 3 participants who do not identify themselves as either female or male (age range: 30-62, M = 46.8 years, SD = 9.8). The “not COVID unit” sub-sample was composed of 16 men (age range: 30-67, M = 45.6 years, SD = 10.9) and 66 women (age range: 27-68, M = 46.9 years, SD = 9.9). The general population sample was composed of 18 men (age range: 19-68, M = 45 years, SD = 11.2), 28 women (age range: 20-67, M = 39.6 years, SD = 12.2) and 1 participant who did not identify as either female or male (aged 28).

2.2. Measures

Candidate HEDQ items

First, in order to identify significant domains and generate adequate items, we used a “deductive method” (Boateng et al., 2018): we examined the relevant scientific literature about the psychological impact of the worldwide COVID-19 pandemic in healthcare workers, in particular as regards risk and protective factors for psychological distress and PTSD-related symptoms (e.g., Buselli et al., 2020; Lai et al., 2020; MHPSS, I. R., 2012; Preti et al., 2020).

Taking into account our purpose of identifying the healthcare workers at risk for psychological and moral distress, and PTSD-related symptoms during emergencies, the following domains were identified: 1) PTSD risk factors (i.e., intrusion symptoms, avoidance, hypervigilance); 2) Somatic-Affective Symptomatology (i.e., alterations in arousal and reactivity levels and physiological and emotional reactions); 3) Moral distress (i.e., problems stemming from ethical recommendations; organizational problems due to personnel shortage and problems caused by time pressure); 4) Protective factors (perceived support from family and friends; hobbies); and 5) Team support (sense of trust in the working team).

In order to detect the items suitable for investigating the above-reported domains, the following existing instruments were analyzed since they were considered particularly useful to assess different facets of distress in healthcare workers, especially during health emergencies:

1) The Impact of Event Scale-Revised (IES-R; Pietrantonio et al., 2003), which is a tool for assessing subjective distress caused by traumatic events, as experienced in the past seven days. The IES-R has 22 items (Weiss & Marmar, 1997), 5 of which were added to the original IES (Horowitz et al., 1979) to better capture the American Psychiatric Association (APA) and Diagnostic and Statistical Manual of Mental Disorders (DSM) criteria for PTSD. This 22-item scale is composed of three dimensions, namely: Intrusion, Avoidance, and Hyperarousal. The IES-R was designed with a five-point Likert scale rated from 0 (“Not at all”) to 4 (“Extremely”). A score ranging from 0 to 13 indicates the absence of PTSD, while a score above 24 indicates the possible presence of PTSD (Asukai et al., 2002; Beck et al., 2008; Creamer et al., 2003). For each subscale, test-retest reliability (Pearson’s r ranging from –.89 to .94) and internal consistency (Intrusion: Cronbach’s α ranging from .87 to .94; Avoidance: Cronbach’s α from .84 to .97; Hyperarousal: Cronbach’s α from .79 to .91) were overall acceptable (Creamer et al., 2003). The IES-R scale scores were found to be intercorrelated, with Pearson’s r coefficients being moderate to strong in magnitude (r ranging from .52 to .87; Beck et al., 2008). Moreover, high correlations were found between the Intrusion and Avoidance scales of the IES-R and the original IES (Pearson’s r = .86 and .66, respectively), thus supporting a good concurrent validity of both measures (Beck et al., 2008).

The Italian version of the IES-R (Pietrantonio et al., 2003) also showed good to excellent internal consistency values (α = .84 for Intrusion, α = .71 for Avoidance, α = .78 for Hyperarousal).

The IES-R has been used in previous outbreaks such as SARS, Swine Flu, and currently during the COVID– 19 pandemic (Hao et al., 2020; Lee et al., 2018; Matsuishi et al., 2012; Wang et al., 2020). Specifically, a recent study (Aljaberi et al., 2021) suggested that IES-R may be a reliable screening instrument for measuring traumatic distress related to the global pandemic of COVID-19 and can be utilized to provide timely psychological health support on the basis of screening results. This finding is in line with our purpose of identifying the healthcare workers at risk for PTSD-related symptoms during emergencies in order to ensure timely psychological support.

2) The Medically Unexplained Symptoms Scale (MUS; Mayou, 1991), which is a 38-item qualitative tool used in the medical sector to assess inflammatory state and related stress in terms of vague and non-specific symptoms. The MUS identifies a variety of symptom types (e.g., headache, gastrointestinal problems, chronic perceived fatigue, difficulty concentrating, and sleep problems). Studies conducted in the last decade have led to the hypothesis that non-specific physical symptoms may be linked to chronic stressful situations, in which stress is both physical (posture, sedentary lifestyle, but also over-training), metabolic (incorrect diet, drugs for chronic use), and socio-emotional (e.g., family or work situations, anxiety). A study published in 2015 recorded the presence of medically unexplained symptoms in about 100,000 cases, thus showing the importance of an early recognition of non-specific stress symptoms (Tsigos et al., 2015). The 38 items of the MUS are rated on a dichotomic scale, with “yes” or “no” response options (Aguzzoli et al., 2020; Hatcher & Arroll, 2008; Lorusso et al., 2019; Olde Hartman et al., 2013; Tsigos et al., 2015). We selected this tool for its items content quality according to our purpose of identifying the detrimental effects caused by exposure to chronic stress – such as the COVID-19 emergency – on the psychophysical well-being of healthcare workers, with possible manifestation of stress-related symptoms.

3) The Moral Distress Questionnaire for clinical nurses (MDQ; Eizenberg et al., 2009), which is a 15-item instrument for assessing moral distress among nurses employed in different work settings. Participants are required to indicate to what extent the situation associated with the care of patients and their families makes them experience stress of conscience. Each sentence is rated on a 6-point Likert scale (1 = “Not at all”, 6 = “Very large extent”), with the possibility to mark the answers considered irrelevant. Three factors representing moral distress were identified: (1) problems caused by work relationships among staff (Relationship), (2) problems due to lack of resources (Resources), and (3) problems caused by time pressure (Time). The internal consistency is good, with α = .85 for Relationship, .79 for Resources, and .80 for Time. The content of these three factors was in line with our purpose of identifying the healthcare workers, caring COVID-19 patients, at risk for moral distress due to resource reduction, work overload, crisis standards of care, intra-professional tensions, and miscommunications. These problems linked to the COVID-19 pandemic were demonstrated to have magnified all of those moral issues that were already a source of daily concern for healthcare workers (Williams et al., 2020; Silverman et al. 2021).

Development process of the questionnaire

The purpose of the present study was to develop a new scale, different from pre-existing stress-related tools since it should comply with the following goals: (a) optimize the timing of psychological clinical screening during a sanitary emergency, (b) analyze all the useful dimensions for a complete assessment of the healthcare workers’ distress condition, (c) assess the impact of the emergency on the person’s psychophysical well-being when the critical event is still in progress.

The 75 candidate HEDQ items belonging to the abovementioned questionnaires were qualitatively analyzed, with an expert-driven process, by two independent experts in the field of emergency psychology. During this process, a content validity criterion was used; specifically, the candidate items were combined to avoid redundancy and adjusted to both ensure their relevance to the domains of interest and overcome vagueness and/or difficulty in understanding.

After the reduction conducted by the experts, we obtained the following collection of items divided by domain: 1) 11 items for the PTSD risk factors domain (e.g., “Anything that reminds me of COVID-19 causes me waves of emotions about it that I can't handle”); 2) 17 items for the Somatic-Affective Symptomatology domain (e.g., “I suffer from insomnia or nocturnal awakenings”); 3) 4 items for the Moral distress domain (e.g., “I feel like my workload is dangerous because it prevents me from providing adequate assistance”); 4) 5 items for the Protective factors domain (e.g., “I believe that my family and friends network is supportive for me”); and 5) 3 items for the Team support domain (e.g., “I feel supported by my colleagues”).

Initial instructions of the questionnaire were as follows: “A number of statements are listed below. In relation to the current emergency situation (COVID-19 pandemic), respond to each of the following items by indicating the rate that best reflects your current life condition on a scale from 0 (never) to 3 (most always). Keep in mind that there are no right or wrong answers. Do not take too long to answer to each statement; often the first answer is the most accurate. Thanks for your precious collaboration.”

Items were rated on a 4-point scale ranging from 0 (never) to 3 (most always).

Validation scale

The Depression Anxiety Stress Scale (DASS-21; Henry & Crawford, 2005; Italian version by Bottesi et al., 2015), which a tool not specific to COVID-19, was used to assess the convergent validity of the HEDQ.

The DASS-21 is composed of 21 items, divided into three scales: Depression, Anxiety and Stress. The Depression scale includes measures of dysphoria, despair, devaluation of life, lack of interest/involvement, anhedonia, and inertia. The Anxiety scale relates to arousal of the autonomic nervous system, effects on skeletal muscles, situational anxiety, and subjective experience of anxious effects. Lastly, the Stress scale is related to the presence of chronic non-specific arousal levels, relaxation difficulties, nervous excitement, irritability, agitation, hyperactivity, impatience. Items are rated on a 4-point scale ranging from 0 (never) to 3 (most always). In the study by Henry and Crawford (2005), both the scales and the total score demonstrated good internal consistency, with Cronbach’s α values ranging from .82 to .93. The Italian version of the questionnaire (Bottesi et al., 2015) showed good to excellent internal consistency values, specifically: α = .74 for Anxiety; α = .82 for Depression; α = .85 for Stress; and α = .90 for the total score. Moreover, two-week test-retest reliability values were all high, with Pearson’s r ranging from .64 to .75.

2.3 Scale construction and data analysis

All analyses were conducted using the statistical software JASP version 0.13.1 (JASP Team, 2020).

First, we calculated descriptive statistics and frequencies tables to highlight the characteristics of the sample.

Then, in order to conduct a cross-validation study, we randomly divided the sample into two subgroups with 161 and 136 participants, by keeping constant the proportion of males and females for both the subsamples of healthcare workers and general population. To be specific, consistently with the proportions observed in the entire sample (see Section 2.1), the first subsample (N = 161) was composed of 72.1% women and 27.9% men, and of 84.4% healthcare workers and 15.5% people from the general population. Along the same line, the second subsample (N = 136) consisted of 72.1% women and 27.9% men, and of 83.8% healthcare workers and 16.2% citizens from the general population.

An exploratory factor analysis (EFA) was carried out on the first subsample (N = 161). The number of extracted factors was determined using parallel analysis, with minimum residual as estimation method and promax rotation. We evaluated the suitability of our data matrix for the factor analysis using Bartlett’s sphericity test (Bartlett, 1951) and the KMO measure of sampling adequacy (Kaiser & Rice, 1974). Only factor loadings ≥ .30 were considered. The fit indices used were the ratio of chi-square to degrees of freedom (χ2/df), the root-mean-square error of approximation (RMSEA; Steiger & Lind, 1980) and the Tucker-Lewis Index (TLI; Tucker & Lewis, 1973).

A hierarchical confirmatory factor analysis (CFA) was conducted on the second subgroup (N = 136) to establish the robustness (stability) of the multi-factor structure derived from the previous EFA. The ratio of chisquare to degrees of freedom (χ2/df), the standardized-root-mean square residual (SRMR, Bentler, 1995), the RMSEA, the Comparative Fit Index (CFI; Bentler, 1990), the TLI, the Akaike’s Information Criterion (AIC; Akaike, 1973), and the Bayesian Information Criterion (BIC; Schwarz, 1978) were considered. The SRMR was used because it is among the most sensitive to misspecified factor correlations, and the RMSEA is sensitive to misspecified factor loadings (Hu & Bentler, 1999). To interpret whether a given factor model provided a good fit to the data, we used Hu & Bentler's (1999) and Byrne’s (1989) empirically derived cut-off values. An excellent fit is indicated by χ2/df < 3, SRMR ≤ .08, RMSEA ≤ .06, and CFI and TLI ≥ .95. A good fit is indicated by CFI and TLI ≥ .90.

Reliability, as internal consistency of the single factors and of the full scale, was estimated through McDonald’s omega coefficient (ω; McDonald, 1999), item-rest correlation and interitem correlation.

Subsequently, the whole sample was used to test convergent and known-group validity. Convergent validity was assessed through correlational analysis using Pearson’s r coefficient, which can be directly interpreted as a measure of effect size, according to Cohen's (1988) criteria: r = .10 (small), .30 (moderate), .50 (large). The known-group validity was evaluated through independent sample t-tests (groups considered: general population and healthcare workers) and analyses of variance (ANOVA) (groups considered: general population, “COVID unit” and “not COVID unit” healthcare workers sub-samples). For each ANOVA, post-hoc tests were conducted using Bonferroni correction.

Gender differences were tested by means of the independent sample t-test, while the relationships between the participants’ ages, years of working experience – in general and in a hospital ward – and the HEDQ’s scales were tested with the correlational test using Pearson’s r coefficient. Then descriptive indices of the resulting version of the HEDQ were computed.

The test-retest reliability of the HEDQ was evaluated using a test-retest design with a two-month interval between measurements. In the second administration (from August to September 2020), data were collected from 101 of the 250 healthcare workers at AULSS 3 Serenissima who had participated in the initial testing phase (a loss of 59.6%) using an internet-based self-report survey. The sample included 65.3% female and 32.7% male participants and 2% who did not specify male or female. Respondents were aged 25 to 67 years (M = 46.4 years, SD = 10.0). Participants dropout could have been caused by healthcare workers' attrition due to overwork cause by the reorganization of regular health activities while the health emergency was ongoing.

Finally, to check whether the psychological distress experienced by healthcare workers changed from the first to the second administration, paired sample t-tests were applied, considering as variables the HEDQ scale scores. Since the retest was carried out during the least critical phase of the pandemic in Italy and given that the HEDQ is supposed to be a measure of acute stress, we expected a significant decrease in scores.

Results

3.1 Sample characterization

The healthcare workers sample was divided between those who worked in COVID-19 hospital units (N = 168; 120 female and 48 male) and those who did not (N = 82; 66 female and 16 male) to identify whether healthcare workers who were directly involved on the front lines of diagnosis, treatment, and care of patients with COVID-19 were at a major risk of developing psychological distress and PTSD-related symptoms.

3.2 Exploratory factor analysis

Bartlett’s sphericity test and the KMO measure of sampling adequacy confirmed the possibility of conducting an EFA on our data matrix (test of sphericity: χ2= 3,200, df = 780, p <.001; KMO = .80).

The output of the EFA indicated a 5-factor solution, with the following fit indices: χ2= 1,069, df = 590, p < .001, χ2/df = 1.81; RMSEA = .080; TLI = .73. This solution was confirmed by visual inspection of the scree plot.

Due to multiple and/or lower than .30 factor loadings, items 4, 6, 8, 11, 12, 13 were excluded from the model. The resulting factors were:

- Factor 1: items 1, 2, 3, 5, 7, 9, 10, 14, 27, 28, 34

- Factor 2: items 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25

- Factor 3: items 29, 30, 31, 32

- Factor 4: items 26, 33, 35, 36, 39

- Factor 5: items 38, 40.

These factors corresponded to (1) Risk Factors-PTSD, (2) Somatic-Affective Symptomatology (3) Moral Distress, (4) Protective Factors, and (5) Team Support. The standardized factor loadings of items on their respective factors were all significant, ranging between .32 and .89.

3.3 Confirmatory factor analysis

A hierarchical CFA, derived from the previous EFA, were fitted to the data. The CFA included 5 first-order factors and one second-order factor.

Due to factor loadings lower than .30, items 9, 10, 26, 39 were removed from the model and another CFA was conducted. This further analysis indicated that items 25, 38, 40 loaded on multiple factors, so these items – and the factor Team Support as a result – were also excluded. Finally, a third CFA showed that the factor Protective Factors correlated the least with the others, so the corresponding items were removed. Moreover, items 24 and 34 had multiple correlated residual covariances; therefore, they were excluded from the model.

The fit indices of the models with five and three factors are shown in .

The final model – graphically presented in – was composed of 21 items, divided as follow:

Hierarchical three-factor model of the HEDQ

Risk Factors-PTSD: items 1, 2, 3, 5, 7, 14, 27, 28

Somatic-Affective Symptomatology: items 15, 16, 17, 18, 19, 20, 21, 22, 23

Moral Distress: items 29, 30, 31, 32

Standardized factor loadings for Risk Factors-PTSD varied from .39 to .85, those of Somatic-Affective Symptomatology from .52 to .76, and those of Moral Distress from .66 to .93. All the first-order factors of the HEDQ were explained by the second-order factor, called Psychological Distress. The standardized estimates of the link between the second-order factor and the first-order ones were: .85 for Risk Factors-PTSD, .92 for Somatic-Affective Symptomatology, and .65 for Moral Distress.

This suggests that symptoms assessed in the HEDQ represent a coherent COVID-19 stress syndrome in people with high scores.

The final Italian version of the HEDQ is shown in Appendix A, while the English version in Appendix B.

3.4 Internal consistency and test-retest reliability of the scales

McDonald’s omega (ω) coefficients were .82 for Risk Factors-PTSD, .86 for Somatic-Affective Symptomatology, .89 for Moral Distress and .91 for the full scale. All coefficients were > .80, indicating good-to-excellent reliability in internal consistency (Tavakol & Dennick, 2011).

Item-rest correlations varied from .44 to .67 for Risk Factors-PTSD, from .52 to .66 for Somatic-Affective Symptomatology, from .70 to .80 for Moral Distress and from .38 to .70 for the full scale. Interitem correlations were between .23 and .67 for Risk Factor-PTSD, between .27 and .61 for Somatic-Affective Symptomatology, and between .61 and .83 for Moral Distress.

Average interitem correlations were all ≥ .40. As suggested by Clark & Watson (1995), these values can be considered adequate for narrow constructs.

Lastly, two-month test-retest reliability values, computed based on responses from 101 healthcare workers, were high for both the HEDQ scales and the total score (Risk Factor-PTSD: r = .87; Somatic-Affective Symptomatology: r = .90; Moral Distress: r = .88; Total score: r = .93).

3.5 Descriptive indices

The descriptive indices of the final version of the HEDQ are presented in and . These indices were calculated for both the general population (N = 47) and the healthcare workers (N = 250), dividing the latter between those who worked in COVID-19 hospital units (N = 168) and those who did not (N = 82). Higher scores indicate greater levels of COVID-19-related distress.

3.6 Convergent validity

shows the correlations between the factors of the HEDQ and the DASS-21 scales. All the correlations were significant (p < .001) and almost all were medium-to-large in magnitude. These findings support the convergent validity of the HEDQ.

3.7 Known-group validity

Independent sample t-test showed significant differences in all the HEDQ’s scales between the general population and the healthcare workers (Risk Factors-PTSD: t= 2.42, p = .016; Somatic-Affective 295 Symptomatology: t= 2.80, p = .005; Moral Distress: 295 t= 15.2, p < .001; Total score: t= 4.93, p < .001), 220 295 with higher average scores obtained by the latter (see and ).

Moreover, ANOVA were conducted to test whether there were diferences in the level of distress between the general population and the healthcare workers who worked in COVID-19 hospital units and those who did not. These analyses demonstrated an overall efect of the independent variable on all HEDQ’s scales (Risk Factors-PTSD: F (2, 294) = 11.2, p < .001; Somatic-Afective Symptomatology: F (2, 294) = 12.8, p < .001; Moral Distress: F (2, 294) = 45.3, p < .001; Total score: F (2, 294) = 26.5, p < .001). The results of the post-hoc tests with Bonferroni correction are shown in . A significant difference was observed in all scales between the group of healthcare workers who worked directly with COVID-19 patients and the other two groups. In contrast, no diference was observed between the general population and the healthcare workers who did not work with COVID-19 patients, except for the Moral Distress scale ().

Plots of differences in the scores on the HEDQ’s scales between the general population (GP), the healthcare works who worked in COVID-19 hospital units (YES) and those who did not (NO)

3.8 Gender differences

Independent sample t-test showed significant diferences between men and women in the following factors of the HEDQ: Risk Factors-PTSD (PTSD) (t295 = 2.844, p = .005), Somatic-Afective Symptomatology (SA) (t= 2.810, p = .005) and in the total score (TOT) 295 (t= 2.486, p = .013). Specifically, female participants 295 obtained higher average scores (M= 6.771, SE = PTSD .323; M= 7.734, SE = .386; M= 18.664, SE = SA TOT .782) compared to males (M= 5.048, SE = .501; PTSD MSA = 5.699, SE = .600; MTOT = 14.928, SE = 1.324).

3.9 Relationship between age, years of experience and HEDQ

The correlational tests did not show any statistically significant correlation between the participants’ ages, their work experience – both in general and in hospital wards – and the HEDQ’s scales.

3.10 workers’ Change psychological over time distress of the healthcare

To verify the hypothesis according to which the healthcare workers’ psychological distress significantly decreased from the first (T0) to the second (T1) administration, paired sample t-tests were conducted, including as variables the HEDQ scale scores.

Results showed a significant change over time of the scores on all the HEDQ’s scales (Risk Factor-PTSD: t100 = 11.95, p < .001; Somatic-Affective Symptomatology: t= 12.94, p < .001; Moral Distress: t= 11.76, p < 100 100 .001; Total score: t= 16.80, p < .001); specifically, 100 lower average scores were obtained at the second administration (Risk Factor-PTSD: M= 6.46, T0 SE = .47; MT1 = 3.19, SE = .26; Somatic-Affective Symptomatology: M= 8.07, SE = .54; M= 4.31, SE T0 T1 = .32; Moral Distress: M= 4.99, SE = .37; M= 2.72, T0 T1 SE = .24; Total score: M= 19.54, SE = 1.12; M= T0 T1 10.22, SE = .67).

Discussion

The HEDQ is a self-report screening questionnaire, developed as an emergency-specific measure that takes into account evidence-based pandemic-related psychological distress responses. The main aim of the questionnaire is to identify the healthcare workers at risk for psychological and moral distress, and/or PTSD-related symptoms while working in an emergency such as the worldwide COVID-19 pandemic. The HEDQ was administered to healthcare workers at AULSS 3 Serenissima in Venice, Italy, during the health emergency period.

The final version of the questionnaire included 21 items, divided into three subscales assessing the foremost COVID-19 distress symptoms: (1) Risk Factors-PTSD; (2) Somatic-Affective Symptomatology; (3) Moral Distress. The hierarchical structure showed adequate factorial validity: this result, on one hand, allows consideration of the total score of the HEDQ and, on the other hand, highlights the connection among the investigated symptoms, delineating a coherent COVID-19 stress syndrome.

Pertaining to the psychometric properties of the questionnaire, both the total score and the scales showed very good internal consistency. In addition, the two-month test-retest reliability values were large for all the HEDQ’s scales, showing an excellent temporal stability of the measure.

The HEDQ also showed good convergent validity, with the correlations between the HEDQ scales and the DASS-21 varying from medium to large. To be more specific, the HEDQ total score was highly correlated with the DASS-21 total score, showing that they both detect a stress-related symptomatology, including cognitive, behavioral, somatic, and afective symptoms. On the other hand, the low correlations of the Risk Factors-PTSD and Moral Distress scales with the DASS-21 scales is evidence they measure diferent domains.

With regard to the change over time of the healthcare workers’ stress-related symptomatology, this was expected to decrease from the first to the second administration, given that the retest was conducted during the least critical phase of the pandemic in Italy. In fact, during August and September 2020, the number of new COVID-19 cases and infection fatality rates in Italian hospitals dramatically fell, compared to the period between April and June (Istituto Nazionale di Statistica [ISTAT], 2020). Results confirmed our hypothesis, showing a significant decrease in the HEDQ scale scores and, therefore, in the healthcare workers’ psychological distress. This finding is particularly relevant because it confirms that the HEDQ detects diferent levels of stress experienced by healthcare workers during diferent phases of an emergency situation, thus supporting the precision of the tool in measuring acute stress.

The COVID-19 pandemic has caused a worldwide health emergency with extraordinary characteristics, not only in terms of a lack of defined and clear protocols and guidelines, but also of the impact severity on the psychophysical well-being of the frontline healthcare workers involved, as highlighted by three key findings of the present study.

The first is that age and years of experience as a healthcare professional were not related to the severity of the symptoms reported. This result difered from previous studies, which found that less working experience was associated with higher psychological burden during the COVID-19 outbreak (e.g., Huang & Zhao, 2020; Peiró et al., 2020). In fact, clinical expertise usually represents a critical resource for healthcare workers since it is linked to knowledge, skills and improvement of coping strategies that may help in managing stress under uncertain circumstances. Nevertheless, in our sample, age and years of experience did not emerge as protective factors against the detrimental effects of the pandemic on healthcare workers’ well-being. This might be due to both the specific characteristics of our sample and the extraordinary psychological efects of the COVID-19 outbreak, which are unpredictable, particularly damaging and different across populations.

The second key finding is the presence of a significant diference between men and women in Risk Factors-PTSD (PTSD) and Somatic-Afective Symptomatology (SA), with women recording higher scores. This result is consistent with previous studies showing that being a woman is associated with experiencing severe depression, anxiety and distress. Consistent with previous studies, 71.1% of all participants were women, of which 53.3% were frontline nurses (79.1% total frontline nurses). Frontline nurses treating patients with COVID-19 are physically and psychologically challenged in their commitment to provide high-quality nursing care because of their close, frequent contact with patients and the longer-than-usual shifts they often work (e.g., Chen et al., 2005; 2020).

Finally, the third important finding is the significant diference that emerged between the healthcare workers who did not work in COVID-19 hospital units and the general population in only the Moral Distress scale. This result supports the specificity of the Moral Distress scale, which represents the dimension of worry and stress experienced in hospital environments, characterized by healthcare workers’ perception that they are not providing adequate assistance due to ethical recommendations, lack of resources, time pressure and overwork (Negrisolo & Brugnaro, 2012).

A specific strength of the HEDQ is that it discriminated between those who work in COVID-19 hospital units and those who do not – including the general population -, with higher scores obtained by the first group in all the scales. This result indicates good known-group validity for the questionnaire and leads to important empirical findings on the nature of stress reactions based on the level of exposure to health emergencies of those who worked in high-risk hospital units (Albott et al., 2020; Chen et al., 2005; Tam et al., 2004).

Considering the overall results of the validation process, the HEDQ represents a potential psychometrically sound measure to assess psychological distress in healthcare workers during a stressful period such as the COVID-19 pandemic. Nevertheless, some limitations need to be addressed. One limitation of the present study is the lack of structured diagnostic assessments (i.e., DSM-5 or ICD-11 diagnoses); nonetheless it evaluates diagnostic criteria of stressful and traumatic events related disorders such as intrusion symptoms, avoidance, hypervigilance, hyperarousal and somatic-affective symptomatology (APA, 2013; Palm et al., 2004). Moreover, our sample included only participants from Venice; this limits the generalizability of the observed results to the whole Italian population. By considering heterogeneous scenarios of national and regional COVID-19 pandemic trajectories, further studies involving participants from diferent areas might provide a stronger test of the validity and the reliability of the HEDQ.

Notwithstanding the abovementioned limitations, our results provide support for the use of the HEDQ as a brief, multidimensional measure of emergency-related stress reactions in healthcare workers. Consequently, the HEDQ can be considered a useful instrument supporting clinical activity to identify those who may be more easily affected by stress reactions in the event of atypically high levels of risk exposure during crises. We expect that the HEDQ, detecting stress-related symptomatology in detail, could lead to important new therapeutic developments for both the assessment and psychological intervention among those who must cope with a health emergencies.

Table 1.

Fit indices of the models with five and three factors of the HEDQ

Model	χ2	df	p	χ2/df	TLI	CFI	SRMR	RMSEA	AIC	BIC
Five factors	895	485	<.001	1.84	.758	.777	.092	.082	9,857	10,072
Three factors	239	176	.001	1.36	.938	.948	.060	.053	6,154	6,369

Note: CFI: Comparative Fit Index; TLI: Tucker Lewis Index; SRMR: Standardized Root Mean Square Residual; RMSEA: Root Mean Square Error of Approximation; AIC: Akaike’s Information Criterion; BIC: Bayesian Information Criterion

Table 2.

Descriptive indices of the HEDQ for the general population

	PTSD	SA	MD	TOT
M	4.77	5.06	0.34	10.2
SD	4.70	5.71	1.24	10.2
5th percentile	0	0	0	0
10th percentile	0	0	0	1
50th percentile	4	3	0	7
80th percentile	8	10	0	16
85th percentile	9	11	0	18
90th percentile	10	12	0	21
95th percentile	14	16	2	25

Note: PTSD = Risk Factors-PTSD; SA = Somatic-Affective Symptomatology; MD = Moral Distress; TOT = HEDQ’s total score

Table 3.

Descriptive indices of the HEDQ for healthcare workers

	PTSD		SA		MD		TOT
	Yes	No	Yes	No	Yes	No	Yes	No
M	7.39	4.90	8.56	5.51	5.54	3.55	21.5	14.0
SD	4.91	3.74	5.54	5.12	3.56	3.80	11.4	10.0
5th percentile	0	0	1	0	0	0	5	2
10th percentile	1	1	2	1	1	0	8	3
50th percentile	7	4	7	4	5	2	21	12
80th percentile	11	7	13	9	9	7	31	22
85th percentile	12	8	15	9	10	8	34	24
90th percentile	14	11	17	12	11	10	36	28
95th percentile	17	13	19	17	12	12	42	36

Note: Yes = healthcare workers who worked in COVID-19 hospital units; No = healthcare workers who did not work in COVID– 19 hospital units; PTSD = Risk Factors-PTSD; SA = Somatic-Affective Symptomatology; MD = Moral Distress; TOT = HEDQ’s total score

Table 4.

Pearson's r correlations between the factors of the HEDQ and the other tools

	PTSD	SA	MD	TOT
DASS-21 S	.56	.70	.46	.72
DASS-21 A	.56	.73	.37	.70
DASS-21 D	.52	.63	.36	.63
DASS-21 TOT	.61	.77	.45	.77

Note: PTSD = Risk Factors-PTSD; SA = Somatic-Affective Symptomatology; MD = Moral Distress; TOT = HEDQ total score; DASS-21 S = DASS-21 Stress scale; DASS-21 A = DASS-21 Anxiety scale; DASS-21 D = DASS-21 Depression scale; DASS-21 TOT = DASS-21 total score.

Table 5.

Results of the post-hoc tests considering the differences in the level of distress between the general population, the healthcare workers who worked in COVID-19 hospital units and those who did not

HEDQ Factor	Levels	Mean difference	t	df	pBonferroni
Risk Factors PTSD	GP vs No	-.14	-.16	127	1.000
	GP vs Yes	-2.63	-3.47	213	.002
	No vs Yes	-2.49	-4.03	166	<.001
Somatic-Affective Symptomatology	GP vs No	-.45	-.45	127	1.000
	GP vs Yes	-3.50	-3.89	213	<.001
	No vs Yes	-3.05	-4.14	166	<.001
Moral Distress	GP vs No	-3.21	-5.19	127	<.001
	GP vs Yes	-5.20	-9.32	213	<.001
	No vs Yes	-1.99	-4.37	166	<.001
Total score	GP vs No	-3.79	-1.92	127	.17
	GP vs Yes	-11.32	-6.34	213	<.001
	No vs Yes	-7.53	-5.16	166	<.001

Note: GP = general population; Yes = healthcare workers who worked in COVID-19 hospital units; No = healthcare workers who did not work in COVID-19 hospital units

Mai			Quasi sempre
0	1	2	3

1	Ogni cosa che mi ricorda il COVID-19 mi fa venire ondate di emozioni relative a esso che non riesco a gestire.	0	1	2	3
2	Ci penso senza averne l'intenzione.	0	1	2	3
3	Le immagini legate al COVID-19 mi entrano nella mente all'improvviso.	0	1	2	3
4	Sto lontano da cose che possono ricordarmelo.	0	1	2	3
5	Evito di parlarne.	0	1	2	3
6	Mi ritrovo a essere guardingo rispetto all'ambiente o alle persone.	0	1	2	3
7	Ho paura di essere contagiato o di essere veicolo di contagio.	0	1	2	3
8	Provo un senso di smarrimento.	0	1	2	3
9	Mi sento più nervoso del solito.	0	1	2	3
10	Mi sento più affaticato del solito.	0	1	2	3
11	Soffro di insonnia o risvegli notturni.	0	1	2	3
12	Vivo momenti di ansia o preoccupazione.	0	1	2	3
13	Mi sento apatico.	0	1	2	3
14	Avverto la comparsa di aritmia/tachicardia.	0	1	2	3
15	Noto cambiamenti nell'appetito.	0	1	2	3
16	Avverto di disturbi gastro-intestinali.	0	1	2	3
17	Avverto una alterazione della sudorazione corporea.	0	1	2	3
18	A causa del sovraccarico di lavoro sento di non aver tempo sufficiente per fornire al paziente cure adeguate.	0	1	2	3
19	Sento di lavorare con un carico di lavoro che considero pericoloso per garantire una assistenza adeguata.	0	1	2	3
20	Mi sembra di discriminare il trattamento dei pazienti sulla base delle direttive o delle raccomandazioni etiche.	0	1	2	3
21	Ritengo che i pazienti ricevano trattamenti differenti a causa della mancanza di risorse (personale, posti letto, forniture, …).	0	1	2	3

Never			Never Almost always
0	1	2	3

1	Anything that reminds me of COVID-19 causes me waves of emotions about it that I cannot handle.	0	1	2	3
2	I think about it unintentionally.	0	1	2	3
3	Images linked to COVID-19 suddenly enter my mind.	0	1	2	3
4	I stay away from things that could remind me of it.	0	1	2	3
5	I do not want to talk about it.	0	1	2	3
6	I feel alert to environment or people.	0	1	2	3
7	I am afraid to be infected or to be a vehicle of contagion.	0	1	2	3
8	I feel a sense of loss.	0	1	2	3
9	I feel more irritable than usual.	0	1	2	3
10	I feel more fatigued than usual.	0	1	2	3
11	I suffer from insomnia or nocturnal awakenings.	0	1	2	3
12	Sometimes I feel anxiety or worry.	0	1	2	3
13	I feel apathetic.	0	1	2	3
14	I feel arrhythmia / tachycardia.	0	1	2	3
15	I notice changes in my appetite.	0	1	2	3
16	I suffer from gastro-intestinal disorders.	0	1	2	3
17	I notice changes in my body sweating.	0	1	2	3
18	Due to my workload, I feel like I do not have enough time to provide the patient with proper care.	0	1	2	3
19	I feel like my workload is dangerous because it prevents me from providing adequate assistance.	0	1	2	3
20	I feel like I discriminate against the treatment of patients on the basis of directives or ethical recommendations.	0	1	2	3
21	I believe that patients receive different treatments due to lack of resources (staff, beds, supplies, etc.).	0	1	2	3