Ethical and psychosocial considerations for hospital personnel in the Covid-19 crisis: Moral injury and resilience.

This study aims at investigating the nature of resilience and stress experience of health care workers during the COVID-19 pandemic. Thirteen healthcare workers from Italian and Austrian hospitals specifically dealing with COVID-19 patients during the first phase of the pandemic were interviewed. Data was analysed using grounded theory methodology. Psychosocial effects on stress experience, stressors and resilience factors were identified. We generated three hypotheses. Hypothesis one is that moral distress and moral injury are main stressors experienced by healthcare workers. Hypothesis two states that organisational resilience plays an important part in how healthcare workers experience the crisis. Organisational justice and decentralized decision making are essential elements of staff wellbeing. Hypothesis three refers to effective psychosocial support: Basic on scene psychosocial support based on the Hobfoll principles given by trusted and well-known mental health professionals and peers in an integrated approach works best during the pandemic.

Introduction

Psychosocial aspects in the COVID-19 response

The COVID-19 pandemic has spread rapidly and has since become a severe mental health crisis. Measures aiming at the containment of the virus, protection of the risk population as well as mitigation changed the life and routines of many around the globe. Measures required during the response, such as quarantine, can have a severe impact on the mental health state of the population [1].

Higher stress in healthcare workers during pandemics

By caring for patients infected by COVID-19, health care professionals all around the world are involved in the acute response to the crisis. Based on the experiences made during the COVID-19 pandemic, the high exposure to stressors as well as the need for Psychosocial Support for health care workers has been well communicated [2-4]. According to findings from the SARS outbreaks in 2003, H1N1 outbreak in 2009, as well as after several outbreaks of Ebola, healthcare personnel working on the front-line might be at risk of experiencing higher levels of stress immediately after the response. Working on the front-line in pandemics can also negatively impact the development of mental health problems in the long-term which can include higher levels of PTSD symptoms, burnout and other mental health problems [5-14].

Similar results can now be found in COVID-19 [15-19]. Tan et al. [19] analysed data from 1257 Healthcare workers in 34 different hospitals from January 29th to February 3rd, 2020. Participants were aged between 26 und 40, 764 (60,8%) were nurses, 493 (39,2%) doctors. Many showed heightened signs of depression (50,4%), anxiety (44,6%), insomnia (34%) and feelings of distress (71,5%). Pappa et al. [18] did a meta analysis and found increased values for depression (23,2%) and anxiety (22,8%). Especially sex and type of job (women and nurses versus men and doctors) showed more affective symptoms. Insomnia was rated 38,9%.

Stressors

Several factors that caused stress among health care professionals during earlier pandemics have been identified. Many of those stressors could also be verified in first studies on the COVID-19 outbreak. Dai et al. [20] did a study on concerns of 4357 healthcare workers. According to the data, their main concerns were the infection of colleagues (72,5%), infection of family members (63,9%), protection measures (52,3%) and ethical concerns (48,5%). 39,1% of the participants showed clinically relevant psychological problems, especially those who were based in Wuhan or those who were in quarantine and/or had infected a colleague or family member.

High exposure

Generally, concerns about infecting family members and friends are central, as responders during pandemics are being exposed to infected patients at work [10, 13, 21–23]. In a study conducted by Goulia et al. [13] on the H1N1 influenza outbreak, the most frequent concern of health care workers was the possibility of infecting family and friends and the health consequences of the disease. The same concern among health care workers is prevailing during the current COVID-19 pandemic and might be especially true for staff living with people considered to belong to a risk group. Cai and colleagues [21] show that staff between 31–40 years of age had the greatest concern regarding viral transmission to their families. A possible explanation is that more people in this age group are living together with older people and children. Also Dong et al. [17] found that healthcare workers who had fears about their own physical health and who had friends or relatives tested positive for COVID-19, had a higher probability of anxiety and depression. Concerns of spreading the virus are also fuelled where there is limited access to testing and/or personal protective equipment (PPE), which makes the own risk assessment of health care personnel somewhat more difficult [24]. Not being rapidly tested when developing symptoms may not only lead to fear of spreading the disease among friends and relatives but also among patients and colleagues at work [24].

Decreased feeling of safety and loss of trust

Furthermore, a decreased feeling of safety due to limited access to appropriate personal protective equipment seems prevailing [3, 22]. A feeling of unsafety is also grounded in often not having access to up-to-date information and if clear guidance is not communicated accordingly. During the H1N1 outbreak, personnel with better access to information scored lower on stress symptoms than those with limited access [14].

Social stressors

Social factors inherent in the everyday private life of health care workers cause difficulties during pandemics. Not being able to access childcare or to provide support for other personal and family needs might be a barrier to working [25]. Furthermore, in many cases health care workers feel stigmatized, as they are avoided or treated differently due to high exposure to infected patients [12, 13, 26].

Quarantine experience

Studies suggest that being quarantined may have a more severe impact on the group of health care workers than other population groups. However, the authors argue that this effect may not be unique to their health care workers status but may be influenced by their job-related experiences, which results in recognition of higher personal risk or more knowledge regarding the severe end of SARS on the clinical spectrum. In their study, health care workers also had a better understanding for the rationale for quarantine and showed more compliance with quarantine behaviors [27]. Several studies on health care workers affected by the SARS outbreaks in Canada, China and Taiwan found quarantining experience to be predictive of subsequent general psychological distress [9, 11, 28, 29]. However, in a study by Hawryluck et al. [30] on the psychological effects of quarantine in Toronto during the SARS outbreak, health care worker status was not correlated with negative effects such as PTSD or depressive symptoms.

Moral injury and ethical dilemma

Several researchers state that moral injury is one of the mental health challenges faced by health care workers during the COVID-19 outbreak [31-33]. According to Shay [34] the term moral injury “has been used in two related, but distinct, senses; differing mainly in the ‘who’ of moral agency”. According to Shays own definition, moral injury is present when there has been (a) a betrayal of ‘what’s right’; (b) by a person in legitimate authority (e.g. a leader) and, (c) in a high stakes situation. The definition of Litz et al. [35] also entails participation by one’s self in acts that transgress such moral beliefs.

Both forms affect trust and lead to psychological distress. Moral injury has mainly been described in victims and perpetrators of violence such as child soldiers. However, not only intentional interpersonal disasters may involve moral injury, also other disasters like pandemics include these types of trauma. ln the COVID-19 pandemic, health personnel faced a variety of those injuries in both of the above mentioned meanings.

In a global pandemic, a different set of rules has to be applied to healthcare delivery as complex dilemmas in care may evolve. Understanding general principles of collective ethics may help, nevertheless clinicians still have to take decisions for their specific patients, which can lead to significant distress [31]. The ethical bitterness of triage decisions is well-known and medical organisations and professionals have found different ways to deal with it [36, 37]. Some of the most common ethical challenges in the response to COVID-19 can be conceptualized in triage, shortage of personal protective equipment and non-pharmaceutical interventions. The following strategies can be identified to deal with the dilemmata [38-40].

Triage

While cost benefit analysis in the utilitarian model of resource distribution is based on criteria such as numbers of lives saved, number of years of life saved or quality of life that might not be compatible with human rights regulations, other criteria have to be found to allocate resources such as the criteria of medical need and efficacy of treatment.

PPE shortage

Regarding the protection of EMS clinicians [41] and of health care workers, a balance between duty to care and protection of health care workers has to be found. Especially in a situation in which the availability of health care workers is crucial, they should have priority in the distribution of resources.

Non-pharmaceutical interventions

All non-pharmaceutical interventions that are critical (with regards to e.g. data protection), need to be appropriate with regard to necessity, proportionality and minimization (of e.g. data use).

Effective support

Several factors that are specific to the pandemic context have been found to help in reducing stress among health care workers. Strengthening individual as well as organizational resilience can help in mitigating the negative long-term influence of stressful experiences among health care professionals [26].

Altruism

While being at risk, studies show that moral and social responsibilities as well as altruistic attitudes drive health care workers to continue working in an environment that might be extremely stressful [9, 21, 23, 25]. Data showed, that altruistic risk-acceptance during the SARS outbreak could decrease the odds of higher levels of depression-symptoms three years after the outbreak [9]. Wu et al. also identified altruism as a protective factor against negative impacts. Underlining the valuable altruistic attitude can reduce psychological distress especially in those who are quarantined [10]. Altruism being a protective factor, elements that reduce altruism and endanger one´s self view as a helper may do a lot of damage for health care personnel.

Guidance and shared decision making in ethical dilemma situations

Receiving good guidance and not having to take decisions on their own as well as good leadership may minimize the impact of moral injury. Shared decision making and tools may thus be very helpful to mitigate negative impact [31, 32].

Information

Information and efficient and fast communication are essential elements in order to decrease worry and promote a feeling of safety among staff [13]. This includes information about the virus, routes of infection, possibilities and practical guidance for treatment or protection measures, but also factual information on the situation within the given hospital, such as capacities or number of infected people among staff [18].

Training

Another stress-reducing factor is training and support in using PPE. The provision of equipment, including masks and suits as well as infection control guidance by the hospital leadership have been reported to promote a feeling of safety in the MERS-Cov outbreak [23] as well as SARS [29] and other outbreaks. Marjanovic et al. [11] showed that trust in infection control initiatives as well as protective equipment predicted lower levels in correlates of burnout and stress among nurses engaged in the SARS outbreak in Canada. Raising awareness of the effects of disease prevention measures among staff with reduced numbers of reported cases can reduce staff stress [21].

Psychosocial interventions

Furthermore, psychosocial interventions may be more helpful than support that is too much focused on clinical psychological interventions [22]. Psychosocial services among health care workers provided either by peers or by well-known and trusted mental health professionals—such as psychologists, clergy or psychiatrists—have been shown to be especially helpful [10]. Signalling in-group messages that all staff are in this together and nobody has to carry decisions alone are considered as very stress relieving [9, 10, 21].

Opportunities for psychoeducation and psychosocial counselling are essential for the protection of personnel. However, during the pandemic the provision of psychosocial support has been reported to be most effective when including hospital management and leadership and focusing on pragmatic support based on the actual needs of the staff. This can be done by taking care of basic needs, such as break and resting areas, food, daily living supplies and ways to be in contact with their families [22].

Pre-job-trainings on identification and responding to psychological problems in oneself and others, as well as on coping with stress have been reported as helpful during the SARS outbreak as well as the COVID-19 situation in China [22, 42]. The acceptance and use of psychological support might be higher when provided low-threshold access. Counsellors visiting break and resting areas where stories and difficulties are shared, and providing support accordingly is one way of providing low-threshold access [22].

In many cases, anonymous helplines and counselling have been established in order to reach highly affected staff in the hospitals. However, considering the access to psychological services, it should be taken into account, that staff with subthreshold and mild levels of mental health disturbances might actually be more likely to take action and be motivated to acquire skills to help others, than those with more severe disturbances who have more motivation to learn self-help techniques [3].

Resilience

If we look at the concept of resilience in the context of natural hazards we can identify three core elements: resistance, recovery and adaptive capacity [41, 43]. Resistance is the ability of a system to withstand a threat, which is mostly a consequence of strength and preparedness. Recovery is the ability to bounce back and be able to come back to a state of normal functioning after a disaster has hit and adaptive capacity refers to the ability to learn and change due to the experience made [41].

After the SARS outbreak in Canada, Maunder et al. [42] did a study on resilience in healthcare workers. Although the authors recommend special trainings for healthcare workers, evidence points in the way that resilience building on the organisational level is of utmost relevance for pandemics. This goes way beyond the development of adequate training and psychosocial interventions, although these are also very important to have.

According to the authors, two constructs have been relevant for organizational resilience in a pandemic: Organizational justice and “magnet hospitals”. Organisational justice refers to the amount to which leadership and management take the opinions of healthcare staff into account and take their concerns seriously. The first element of organizational justice is relational justice, the ability of team leaders and managers to suppress their own prejudice and treat their staff in an honest and just manner. The second element is decisional justice, which mainly refers to fair decision making. Organisational aims, which are based on the needs of patients as well as the needs of healthcare workers, lead to more (crisis) resources.

Another relevant construct for organisational resilience was found in so called “magnet hospitals”. These were characterized by decentralized decision-making and nurses being amongst administrative staff and management, a flexible approach to shifts and continuous effort in training and self-administration of units.

These results confirm other findings from organizational psychology that have shown that a high amount of demands mixed with low control (low decision making capacity and low influence in staff) and an imbalance between effort and reward has negative effects on staff health and wellbeing [44, 45].

Methods

In this study, we are trying to identify the subjective experience, stressors and collective coping strategies used in hospital care to relieve stress from a psychosocial perspective and manage the challenging situation of the pandemic. The objective of the study was to find out what the main stressors as well as the main stress reducing factors were during the first phase of this pandemic for healthcare workers in the hospital sector. As many of the psychosocial challenges during the pandemic have to be considered from an ethical perspective, we were also interested in ethical questions in COVID-19. Amongst others, these included resource distribution when there is shortage in supplies as well as the duty to help while risking one’s own health and health of relatives considered vulnerable. We assumed to find indicators of moral injury in healthcare workers to be of importance during this first phase of the pandemic. However, we were not only interested in the negative effects of the pandemic on healthcare workers. We were especially interested in (organizational and team) resilience, namely indicators for resistance, recovery and adaptive capacity of healthcare workers themselves as well as the systems and organisations they work in.

The data presented in this article is part of a larger study done in the course of an EU Project (NO-FEAR) where we use a mixed methods approach alternating between quantitative (online) surveys and qualitative interviews and focus groups. The institutional ethics committee of the Institute for Psychology at the University of Innsbruck as well as the NO-FEAR Project External Ethics Advisory Board reviewed and approved this study.

As a first step, we chose to conduct expert interviews. The reason for this choice was the fact that we expected an explorative approach to be the best way to gain more detailed insights in both, the positive and negative subjective experiences of healthcare workers during this first and very stressful phase of the disaster. As we did our study very early in the crisis we knew that healthcare workers were still in the middle of a very stressful job and expected them not to be willing to do long interviews or fill out questionnaires. We thus focused on a small number of experts working in the field including mental health professionals as well as experts from the medical area. According to the principle of Grounded Theory [46], we aimed at maximum contrast and thus tried to get interviews from rather different contexts and backgrounds. Our main factors for the choice of experts was that they were very experienced, working in the field and in a leading position or a position that allowed to speak for their colleague healthcare workers from an expert position.

Our aim was to develop some basic hypotheses regarding resilience and stress experience of healthcare workers in the first phase of the COVID-19 pandemic. These hypotheses shall be a basis for the interpretation of data from questionnaires and interviews gained in later phases of the pandemic. We are well aware that these data are far from generalizable, but expect our hypotheses to be a good basis for further research as well as practical conclusions.

In total, 13 experts were interviewed by two teams. Participation of experts was requested via telephone calls. Eligible experts included in the study sample had to be health care workers involved in the current COVID-19 response either in hospital settings. Health care staff had to have more than 5 years of experience. Psychologists and intensive care physicians had to have more than 10 years of experience. This included doctors, nurses and psychologists in leading positions in Italian and Austrian hospitals. All professionals interviewed were informed about the scope of the interview and their rights. They subsequently signed an informed consent that was previously reviewed by the NO-FEAR Project External Ethics Advisory Board.

The Austrian team consisting of male and female scientifically experienced psychologists conducted two semi-structured interviews with mental health experts from hospitals in Italy and one Focus group discussion with four psychologists involved in the hospital response in Austria. The Italian team, consisting of female scientifically experienced researchers in the medical field conducted 7 semi structured interviews with healthcare professionals from the hospital, revising the questions based on the experience and the role of each interviewee. Questions focused on the main challenges that health care workers experienced in the acute phase of the COVID-19 response as well as the needs they have. Additionally, we focused on strategies that health care workers experienced as helpful or stress relieving to identify possibilities for psychosocial support measures.

One additional interview was conducted on challenges from the ethical and legal perspective in hospital care with Laura Palazzani [40].

Data was collected from the beginning of March, when the virus began to rapidly spread in Italy and Europe until the end of May.

Table 1 is giving an overview on basic demographic data of the study sample.

Table 1

Study sample including information on function, gender and country of origin of interviewees as well as the method used to gather information.

Occupation/function	Gender	Country	Method
ED intensive care physician	m	Italy	interview
Head nurse of ED	f	Italy	interview
Psychologist	f	Italy	interview
Psychologist	f	Italy	interview
Psychologist, Head of unit	f	Italy	interview
Director of intensive care unit	m	Italy	interview
Trainer for HCW protection and PPE, chief registered nurse	f	Italy	interview
Psychologist	f	Italy	interview
Psychologist	m	Italy	interview
Four Psychologists	m/f	Austria	Focus group

m = male; f = female; ED = Emergency Department.

The interviews were transcribed and eventually analysed using Grounded Theory methodology. Data was discussed with participants to allow for comments and correction. We coded data into several concepts and subcategories to identify psychosocial and ethical challenges during the COVID-19 response. Codes were subsequently related to each other using Axial Coding. We created a basic explanatory framework on psychosocial effects of COVID-19 consisting of subjective stress experience, causal factors, stressors and resilience factors during the response.

Results

Psychosocial effects of COVID-19

The following paradigmatic model shall give an overview of the main categories and their links. Causes, intervening stressful and stress reducing factors and coping strategies are gathered around the phenomenon of interest namely the subjective stress experience of healthcare personnel.

Key category 1: Fear, guilt feelings, frustration, loss of trust and exhaustion

Emotionally, fear of getting infected and subsequently infecting families and friends are common feelings. Experts point out the prevalence of feelings of guilt and shame as employees feel like being “plague spreaders”.

“…staff is scared because they live with kids or elderly/immunodepressed people. They are concerned to take the COVID home. Normally they avoid to bring home problems or issues related to their job but now they feel they can bring home COVID but also their feelings. ED staff also try to avoid to send mothers or fathers of kids to the infected patients”(Interview partner 1, Italy).

This resulted in many health care workers not returning home as they did not want to put their own families at risk. Some staff have been reported to be sleeping with masks at home. Many health care workers feel unsafe and experience a loss of control, as well as a loss of trust in the system, as they do not feel sufficiently protected.

Frustration and powerlessness are prevailing. Among nurses, rage and demands for appropriate compensation is central. Among doctors, more silent reactions seem common which has been described by one interview partner as a “paralysis of thought”.

“Nursing staff is reacting with rage to the stress the COVID 19 is generating, while doctors are reacting with silence, stunned silence (it has been defined by some as a “paralysis of thought”)” (Interview partner 3, Italy)

One other interview partner points out “that fear of infection among health care workers is higher after the shifts when resting at home” (Interview partner 1, Italy).

Socially, stress reactions included increased lack of trust in external or newly hired workers, as more experienced staff are afraid that unexperienced workers might make mistakes. But also a loss of trust in the hospital and in oneself as well as a need for a caring leadership was observed.

“Hospital staff is suffering from a sense of abandonment from the institution (which includes medical direction of the Hospital and in some cases also by the chief physician as well as the chief nurse). They are looking for someone (above mentioned) that nurture them and would like to have a deeper recognition of the work they are performing and more gratifications. One aspect that is frequently mentioned is the “sharing of moments”, be they for discussion of clinical cases to decision-making and especially directives on how to relate to the patient and the relatives. The relationship the personnel is looking for is more similar to a child-parent scheme than subordinate-superior scheme.”(Interview partner 3, Italy).

Physically, exhaustion and fatigue are prevailing. This is also true for staff in the laboratories due to high amounts of positive blood test results. Psychologists report about insomnia as a common issue of health care workers according to the experts that were interviewed.

Cognitive stress reactions include confusion and unrest as well as high levels of dissociation among health care workers involved in the COVID-19 response. One expert points out signs of dissociation among health care workers, stating “It’s like a film scene from a war, just without visible destruction” (Interview partner 5, Italy).

Also positive emotions were reported. Pride of having managed a difficult situation together as well as a feeling of high team cohesion and solidarity among the healthcare personnel and a deep commitment to their job as a healthcare worker have been mentioned by many experts. This also led to a feeling of “us” versus “them” meaning “us frontline workers” versus “them in the background”.

Key category 2: Causal factors: Rapidly evolving situation with high uncertainty

The COVID-19 pandemic is a rapidly evolving situation in which the protection of the population, but especially of people with high exposure to positively tested patients, such as health care workers in the hospitals, is central. The situation in the first wave of this pandemic was characterized by the uncertainty of information and lack of knowledge about the virus and its characteristics. Additionally, the rapidity of change in information as well as the rapidity with which patients tested positive came into hospitals and/or the rapidity of change in their medical conditions is pivotal. A further characteristic is the fact that one may infect others also when not (yet) having symptoms. This increases insecurity. Factual or anticipated lack of resources such as unsafe or damaged PPE, as well as lack of knowledge when the first wave or pandemic as a whole will end, increased stress for healthcare workers.

“One main stressor was the communication and information of colleagues, the permanent and fast changes were very challenging, often the overview was lost what is or is not the present procedure lead to different ways of action in some cases…” (Focus group member 4, Italy)

Key category 3: Stressors

Stressor 3.1: New roles/tasks and broken routines

Healthcare workers often had to work in newly formed teams that consisted of staff that did not know each other well. A feeling of powerlessness was often grounded in the fact that new roles had to be carried out, many of which health care workers did not feel appropriately prepared for. Broken routines and lack of opportunities for social exchange were another common stressor. Normal social exchange during breaks as well as common routines could not be used during the shifts. Stress is also grounded in the unavailability of “safe places”. In many cases there was no space for social support or to share experiences such as in common rest areas. One expert pointed out that

“…all the small relieving things that staff are used to were missing, such as sitting and eating together”.

Stressor 3.2. Working with PPE

Usage of PPE in itself is a stressor. High temperature, uncomfortable usage, resulting skin and other health problems are some of the distressing factors. Additionally, health care workers did not drink during work and did not use the toilet often for 12 hours and more because PPE has to be exchanged when taking a break. Many workers were working voluntarily overtime, leading to less time for adaptive processing of emotions.

Stressor 3.3. Loss of professional distance, changed relationship with patients and relatives, moral injury

Moral injury has been expressed by almost all experts. As expected, Triage and PPE shortage were the most prominent stressors. In many cases, staff complained about shortage in supplies, PPE, medications or replacement parts of instruments. This increased the feeling of insecurity among health care personnel. In some cases, scared staff hoarded material for personal protection, which aggravated in further imbalance between demand and supply of equipment. In other cases, lack of resources has not been a major issue. However, even then the anticipated lack of resources created a lot of anxiety and stress. Triage decisions, as well as decisions regarding the distribution of PPE were experienced as extremely stressful especially by the medical doctors involved.

Further indicators of moral injury were mentioned such as lack of being protected, missing or damaged PPE, authorities and leaders knowingly endangering staff, fear of staff to infect others, dealing with dead bodies without the religious rituals normally used or isolation of dying patients from their relatives. All these stressors together created a climate where trust in the system as well as the self-image as a good person were endangered.

The situation was enhanced by a loss of professional distance: Many workers were mediating between families and patients, as relatives were not allowed to visit. These tasks differed significantly from the daily routines that staff were used to and forced the healthcare workers into an unusual intimacy towards patients and relatives that endangered their professional distance. Many healthcare workers were providing psychosocial support to patients who were dying alone in the hospitals. In these cases, part of their professional distance got lost, as they were not able to use the protective strategies they normally use. Due to protective measures, the relationship to patients was experienced differently. Healthcare workers experienced PPE as limiting communication and trust building especially in relationships with “strangers”.

“….patients are dying alone. Few members of the staff enter in the patient room and say "this patient is going to die, what should I do for him/her?" Clearly this is totally different from what they learnt and what is normally happening. They feel like they have the role of mediation and communication between the patient and the family (staff help patients in talking on the phone with relatives)” (Interview partner 1, Italy)

Nurses had to care for dying patients without their routines (having relatives and friends present). Often they were the last ones to see the dying patient and relatives often want to talk to them as they have been the last ones who have talked to their dead relative” (Interview partner, 10, Italy)

Many health care workers experienced stigma as they were avoided or treated differently because of their high exposure to COVID-19 patients. They were perceived as dangerous in their private environment, whilst at the same time being treated as heroes in their professional environment and in the public opinion.

“they felt like virus spreaders”(Interview partner 3, Italy)

For many health care workers appropriate risk assessment was difficult, as staff was not tested regularly in the beginning because of test shortage. Many workers were worried of infecting relatives they considered as vulnerable such as children and older relatives living with them at home. As a reaction, many healthcare workers did not go home to sleep and avoided their families. This strategy of prevention robbed them of the most important resource for recovery.

Key category 4: Resilience factors

According to the experts, health care workers have proved to be a very resilient group. Despite major stressors such as being in new teams and having to deal with dying patients and their families as well as the many experienced threats and moral dilemmas, healthcare personnel adjusted to the new situation and roles and found new routines. Many expressed their pride to have managed a difficult situation together as a team. Regarding social exchange, healthcare workers found new ways to interact with and support each other during the beginning or end of the shifts or during training sessions as well as during shifts when working in pairs.

4.1 Organisational resilience: Leadership, dialogue, protection and guidance

Nevertheless, “external” support by leadership, peers and mental health professionals have proven once more to be of utmost importance. From an organizational point of view, a strong and trusted operational leadership, backed up by multi-professional crisis management staff in the background providing support and strategic guidance, was experienced as the most valuable support.

Leadership takes a central role in how challenges are experienced by health personnel. Several experts pointed out, that clear guidance provided by the management led to more stable routines and therefore was experienced as stress relieving. There was a need for regularly updated guidelines for protection of health care workers as well as accurate information and training e.g. on the use of protective equipment. The better the communication worked and the more the messages were experienced as clear, the better they were received by the personnel. One expert pointed out, that

“written information wasn’t read and thus did not reach all health care workers, while recorded videos by the management were much more efficient with hundreds of employees watching (Interview partner 5, Austria).

Another strategy that was well received was holding regular meetings of team leaders on the balcony in order to exchange important information. Pre job trainings for everybody held face to face in small groups and visits by a mixed team of hygiene experts and mental health professionals directly at the units before shifts in order to support them with PPE use were experienced as very helpful.

Enabling a dialogue between management and staff was very much appreciated. In some hospitals, feedback mechanisms were established. For example, one person from the management was made responsible for “listening to staff” and to take care that reported needs and problems were addressed in a timely manner. These were often just “little things”, but of a high importance to health care worker´s wellbeing. Possibilities to ask questions online were established, so that leadership could react e.g. with information provision via videos. Having hygiene teams sometimes mixed with psychologists available in the wards at the beginning of shifts also seemed to be helpful to reassure personnel in the usage of PPE and provide a sense of safety and appreciation.

Protection of health care workers including sufficient materials, as well as clear guidance and training on how to use equipment has been reported as one of the most important strategies in order to relieve stress for health care workers. In some hospitals, workers with children at home were not sent to work in Emergency Departments. When possible, working in pairs promoted reassurance and thus reduced feelings of insecurity in difficult tasks also because there is mutual help in donning and doffing PPE. Clear directions on how to relate to severely ill patients and relatives were experienced as helpful. Additionally, possibilities to exchange among peers on clinical cases and decision making as well as sharing of special moments has been pointed out as helpful to promote peer support and relief stress among staff. This often asked for an adaptation of areas where people can meet and exchange while keeping the necessary distance, like outside areas or larger rooms.

Guidance and joint decision making regarding triage has reduced a lot of stress. One example is a pro forma “validation” of each patient before a shortage appeared in a medical team according to criteria that have been developed by a mixed team of medical, ethical and psychosocial experts and approved by the management. Thus, in situations when a rapid decision had to be made, for example during a nightshift, the person taking the triage decision had a basis from which to start and did not feel completely left alone.

4.2 Mental health and low threshold psychosocial support

Mental health support was described as most successful when medical leadership and mental health leadership were working openly together. However, mental health support was directly requested from the healthcare workers mainly for patients and relatives.

In many cases, Psychosocial Support did not only include psychologists but also hospital clergy was directly requested when help was needed in mediation between patients and families or in supporting severely ill or dying patients. Direct face to face support as well as coaching of healthcare staff how to deal with families and patients given by clergy, psychiatrists or psychologists, was very well received and reduced stress in healthcare staff. Group interventions for staff were often not possible as staff was too overworked to attend. However, free and anonymous service for staff is essential in providing low threshold support for all staff in the hospitals and also in pre-hospital settings in which anonymous helplines have been well received. Proactive contact to healthcare personnel who had to stay at home in quarantine has also been reported as well received but only when done by someone known and trusted.

Focus on the actual and practical needs of health care workers in the acute phase has been reported to be interventions that are more successful than classical clinical approaches. For example meeting the need for social gatherings (e.g. meeting once a week on the balcony or during trainings in a big room) as well as support in providing small islands of normalcy and common rituals (e.g. Easter) were experienced as extremely helpful. Another example was setting up a piano in the ward for certain hours in order to confront silence.

Psychologists and/or clergy visiting the wards in an outreaching manner was described as much more effective as remote forms of support. Direct contact by mental health professionals and management has been highly appreciated by staff. One expert even spoke of a “knight blow” for the psychosocial staff and several experts pointed out that medical staff did not take it well if mental health professionals stayed in the background only.

Discussion

The output of this article is directly related to the first one and a half months of the COVID-19 pandemic and healthcare crisis in Italy and Austria. The evolving situation will probably lead to different conclusions that will be drawn during the “lesson learning process”, that will take place at different levels in all the healthcare domain.

However, this output has to be viewed as a gathering of first level immediate impressions and experiences from healthcare staff involved in this massive healthcare crisis. Like a ship’s logbook it contains the day to day evolution and perception of the healthcare personnel through the lens of mental health and medical experts who themselves were deeply involved into the mission of “fighting corona”.

In knowledge management, lesson learning is a process that is usually done after the crisis with a debriefing mechanism and having players of multiple roles sitting at the table to draw common conclusions. Lesson learning is usually defective in the part that deals with retrieving and analysing first-hand experiences. We do not want this to happen for the COVID-19 crisis for what pertains to the in-hospital personnel psychosocial response.

The importance of this work can be summarized in the following three points:

Gathering of first-line experiences and impressions

Steering the steps of a future lesson learning process

Precociously identifying issues that due to the long timeline of the COVID-19 crisis might be of help to countries whose response will be postponed in time as well as to all those who are faced with further waves of the pandemic.

One of the main hypotheses that we derived from our data is that moral injury seems to be a common and central outcome of this crisis in many healthcare workers, be it in areas where the virus has had an extreme negative impact on the healthcare system or in areas where the pandemic had not so much negative impact (yet). Even though extreme measures like triage did not have to be applied in all areas, other challenges to healthcare workers’ ethical self-image and worldview have been badly shaken by this crisis. To be forced to separate dying patients and their families, to have to handle a dead body without the usual rituals, to be not only a helper but at the same time a possible threat to your own colleagues and family have been experiences shared by many healthcare workers during this pandemic.

Our second main hypothesis is based on the findings that organizational justice as well as a flexible decentralized approach to leadership lead to more collective and organizational resilience during this pandemic. Our findings support the assumptions of Greenberg et al. [32] or Williams et al. [33]. Loss of trust in oneself as a helper as well as loss of trust in the system or in the leadership are common effects of moral injury. From a practical point of view, the challenge that we are facing now after the first phase of the pandemic is to rebuild trust in ourselves as helpers as well as in the systems that we are working in. As Maunder et al. have stated in 2008, those hospitals that have a structure of flexibility, decentralized decision making as well as continuous training and self-administration of units fared better during this crisis as highly centralized organisations with an administration that did not understand the needs and concerns of the staff “on the ground” [42]. Furthermore, as Lancee et al. [26] also stated, leadership and organisational justice played an important role in counter balancing moral injury. Being taken seriously in their concerns as well as visible efforts to take just and well-grounded decisions were main resilience factors also in our experts´ view.

Our third hypothesis is that psychosocial support of healthcare workers during this pandemic only works if it is given by trusted mental health professionals or peers in a very basic manner integrated into the overall support by the management (trainings, information, frameworks provided). The best support given during the first phase of a disaster response is on scene support guided by the motto ‘to act with the people and not for the people’ [47]. Thus, mental health professionals who acted with the healthcare personnel and not for them (from an external safe position) and who acknowledged and appreciated the resilience that this group already has and did not treat them as potential patients were the only ones whose support was accepted.

Williams et al. [33] recommend several strategies to deal with moral distress and moral injury faced during COVID-19. They base their recommendations on the well-known elements of effective psychosocial support: safety, connectedness, self and collective efficacy, calm and hope [48]. Authors such as Juen and Warger [49] as well as Dückers et al. [50] totally agree with this. Hobfoll et al. [48] mention the need to translate the elements into the given context of each disaster.

Conclusions

According to our data, safety refers to good, honest and timely information given most directly by trusted leaders. Sufficient PPE but also visible efforts to support and protect staff by the management as well as interest in how they manage the situation leads to a feeling of safety in a situation where many healthcare workers had the feeling that there is no safe place left. Paradoxically, those who worked in the COVID units often mentioned to feel safer than those in other units where no one knew when a patient could be tested positive. This was especially true during the very first phases when PPE was not available for everybody. Connectedness refers to efforts of staff themselves and their leadership as well as mental health professionals and peers to enhance group cohesion and allow for social exchange in spite of difficult circumstances. Examples were meetings on the balcony after the shift for a quick exchange. Connectedness also was expressed by proactive contact to those who had to stay at home because of infection or quarantine. Calm was reached by providing enough space for rest and recovery (e.g. accommodation when staff does not want to go home). Calm was also reached by re-establishing normalcy including common rituals during holidays or other efforts to establish a normal working environment. Self and collective efficacy has been reached by preparing staff for their new tasks, taking joint decisions for example by discussing ethical guidelines together and trying to find joint solutions. Caring for dying patients and connecting with their families have been main stressors, but also sources of self-efficacy and pride. The element of hope refers to the fact, that healthcare staff, despite their distress, exhaustion and many sacrifices, went through this crisis together and managed the task successfully. From what we have learned, we can conclude that many of the healthcare workers have shown an extraordinary resistance. They have been able to work together and function well as teams in very challenging circumstances (new tasks, new teams, new challenges, rapidly changing environment). They have often overworked themselves until they were completely exhausted. Many of them took damage to their health or lost their lives due to the pandemic. Faced with extremely distressing ethical dilemmas, they have been able to take decisions based upon rational models and guidelines while at the same time providing the best care possible for their patients in these difficult times.

COREQ checklist.

(PDF)

Click here for additional data file.

Psychosocial considerations for health care workers in the COVID-19 pandemic.

(DOCX)

Click here for additional data file.

Ethics statement by the institutional review board of the University of Innsbruck and external ethics review board of the NO-FEAR project.

(PDF)

Click here for additional data file.

12 Jan 2021

PONE-D-20-34707

Ethical and Psychosocial considerations for hospital personnel in the COVID-19 crisis: Moral Injury and Resilience

PLOS ONE

Dear Dr. Kreh,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by 5 February. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Francesco Di Gennaro

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include additional information regarding the interview guide used in the study and ensure that you have provided sufficient details that others could replicate the analyses.

For instance, if you developed an interview guide as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information.

In addition, we recommend that authors use the COREQ checklist, or other relevant checklists listed by the Equator Network, such as the SRQR, to ensure complete reporting (http://journals.plos.org/plosone/s/submission-guidelines#loc-qualitative-research). Please attach such a checklist as Supporting Information.

3. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

Additional Editor Comments:

dear authors follow reviewer suggestion to improve your paper

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: the study was a descriptive based on intensive interviews and was writing in clear standard English understandable for scientific community as well as lay person everything was technically sound and I have no comments on it

Reviewer #2: 1.Only 13 experts were interviewed in this study. Is it fully representative and statistically significant？Whether there is a bias in the selection of study object？

2.Please provide the interview outline and questionnaire.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Dengchuan Wang

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

12 Mar 2021

Dear Editors,

dear Reviewers,

Thank you for reviewing our submission entitled Ethical and Psychosocial considerations for hospital personnel in the COVID-19 crisis: Moral Injury and Resilience (PONE-D-20-34707).

Attached to this letter we are uploading a marked-up copy of our manuscript that highlights all changes made as well as an unmarked version of the revised paper.

In the following, we want to respond to all points raised by the reviewers.

1. Style requirements

We updated the manuscript according to PLOS ONE’s style requirements.

2. Additional information

We now include our interview guide in the supporting documents section.

Given that we conducted our research in the very beginning of the COVID-19 pandemic, an exploratory approach was necessary to gain insights on the health care worker’s perspective in this new environment. We thus followed the principles of openness and induction, as we stated in the methods section of the revised manuscript. Our interview guide contains topics that provide a framework of orientation to ensure comparability and accompany the communication process. However, instead of asking specific, predetermined types of questions the guide relies on the interaction with the interviewee to steer the interview process and helps constructing questions on the topics of interest as the interview progresses – a method that is widely used in qualitative research (see e.g. Witzel, 2000; Edwards & Holland, 2013, p. 55)

We also now provide the COREQ checklist to ensure that we addressed all points relevant for complete reporting.

3. Data availability statement

We were asked to make all data underlying the findings in our manuscript fully available.

However, in our first submission we pointed out concerns as our interview transcripts contain sensible participant information. Publishing the full transcripts is also excluded in the informed consent our interviewees signed. In order to meet transparency needs, excerpts of the transcripts relevant to the study are made available within the paper.

We also provided an overview of our conceptualized data as in this form it does not contain sensible participant information. It is available in a public repository:

“Full data cannot be shared publicly because they contain potentially identifying or sensible participant information. However, conceptualized data after using grounded theory methodology can be retrieved from doi.org/10.5281/zenodo.4241858”

4. Additional review comments

Reviewer #2 pointed out concerns regarding our sample size, whether there is a bias in the selection process and whether it is fully representative and statistically significant.

We would like to point out the inherent principles of sampling approaches in qualitative research. We did not use a random sampling approach in order to find a statistically representative sample (as it is indispensable in quantitative research). Instead, we purposefully selected information-rich cases, aiming at in-depth understanding rather than statistically proven generalizations (see e.g. Patton, 2002, p.230). We thus do not view our sample as biased, but consider the selection of highly experienced interviewees directly involved in the response of the pandemic as a quality criterion of our data.

We furthermore believe that our approach is in line with trends of narrative-based medicine in recent years (see e.g. Kalitzkus & Matthiessen, 2009).

Our aim in this manuscript was to develop some basic hypotheses regarding resilience and stress experience of healthcare workers in the first phase of the COVID-19 pandemic. However, we agree that a statistical survey would add to the value of our findings. We are therefore currently conducting survey studies with health care workers. The construction of the surveys is based on the findings from qualitative research presented in this manuscript. Findings from quantitative follow-up studies will be presented at a later stage.

Sources

Edwards, R. & Holland, J. (2013). What is qualitative interviewing? Bloomsbury Academic.

Kalitzkus, V. & Matthiessen, P. (2009). Narrative-Based Medicine: Potential, Pitfalls, and Practice. The Permanente Journal, 13(1), 80-86.

Patton, M. Q. (2002). Qualitative research & evaluation methods. 3rd (ed.). Thousand Oaks, CA: Sage.

Witzel, A. (2000). The Problem-Centered Interview. Forum: Qualitative Social Research, 1(1), Art. 22. Available at https://www.qualitative-research.net/index.php/fqs/article/view/1132/2521

Thank you for taking our manuscript under further consideration.

Best regards,

Alexander Kreh

Corresponding author

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

22 Mar 2021

Ethical and psychosocial considerations for hospital personnel in the COVID-19 crisis: Moral injury and resilience

PONE-D-20-34707R1

Dear Dr. Kreh,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Francesco Di Gennaro

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

dear authors congratulations

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: Yes: Dengchuan Wang

26 Mar 2021

PONE-D-20-34707R1

Ethical and psychosocial considerations for hospital personnel in the Covid-19 crisis: Moral injury and resilience

Dear Dr. Kreh:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Francesco Di Gennaro

Academic Editor

PLOS ONE