Using a human factors-centric approach to development and testing of a face shield designed for health care workers: A COVID-19 case study for process and outcomes.

BACKGROUND: Face shields are a critical piece of personal protective equipment and their comfort impacts compliant use and thus protectiveness. Optimal design criteria for face shield use in healthcare environments are limited. We attempt to identify factors affecting face shield usability and to test and optimize a face shield for comfort and function in health care settings.
METHODS: A broad range of workers in a large health care system were surveyed regarding face shield features and usability. Quantitative and qualitative analysis informed the development of iterative prototypes which were tested against existing shields. Iterative testing and redesign utilized expert insight and feedback from participant focus groups to inform subsequent prototype designs.
RESULTS: From 1,648 responses, 6 key elements were identified: ability to adjust tension, shifting load bearing from the temples, anti-fogging, ventilation, freedom of movement, and durability. Iterative prototypes received consistently excellent feedback based on use in the clinical environment, demonstrating incremental improvement.
CONCLUSION: We defined elements of face shield design necessary for usability in health care and produced a highly functional face shield that satisfies frontline provider criteria and Emergency Use Authorization standards set by the Food and Drug Administration. Integrating human factors principles into rapid-cycle prototyping for personal protective equipment is feasible and valuable.

Introduction

To protect health care personnel (HCP) from SARS-CoV-2 exposure, both Centers for Disease Control and Prevention and World Health Organization guidelines recommend that HCP entering the room of a suspected SARS-CoV-2-infected patient don Personal Protective Equipment (PPE) appropriate for airborne and contact transmission, including medical mask, eye protection, nonsterile full-body gown, and gloves. , Eye protection can be in the form of goggles or face shields.

Face shields, when worn in conjunction with a face mask that covers the mouth and nose, facilitate both eye protection and an additional barrier for the medical mask. Using a cough simulator, face shields blocked 97% of large particle transmission to the providers’ respirator mask, blocking disease transmission through facial mucous membranes. A case-control study identified PPE misuse as a significant risk factor for HCP COVID-19 infection, and a recent systematic review has shown significant reductions in HCP COVID-19 infection with the fastidious use of face shields.

During the initial wave of COVID-19, many aspects of the worldwide supply chain were interrupted by lockdowns, sourcing competition, export bans, and other distribution disruptions. These disruptions affected 35% of manufacturers and coalesced with a massive spike in PPE usage by HCP to create a shortage of PPE. The US Food and Drug Administration (FDA), issued an Emergency Use Authorization (EUA) on the use of improvised PPE due to inadequate output from traditional production lines. At that time, the abrupt paucity of PPE had necessitated ingenuity for the generation of alternative supply chains that rapidly manufacture unregulated and untested products.

The widespread dissemination of information using social media at the time facilitated a “citizen supply chain”, comprised of community volunteers who use their personal 3D printers to fabricate face shield parts using templates downloaded from the National Institutes of Health (NIH) 3D Printing Service. , Many designs from around the world were and are freely accessible from the NIH to users who can independently produce and assemble completed face shields at a low cost with minimal labor [https://3dprint.nih.gov/].

Since the initial emergency response and the EUA issuance, updated guidance has been provided for face shields specifically. However, it is likely that the increased virulence of variants of SARS-CoV-2, and the experience of living within a pandemic will result in new perceptions by health care organizations about the importance of PPE. For example, individuals may feel more comfortable wearing masks or face shields as part of normal work, rather than for specific clinical tasks. In addition, face shields will continue to be used for protection during procedures and at other points in clinical care. Therefore, the need to design a face shield to promote comfort and wearability remains, even once the pandemic has further abated.

Comfort, fit and functionality can dramatically impact face shield use, particularly when used continually throughout a 12-hour shift. Despite obvious and documented personal risk, more than half of HCP self-report noncompliance with their use of facial PPE. Even amidst the 2002 sudden acute respiratory syndrome outbreak and 2009 swine influenza (H1N1) pandemic, HCP frequently misused facial protection and cited discomfort and poor usability as reasons for noncompliance.13, 14, 15 COVID-19 cases correlate inversely with PPE compliance, demonstrating the need to ensure the usability of face shields for the health of HCP and thus reduce the risk of exposure.

A systematic approach to PPE for infectious disease should entail multidisciplinary assessment of PPE based on exposure risk, job hazard analysis, proper PPE selection, and evaluation of selected PPE to optimize usability and protection. Even after the implementation of a design, it is important to iteratively re-evaluate in order to understand where possible redesigns are needed. The field of human factors is concerned with understanding how human cognitive, physical, and interpersonal capabilities and limitations influence how humans work, with the ultimate goal of designing the work to fit the human (rather than vice versa). In this case, we are particularly curious how to design face shields that will be comfortable enough to be worn for extended periods of time for users with different craniofacial structures, those with glasses and without, and those that engage in rigorous physical movement during their job.

The impetus behind this study was initially practical; SARS Cov-2 response dictated that individuals wear face shields for long periods of time, and the team wanted to help individuals be able to adhere to these recommendations without discomfort. In the longer term, our team realized that there are opportunities to design health care PPE with the same rigor that other high-risk industries (eg, welding or manufacturing) do, with a design that promotes comfort and decreases workplace injuries/exposures. To our knowledge, there is not a rigorous study that has developed specific criteria to inform face shield redesign based on HCP-reported usability in the clinical context. In this project, our goal was to determine usability issues that frontline HCP experience with current face shields and to develop design criteria for a prototype. Using this data, we developed a prototype and, via randomized testing, iteratively tested and updated our design. Our goal was to use this human-centered design approach to maximize face shield comfort, use, perceptions of safety, and intention to wear throughout a shift.

Methods

This study took place in 4 phases, and used a mixed method approach. It was approved by the Institutional Review Board (20-1055). The sample demographics for each phase of the study can be found in Table 1 .

Table 1

Demographics of the sample population for each phase of the study

Phase	Number of responses/participants	Clinical roles	Experience (years ± S.D.)
1	1,609	N/A	N/A
2	26	Registered nurses (18)	8.09 (± 10.35)
		Clinical associates (2)
		Environmental service (1)
		Occupational therapy (1)
		Rehab (1)
		Respiratory therapist (1)
		Unidentified (2)
3	10	Registered nurses (4)	2.02 (± 1)
		Clinical associate (1)
		Rehab (1)
		Speech pathologist (1)
		Unidentified (3)
4	13 (4 incomplete or no response)	Registered nurses (9)	7.27 (± 7.52)
		Clinical associate (1)
		Rehab (1)
		Speech pathologist (1)
		ICU unit director (1)

Phase 1: Shield assessment

Survey process

Respondents to this survey consisted of HCP including frontline and office staff, novices, and experts, across all departments. Requests for participation were sent to each individual in the organization in the daily leadership briefing email and in the daily briefing email at our institution. Any staff that used a face shield, regardless of the frequency of use during a shift, was eligible to participate and provide their feedback in regard to the evaluation of the use of the face shields.

With an estimation of over 11,000 HCPs across the organization receiving the request for participation in the Phase 1 study, a total of 1,648 responses were received, with a response rate of 15%. Among all the responses, 39 responses were incomplete, completed by participants that did not use a face shield, or the respondent indicated use of a design other than the six of interest, resulting in 1,609 eligible surveys.

Face shields evaluated

Six of the most commonly available face shield designs were identified (Appendix A). Participants were asked to identify the face shield design that they used most often. These designs were chosen because they were immediately available to HCP through dissemination by our institution and allowed for the identification of key elements that may be unique to a particular design.

Survey design and dissemination

The survey consisted of 4 questions, 3 of which were quantitative questions, and one question was an open-ended free-text response. The survey was created using Microsoft Forms (Appendix A) and was distributed to the hospital organization via email and in-person onsite; data collection remained open for 20 days.

A descriptive analysis was applied to understand user preferences and comfortability for existing face shields. Further, usability issue trends were identified using the picklist data from survey question 3 and a thematic analysis from survey question 4. The thematic analysis was performed by 3 Human Factors experts reviewing the 973 free text submissions. Face shield issues were grouped into themes and common problems using constant comparative methods, adding an additional 15 reported problems to consider for analysis. A Multivariate Analysis of Variance (MANOVA) was performed with the face shield type as the independent variable and 26 categorized survey themes as dependent variables (DVs).

Phase 2: Prototype I design and testing

Prototype design

Our team, comprising a multidisciplinary group of experts representing human factors, emergency medicine, internal medicine, respiratory therapy, and mechanical engineering, synthesized the Phase 1 findings into 6 design criteria. We then used these design criteria to update the prototype and manufacture (3D print) a set of test face shields for use.

Participants

Participants of prototype testing were clinical staff recruited from throughout the health care system but focused on acute inpatient care (nurses, environmental services, physical therapy, occupational therapy, phlebotomy, etc). Each participant's written informed consent was obtained, and they were informed that the purpose of this study was to obtain their feedback on the face shield they used for the duration of this study to reveal usability issues that can be further addressed, and that their personal information and feedback responses would be de-identified. We approached 47 HCPs to be part of the Phase 2 study, resulting in 26 participants due to 16 nonresponses, 4 refusals due to scheduling conflicts, and 1 incorrect contact information.

Prototype testing

After a list of interested participants was generated, they were screened by researchers. Screening criteria included participants who regularly use a face shield when working in a clinical setting and are scheduled to work at least one shift during the week of testing. Participants also had to pick up and drop off of the face shield design being assessed to be eligible and were then randomly assigned to either the Control Group (CG) or Treatment Group (TG). In this 1-week study (from September 2, 2020 to September 9, 2020), participants in the CG were asked to wear their current existing face shield; participants in the TG were asked to wear the new face shield prototype. TG participants were free to return to their old face shield if they felt unsafe using the prototype.

Prototype evaluation

Both groups were asked to fill out a System Usability Scale survey (Appendix B) at the beginning and end of the study. Participants also completed a daily survey evaluation of their face shield (Appendix C).

Comparative analysis

Descriptive analysis of both surveys was conducted to detect variations between pre- and post-System Usability Survey results and daily ratings between CG and TG. A MANOVA was conducted with the face shield type as the independent variable and categorized survey measures as DVs to further highlight the differences between existing (CG) and newly designed (TG) face shields. A post-hoc Tukey HSD test for significant DVs was performed to see which face shields significantly differed in usability parameters.

Prototype focus groups and analysis

A focus group study was held for the TG after their use of the prototype face shield. The following questions were discussed: (1) “What did you like about the prototype face shield?” (2) “What did you dislike?” (3) “Does it get in the way of your tasks and how?” (4) “What suggestions do you have to make it better?” Three researchers were present in all focus group sessions to conduct note taking in real time; participants’ qualitative feedback surrounding above questions was then summarized and coded into the following categories: (1) Positive comments on the prototype, (2) Negative comments on the prototype, (3) No Difference (from old face shields), and (4) Interesting Findings.

Phase 3: Prototype II Testing

Prototype testing and analysis

Ten frontline HCPs were recruited (4 participants from Phase 2, 6 new participants) to wear the new face shield prototype during one shift. Participants were instructed to treat the prototype as they would normally treat their face shield and return to their old face shield if they felt unsafe using the prototype. Individuals participated in a brief 15-minute interview and a verbal survey (Appendix D) with one member of the human factors team, who summarized and coded the frequency of reported issues as in Phase 2.

Phase 4: Prototype III testing

This Phase was identical to the Phase 3 prototype trial, the only difference being the updated prototype design. A total of 13 frontline HCPs were recruited; 2 were nonresponses, and 2 withdrew in the mid-process of Phase 4, resulting in 9 frontline HCP participants (2 had participated in Phase 3 only, 7 were completely new to the face shield study, and no participant had been part of Phase 2, 3, and 4 throughout). Individuals participated in a 15-minute brief interview and a verbal survey with one researcher, who summarized and coded the frequency of reported issues as in Phase 2.

Results

Face shield designs

All iterations of face shield designs can be found and downloaded online.[https://labs.vtc.vt.edu/parker/research/face-shields/] Calculated in conjunction with manufacturing partners, the current shield design can be manufactured for under $15.

Phase 1

From the survey responses, 4 major design elements were identified: too hot while wearing the face shield, fogs easily, has poor visibility, and is difficult to clean (Table 3 ). Out of 199 participants who have used face shield type A, 88.9% of them reported the problem of fogging up being the top concern. Similarly, fogging up was the most reported problem for face shield designs B, C, E, and F as identified by 88.5%, 88.6%, 79.2%, and 76.5% of respondents, respectively. Whereas, poor visibility was the most frequently reported problem for face shield type D.

Table 3

Top three themes of reported problems by face shield designs (n = 1,609 [ A = 199, B = 655, C = 508, D = 43, E = 106, F = 98])

Themes		Face shield A	Face shield B	Face shield C	Face shield D	Face shield E	Face shield F
Product practicality	Fogs up	88.9%	88.5%	88.6%	69.8%	79.2%	76.5%
	Poor visibility	73.9%	73.3%	72.6%	55.8%	50.0%	66.3%
	Too hot	69.3%	61.8%	63.0%	41.9%	48.1%	64.3%
Product design	Snags hair	28.6%	38.6%	31.7%	27.9%	36.8%	35.7%
	Leaves marks on face	27.6%	19.4%	17.5%	20.9%	23.6%	19.4%
	Too big/small	23.1%	14.2%	14.2%	14.0%	16.0%	18.4%
Staff safety	Hurts face	20.6%	12.2%	14.4%	9.3%	30.2%	20.4%
	Headache	7.5%	5.6%	6.1%	7.0%	14.2%	7.1%
Patient safety	Difficulty breathing	4.0%	3.8%	4.3%	0.0%	4.7%	3.1%
	Patient safety concerns	1.0%	6.0%	3.7%	4.7%	3.8%	3.1%

Qualitative analysis of free text responses identified additional recurrent issues: the face shield got in the way of the task, was problematic for staff with glasses, got scratched, dented, or creased easily, resulted in headaches, and made it difficult to communicate. Five to 10 percent of participants reported that the shield fit too close to the face, had patient safety concerns, difficulty breathing, or created glare.

Four primary themes were developed using the results of both quantitative and qualitative feedback from survey questions 3 and 4 respectively: (1) Product Practicality, (2) Product Design, (3) Staff Safety, and (4) Patient Safety. Findings from the thematic analysis of Survey Questions 3 and 4 are shown in Table 3.

The MANOVA analysis was significant for face shield type (Pillai's Trace = 0.269, F = 3.454, df = (5,130), P < .001), indicating that staffs’ feedback was impacted by which face shield that they used (Appendix E). Post-hoc Tukey HSD analysis (Appendix F) found that no current design had a clear design advantage over any other. For example, participants rated design “D” worse for cleaning and durability, but scored better compared to others on fogging up or being too hot. Based on our quantitative and qualitative findings, our multidisciplinary team integrated common issues to inform the development of 6 design criteria as articulated in Table 4 .

Table 4

Design criteria and respective supporting data

Design criteria	Supporting data
The ability to adjust the visor (both closer or further from the face and vertical lift for articulating it up/out of the way)	Visor being too close, fogging, too hot, problematic for staff with glasses or wearing additional PPE under the visor, leaving marks on the face, headaches, difficult to communicate
Lighter design (shift weight to the top of the head)	Leaving marks on the face, headaches, hurts face, claustrophobia
Ventilation	Too hot, difficulty breathing, fogging, inability to properly care for patients
Anti-fogging, anti-glare shield	Fogging, glare, inability to properly care for patients
Flexibility of movement	Inability to properly care for patients, too big/small, headache
Durability	Difficult to clean, dents or creases

Phase 2

To address the major problems reported in Phase 1, a newly designed face shield was prototyped by our team, including engineering collaborators. This face shield consisted of 3 components: (1) An anti-fog coating on the inside of the plastic face covering and an opaque visor on the top of the forehead to reduce glare,[https://fsicti.com/] (2) 3D-printed core plastic with a controllable hinge to flip the front covering up and down, as well as adjustable venting interspace above the forehead to reduce heat, and (3) elastic straps with adjustable tightness as well as support preference on the top and/or back of the head to accommodate migraine due to tightness.

Throughout the week of the study, the average and standard deviation for shifts worked by participants for CG was 3.75 ± 1.36 shifts, and TG was 3.57 ± 1.16 shifts. The issues with the largest differences in reporting between groups were: fogs up, difficulty communicating, poor visibility, too hot, and easily scratches or creases; however, the TG largely reported issues with the prototype's size and mobility for head movement compared to the CG. From pre- to post-study, CG's System Usability Survey score decreased from 66.8% to 62.2%, whereas TG's System Usability Survey scores showed an increase from 70% to 81%. The MANOVA was significant for face shield type (Pillai's Trace = 0.859, F = 16.649, df = (1, 25), P < .000), indicating that staffs’ feedback was impacted by group assignment (Appendix F).

A total of 8 participants (57%) from the TG participated in the focus group session, and 3 focus group sessions were held, each lasting about 30 minutes. Summarizing all the qualitative responses, the majority of the responses (55%) keyed in the Positive category, while Negative and No difference reflected 25% and 7% of remarks, respectively. Positive remarks included “Anti fog helps with eye contact” and “Old one caused migraine, but this one did not bother too much”. An example of a Negative finding was “Sometimes compensate glare by tilting head“. Themes also included unique weighting comments, such as “Prefers glare over fogging”.

Phase 3

Phase 3 participants had an average (± S.D.) of 2.02 ± 1 years of experience, with 3 unidentified expertise in their role. During individual interviews, participants provided a total of 18 positive and 15 negative comments. A majority of participants liked the face shield (80%) and specific remarks complimented the straps, distance to face, anti-fog, headache resistance, and flipping feature. However, out of the 15 negative comments, 47% of the negative comments corroborated the issue of the hat bill being too tight, creating pressure against the forehead, resulting in headaches. The remaining negative comments were: communication difficulty (20%), trouble flipping (13%), glare (7%), loose fit (7%), and hard to clean (7%). In comparison with Phase 2′s face shield design, user perceptions of Phase 3′s face shield showed improvements in the sizing, head mobility, and compatibility with conducting tasks. However, respondents still perceived the updated design as worse in communication and head comfort.

Phase 4

Phase 4 participants had an average (± S.D.) of 7.27 ± 7.52 years of experience. Mirroring our Phase 3 methodology, the number of reported issues was reduced from 15 to 10 in Phase 4. Certain issues were reported at greater than 10% frequency: gets in the way of tasks, glare, poor visibility, and problematic with glasses. However, issues including headache, leaves marks on face, and wear it correctly all decreased, and all of the following issues were outright eliminated: (1) Hurt face, (2) Snags hair, (3) Too big/small, (4) Difficulty moving head, or (5) Fog up.

Discussion

We have demonstrated important elements necessary in face shield production and specific criteria to inform face shield redesign based on HCP-reported usability in clinical environments. Our data identified 6 specific face shield design criteria that were informed by the input of a diverse group of HCP in a large health care system (Table 4). These 6 criteria serve 2 important functions. First, they concisely convey the chief concerns confronted by HCP in the clinical context, which allows for iterative input and redesign based on empiric feedback from those staff that uses the face shields. Second, through a partnership with engineers, this information and 3 rounds of iterative testing allowed the development of freely accessible face shield designs with progressive improvement. Unsurprisingly, previous research during the COVID-19 pandemic has shown that HCPs prefer face shield designs that are optimized with user feedback. ,

The reduction in dissatisfaction of market-available face shields seen in Figure 1 illustrates the importance of feedback and iterative design. Issues of heat, visibility, problems with glasses, and fogging were essentially eliminated. The issues experienced by providers changed over time and with newer iterations of the face shield. Issues of glare, difficulty communicating, marks on face and headache were never fully resolved. However, the statement that participants would leave the face shield on for the duration of a shift suggests that perhaps the baseline on which discomfort was being described may have shifted.

Fig 1

Comparisons of reported issues between Phase 2 (CG; n = 12), Phase 2 (TG; n = 14), Phase 3 (n = 10), and Phase 4 (n = 9).

The material from which the shield itself (other than the visor) was made seems to have had a significant impact on many of the perceived issues from frontline staff. For example, there was a dramatic decrease in concerns about fogging once our shields utilized a specific plastic with a coating design to minimize fogging. With each iteration, the team learned and was able to update their design. This type of iterative process is frequently undertaken by design firms, but we were able to reproduce it with frontline staff, while doing their clinical work.

A recent study by Moshaghimi et al showed that the majority of staff found their feedback-informed design to be much more comfortable than the standard-issue face shield. In their study, they identified that visibility remains a challenge, an issue we were able to address in our design. We believe our study supports findings of others and advances the science by providing insight from frontline providers using the shield in a clinical environment for an extended period of time, which we believe is a unique contribution to the literature on PPE design. The documentation of iterations shown here highlights the importance of an interactive process to finalize a design that will be broadly used.

This work also demonstrates both the feasibility and value of rapid iterative testing and redesign in clinical environments, which is uncommon, and we hope that this project serves as a roadmap. Stepwise testing uncovered design issues in subsequent rounds that were likely obscured initially by the major dissatisfiers. While not easily described, our team found that by eliminating the 6 major issues, test groups seemed more “able” to focus on the minutiae of the design and allowed for additional refinement. Given the immense variability in human factors (anthropomorphics, tasks performed, HCP preference, etc.) and other constraints (cost, availability, time, etc.) regarding face shields, it is worth noting that a truly perfect design is an impossibility, as demonstrated by our intentional and incremental improvement in usability ratings over time.

The goals of this study were highly translational, and the study itself took place in a health care system that was overstretched due to patient care during a pandemic with associated limitations due to that lack of control. Therefore, we made multiple decisions in our Phase 1 data collection process to minimize the time required for participation in favor of a higher response rate. Demographic information (e.g., specialty, expertise, age, gender, etc.) was not collected, though we feel that our sample size presumes an adequately diverse HCP population in Phase 1. We requested a broad sample to elucidate face shield challenges faced by all frontline staff with feedback based on usability in the environment of use, the clinical setting. We also believe that we have a representative sample of face shield designs, particularly given the immense variability in such. Phases 2, 3, and 4 similarly faced pandemic-induced stressors. Participant recruitment mobilized abruptly and had to take place digitally via phone call or email over the course of fewer than 2 days. These constraints limited sample size as well as the repetitiveness of certain participants participating in multiple phases, rather than having no experience prior to each testing, though this was compensated for in Phase 2 by daily surveys from participants, which provided consistent results over time. We intentionally avoided targeting a specific group (Intensive Care Unit, Emergency Department, Physical Therapy, etc) for testing to better understand the context of use in a wide variety of areas.

Although we solicited iterative input from infectious disease to corroborate prototype protectiveness, aerosol testing to quantitate the true protectiveness of our shield prototypes was not performed. The protectiveness of the shield was not altered in our design, rather the design of the visor and the material used for the shield itself.

This study represents a human factors approach to iterative face shield redesign in the context of rapid prototyping and mobilization in the COVID-19 era and an important addition to the translational literature. We believe it remains relevant for ongoing response to SARS-CoV-2 variants and for ongoing PPE usage. The breadth of survey responses lends credence to the widespread applicability of these design criteria within other healthcare systems.

Table 2

Phase 1 survey questions, their objective, and the method of data collection

Question	Objective of question	Type of data
Do you like this face shield?	To illustrate variations in face shield preference	Likert 5 point scale
How comfortable is this face shield?	To illustrate variations in face shield comfortability	Likert 5 point scale
What problems have you experienced with this face shield?	To understand what issues were present in the design	Multipicklist: Hurts my face, Too big/small, Snags hair, Leaves marks on face, Fogs up, Too hot, Poor visibility, Difficult to clean, and Breaks easily
What other problems have you encountered with this face shield?	To capture any other issues not yet identified and/or to capture problems not selected in question 3	Free text