In hospital verification of non CE-marked respiratory protective devices to ensure safety of healthcare staff during the COVID-19 outbreak.

AIM: To develop a protocol to ensure the quality of respiratory protective devices for healthcare workers nursing and treating patients with possible or confirmed COVID-19 in the Catharina hospital.
BACKGROUND: Due to the COVID-19 outbreak a shortage of respirators is occurring worldwide; more specifically, CE-certified FFP2 respirators. This has resulted in an increased supply to hospitals of alternative respirators of uncertain quality. Nevertheless, the quality of the respirators used by our healthcare workers must be ensured.
METHOD: A protocol and criteria based on applicable standards was developed to ensure the quality of respirators. The protocol has been implemented at the Catharina hospital and includes verification of the documents accompanying the respirator, visual inspection of the respirator and a test for total inward leak of particles into respirators.
FINDINGS: 67% of the respirators brands and types received in the Catharina hospital did not meet quality criteria.
CONCLUSION: With a simple verification protocol the quality of the respirators can be checked and guaranteed while there is a shortage of the CE approved respirators which are normally used. With this in-hospital protocol health care workers can be equipped with safe-to-use respirators.

Introduction

Healthcare workers caring for patients with possible or confirmed coronavirus disease 2019 (COVID-19) have to be protected against contamination. In Europe, in normal circumstances, the ‘respiratory protective devices’ (in short, respirators) used by these healthcare workers have to comply with the standard EN149+A1:2009 [1]. Most respirators used in hospitals are single-use. Due to the COVID-19 pandemic, an exceptionally large demand for these respirators has arisen and shortages have occurred. Authorities have published recommendations to overcome shortages of these respirators [2,3]. In Europe, national authorities have indicated that during the COVID-19 crisis the restriction of solely using Conformité Européene (CE)-certified respirators no longer applies [4]. As a consequence, the European market is flooded with respirators of unknown or uncertain quality [5]. Nevertheless, it is imperative that the safety of the healthcare workers is ensured. Therefore, the Catharina Hospital developed a protocol that would allow local (in-hospital) verification of the quality of these respirators before clearance for use by staff. The protocol was developed through an interdisciplinary project team comprising a pharmacist, biomedical engineer, infection preventionist, project manager, and a physicist. The protocol includes verification of the documentation accompanying the respirator, a visual inspection of the respirator, and finally the testing of full filter efficiency of the respirator. The filter efficiency is compared with the values of a premium CE-marked respirator that complies with EN149+A1:2009 [1].

The standard EN149+A1:2009 [1] specifies three categories – FFP1, FFP2 and FFP3 – where FFP stands for ‘Filtering Face-Piece’. FFP1-, FFP2- and FFP3-certified respirators must meet a ‘total inward leak’ (TIL) of 22%, 8% and 2% respectively. These percentages indicate the maximum number of particles that may ‘leak’ through the respirator during testing. The test should reflect the practical use of the respirator. The Dutch National Institute for Public Health and the Environment (RIVM) recommends the use of the FFP2 type when nursing and treating COVID-19-infected patients during aerosol-generating procedures [6]. Apart from the FFP2 respirator there are two other types of certified respirator on the worldwide market that have comparable properties. These are the N95- and KN95-qualified respirators. Both types of respirator have been introduced in Dutch hospitals due to the shortage of FFP2 respirators.

The N95 variant is commonly used in the USA and meets the requirements of the NIOSH-42CFR8 [7]. The US Food & Drug Administration specifies: the N95 designation means that when subjected to careful testing, the respirator blocks at least 95 percent of very small (0.3 μm) test particles [8]. The other variant is the KN95 respirator meeting the Chinese requirements [9]. This respirator has a requirement for particle blocking similar to that of the N95-qualified respirators.

In normal circumstances the production facility/company has responsibility for the claims and quality of the respirators. The quality system of the production facility and quality of the final product is approved by a European-certified notified body. However, currently, the origin of the respirators that are supplied to the hospitals is not always traceable.

In this article a protocol is specified for a quick and robust method to locally (in hospital) verify the quality of non-CE-marked respirators to ensure safety of the staff in the Catharina Hospital during the COVID-19 pandemic.

Methods

During the COVID-19 crisis, the process of document verification and first test of alternative respirators upon arrival in the hospital comprises the following steps:

verification of the documents accompanying the respirator:

verification of whether the documents belong to the delivered product

verification of the certificates of the product

verification of the instruction for use

visual inspection:

visually check the quality and marking on the respirator (e.g. conformity mark, notified body number, classification, expiration date)

correct method of packaging (e.g. packaging is dust-proof and correctly sealed)

information on the packaging (e.g. manufacturer, type and duration of use)

face-fitting of the respirator on at least five people

ensure that the nose piece closes properly

visual inspection and/or soft blowing through the nose to identify potential leakage on the side and seals (if unexpected leaks are observed the respirator is rejected)

test for water repellence by dropping a few drops of water on the outside of the respirator (droplets should roll off and should not be absorbed and penetrate the respirator)

test the sealing welds by illuminating with a lamp and carefully pulling the seal apart (if the welds are transparent and the seal simply tears, the respirator is rejected)

cut the respirator open for visual inspection of the filters; broadly, there are two types of respirator:

respirators consisting of three layers: outside moisture-proof, inside polypropylene fibres (coarse-mesh and feels firm), inside comfort layer, and

respirators consisting of three or more layers as above, but the middle layer consists of one or more layers of fine-meshed organic material (high-quality respirators have an inner layer consisting of three layers, so that the respirator comprises at least five layers).

If the expected layers are not recognized or present, the respirator is rejected.

If the documentation verification and the visual inspection meet the requirements, the respirator is subjected to a TIL test to evaluate the full filter efficiency in a test-rig. ‘Full filter’ means that the area of the respirator that functions as a filter is subject to the test, rather than a part or parts of the filter area.

In the cases in which particle tests have to be performed, ten respirators are randomly taken from the received batch. If the first five do not meet the criteria for the full filter test the respirators are rejected for use in the hospital (Figure 1 ). If the first five respirators meet the criteria, another five are tested using the same test procedure. If the second five also meet the criteria, the respirator batch can be released for use. Therefore, before releasing a batch of respirators for use in the hospital, ten respirators are tested.

Figure 1

Flow chart of data acquisition for inward leakage of the respirators.

Test-rig

The lay-out of the test-rig is shown in Figure 2, Figure 3 . The test-rig is laid out to facilitate testing of the full or complete filter of the respirator (Figure 3b), with a channel diameter of 0.16 m. The total length of the test channel is ∼2.30 m, with 1.30 m between the respirator and the open end of the channel, a perspex mould to house the respirator, and a ‘clean’ area of ∼1 m between the respirator and the closed end of the channel. For each respirator brand and type, a dedicated mould is constructed (e.g. Figure 3c). The moulds were constructed by the technical department of the hospital. Figure 3d displays the particle counters (type 3016; Lighthouse worldwide solutions, Fremont, CA, USA). One particle counter measures the ‘dirty’ area before the filter, the other measures the clean area after the filter (Figure 2). The unexpected and immediate situation of shortage of masks forced the hospital and project team to build a test-rig with equipment that was available at short notice. To ensure that the measurements were valid, a premium and CE-certified mask was used as reference. The results of the measurements with this mask demonstrated that the method of measuring was compliant with EN149+A1:2009 [1] (Figure 4 ). Results of all tested masks were compared with the results of the reference mask and the EN149+A1:2009 [1].

Figure 2

Schematic overview of testing respirator efficacy for filtering particles. The set-up includes a metal channel with a total length of 2.30 m and a diameter of 0.16 m (Figure 3). The open end (left) of the tube has free access to the environment. At the closed end (right) a vacuum pump is installed. The green vertical bar represents a mould in which the respirator to be tested is placed (Figure 3c). The particles coming with the flow into the channel are measured at two locations with two separate particle counters (Figure 3d), one before and one after the location of the respirator. Additional to the iso-kinetic measurements system of the particle counters, the sampling for particle counting is performed via a horizontally placed Pitot tube. This ensures that the sampling is in the axial opposite direction of the air flow. The black dots between the mould and channel represent a seal to exclude air leaks into the channel. To exclude the possibility of accumulating static electric charge in the channel wall, the metal channel is grounded to earth. All equipment used in the test-rig was available in the technical department of the hospital.

Figure 3

Test-rig for respirator testing. (a) Overview of test-rig. At the left end of the channel is the open end of the environment; at the right end, the hose to the vacuum pump. (b) Front view of test-rig with a respirator loaded in channel for testing. (c) Example of a respirator in a dedicated mould. (d) Particle counters, dirty and clean side.

Figure 4

Data from 13 tested respirators in the test-rig (Figure 2, Figure 3): averaged values of the test sessions per particle size (Figure 1). In every session the measured particle sizes were 0.3, 0.5, 1.0, 5.0, and 10.0 μm. ‘ref’ stands for reference: the reference values of an A-brand FFP2 respirator. The horizontal magenta lines indicate the maximum particle values for FFP1, FFP2, and FFP3 respirators, according to the standard: 22%, 8%, and 2%, respectively [1]. Because the reference stays within the limit of the FFP2 respirator, this limit is used as the acceptance criterion for the full filter test. Based on these results the respirators D, E, G, F, H, J, and L meet the criteria. The brands and types are not disclosed because it does not add value in the development of the test protocol for respirators.

The air flow is induced by a vacuum pump placed on the closed end of the ‘clean’ side of the channel.

With the dimensions of the test channel the Reynolds number () can be calculated [10]. With the density of air (∼1.29 kg/m3), V the velocity of the air inside the channel (in the order of 0.5 m/s), d the diameter (0.16 m) of the channel and the dynamic viscosity (17.1 × 10−3 Pa·s), this yields a Reynolds number of 6. This is far below the value of 2100 and indicates that in the channel the flow is laminar if the flow is steady and fully developed [10]. The tip of the Pitot tubes in the test-rig is positioned at approximately five times the diameter of the channel from the entrance of the tube to the respirator position (sampling point ‘dirty’ area) and after the respirator position (sampling point ‘clean’ area). With the low flow velocity and a stabilization time of 1 min before particle measurements are started, the flow in the channel is assumed to be a laminar flow. The velocity of the air flow, measured in the axial centre, is in the order 0.5 m/s depending on the load in the channel. The average velocity can then be calculated with , and is in the order of 0.25 m/s [10]. This is in the orders of the flows as specified in the standard EN149+A1:2009 [1].

In the absence of a particle generator or salt-spraying equipment in the hospital, the background load of particles in the test environment was used as load for the filter test. This particle distribution is likely to be similar to that found in a ward of the Catharina Hospital and is therefore a suitable particle load for the test.

Measurements

After stabilization of the flow, five sequential samples are taken simultaneously with both particle counters, each with a sample time of 1 min. In each sample the particles are counted by the particle counters. The measurements are performed using a hand-held Lighthouse 3016 particle counter (Figure 3d). This device distinguishes between particles sized 0.3, 0.5, 1.0, 3.0, 5.0, and 10.0 μm. In a measurement session, five of the same brand and type of respirators are measured (Figure 2). This delivers a total of 25 measurements per brand-type of respirator per session.

All tests with the test-rig are performed with an environmental temperature of 23 ± 2°C and a relative humidity of 36 ± 5%. These values are within the specifications of the standard [1].

In the flow chart for data acquisition on particles (Figure 1) it is specified that the measurement or data-acquisition sessions have to start and end with a zero measurement. In these measurements no respirator is positioned in the test-rig (Figure 2, Figure 3).

Data handling

Because the channel does not leak, the gas composition at the sampling point ‘dirty’ and ‘clean’ are similar. Although two identical and calibrated particle measurement devices are used, a (small) difference in measurement between these devices may occur. To eliminate this systematic difference between the two devices, a correction factor (CF) per particle size is applied:

Using the CFs, the percentage of particles passing the respirator is calculated by:

Furthermore, the CFs are used to judge the change (difference) of particles at the start and at the end of a measurement session. The differences in CFs have to be checked manually because the measured number of particles (not presented here) may lead to an incorrect interpretation. An example of a processed data set is presented in Table I .

Table I

Example of a processed data set from a measurement

Item	0.3 μm	0.5 μm	1 μm	3 μm	5 μm	10 μm
Zero measurement before	0.75	0.92	0.91	0.55	0.58	1
	0.75	0.92	0.96	0.83	0.60	0.60
	0.77	0.95	0.90	0.55	1.18	0.29
	0.77	0.90	0.90	0.53	0.52	0.86
	0.79	0.95	0.89	0.56	0.58	0.40
Respirator P-1	10.59a	4.57	2.17	0	0	0
	10.46a	4.12	1.90	0	0	0
	9.94a	4.20	0.69	0.80	0	0
	10.04a	3.88	1.52	0.91	0	0
	9.60a	4.02	1.87	1.12	0	0
Respirator P-2	10.48a	5.07	2.57	0	0	0
	10.48a	4.22	2.61	0	0	0
	10.76a	4.50	1.71	1.06	0	0
	10.39a	4.68	0.76	0	0	0
	10.90a	5.04	1.58	0	0	0
Respirator P-3	10.01a	4.11	1.85	0	0	0
	9.81a	3.96	1.76	0	0	0
	10.28a	4.76	2.77	0	0	0
	10.09a	4.08	2.31	1.29	0	0
	9.50a	3.93	1.58	0	0	0
Respirator P-4	15.56a	8.13a	4.64	1.12	1.25	0
	14.55a	6.88	4.24	0	0	0
	13.89a	6.84	2.98	1.72	0	0
	13.36a	6.20	3.55	0	0	0
	13.36a	6.47	2.34	0.65	0	51.06a
Respirator P-5	15.45a	10.11a	4.88	1.26	3.18	0
	14.47a	8.88a	4.17	4.45	3.82	Div.0!b
	13.82a	7.45	3.89	5.25	0	0
	14.16a	7.74	2.91	0	0	0
	13.38a	7.55	2.77	1.44	0	0
Zero measurement after	0.80	0.91	0.82	0.64	1	0.38
	0.80	0.90	0.78	0.77	0.41	0.33
	0.81	0.90	0.82	0.91	0.58	0.50
	0.82	0.90	0.89	0.65	0.29	0.75
	0.81	0.90	0.83	0.94	0.22	1.50
Correction factor before	0.76	0.93	0.91	0.61	0.69	0.63
Correction factor after	0.81	0.90	0.83	0.78	0.50	0.69
Difference	–0.05	0.03	0.08	–0.17	0.19	–0.06

In this example the acceptable level of particles is 8%. The values for the zero measurements are ratios (Equation 1). The values for the respirator measurements are percentages (Equation 2).

Values >8% are assessed as fail. In the calculated correction factors the change in air composition can be verified.

Because of the ratio a value may result in Division by zero (Div.0!) when no particles are counted. This was typically observed with the larger particles (5.0 and 10 μm).

Data interpretation

To set a reference for accepting or rejecting a respirator, a CE-marked A-brand FFP2 respirator was selected. Ten respirators were tested according to the test protocol (Figure 1). This set was marked as the ‘reference’. Results of all tested respirators were compared to this reference and to the FFP2 criteria. Additionally, a threshold of 8% particles passing the filter is used for accepting or rejecting a respirator as in the TIL test in the standard EN149+A1:2009 [1].

Release of respirator for use

The results of document verification, visual inspection, and measurements of the respirators are evaluated by the specified departments. Only when the final decision on a respirator is positive is the batch of respirators accepted and released for use in the hospital.

Results

At the time of writing, our verification protocol had been applied to 22 batches of respirators, including the reference mask. Ultimately, seven out of the 21 (33%) batches met the requirements of the Catharina Hospital, and were released for use.

In eight (38%) cases the documentation or visual inspection did not meet our criteria; these respirators were immediately rejected and returned to the supplier. Therefore, the full filter test with the test-rig has been performed on 13 respirator batches according to Figure 1.

Figure 4 shows the data from 13 respirators tested for filter efficiency. For the reference data set (indicated by ‘ref’), the maximum average percentage of particles passing the respirator was 4.99% (range: 3.26–6.88) for 0.3 μm particles, which is well below the maximum TIL of 8% for FFP2 respirators, as specified by the EN149+A1:2009. Figure 4 shows that seven out of 13 (54%) of the respirators have met the criteria of the Catharina Hospital for the full filter test with the test-rig. The brands and types are not disclosed because it does not add value in the development of the test protocol for respirators.

Discussion

The results of our test protocol were that only 33% of the tested respirators met the requirements of the hospital. This result stresses the importance of testing received respirators to ensure the safety of our healthcare workers at this moment in time. Therefore, the protocol has been implemented in the hospital. The protocol will remain in place as long as a shortage of respirators persists.

Although the full filter test in the test-rig is in line with the specification in the standard, it differs in some respects [1]. For example, the flow through the respirator is a constant flow, and whereas, in the standard, breathing is mimicked, the flow applied in the test-rig is higher than the flow of a human being. The constant and higher flow is more challenging than the ‘human breathing’ flow. The environmental conditions during the tests meet the requirements in the standard. However, the breath of a human is specified in the standard at 37°C and the relative humidity at 95%. It can be argued that a moist respirator is more difficult for particles and air to penetrate than a dry respirator. It is therefore assumed that the tests in the test-rig are more challenging for the respirator. To ensure that no pathogens can come through the filter material via the moisture, the masks' instructions for use specify the duration of use. This duration of use may vary between masks. The Dutch authorities recommend a duration of 3 h. In the Catharina Hospital this duration of use is limited to 2 h to increase safety [4].

The number of 10 samples for full filter test in the test-rig is limited. However, the EN149+A1:2009 prescribes a sample size of 10 pieces for the TIL test. This sample size is related to the production batches of ≥100,000. Therefore, a sample size of 10 is justified for in-hospital testing with batches of <100,000 masks.

In conclusion, a robust and easy-to-perform protocol was developed for the in-hospital quality verification of respiratory protective devices. By applying this protocol to all batches of respirators delivered to the Catharina Hospital, safety is assured for the healthcare staff during the COVID-19 crisis.

Acknowledgement

The authors wish to thank A. Fransen, F. Jacobs, T. Kaandorp, and B. van Mil for their constructive and practical contributions during setting up and performing the document verification, visual inspections, the test-rig and measurements.

Conflict of interest statement

None declared.

Funding sources

None.