A comparison of facemask and respirator filtration test methods.

NIOSH published a Federal Register Notice to explore the possibility of incorporating FDA required filtration tests for surgical masks (SMs) in the 42 CFR Part 84 respirator certification process. There have been no published studies comparing the filtration efficiency test methods used for NIOSH certification of N95 filtering facepiece respirators (N95 FFRs) with those used by the FDA for clearance of SMs. To address this issue, filtration efficiencies of "N95 FFRs" including six N95 FFR models and three surgical N95 FFR models, and three SM models were measured using the NIOSH NaCl aerosol test method, and FDA required particulate filtration efficiency (PFE) and bacterial filtration efficiency (BFE) methods, and viral filtration efficiency (VFE) method. Five samples of each model were tested using each method. Both PFE and BFE tests were done using unneutralized particles as per FDA guidance document. PFE was measured using 0.1 µm size polystyrene latex particles and BFE with ∼3.0 µm size particles containing Staphylococcus aureus bacteria. VFE was obtained using ∼3.0 µm size particles containing phiX 174 as the challenge virus and Escherichia coli as the host. Results showed that the efficiencies measured by the NIOSH NaCl method for "N95 FFRs" were from 98.15-99.68% compared to 99.74-99.99% for PFE, 99.62-99.9% for BFE, and 99.8-99.9% for VFE methods. Efficiencies by the NIOSH NaCl method were significantly (p = <0.05) lower than the other methods. SMs showed lower efficiencies (54.72-88.40%) than "N95 FFRs" measured by the NIOSH NaCl method, while PFE, BFE, and VFE methods produced no significant difference. The above results show that the NIOSH NaCl method is relatively conservative and is able to identify poorly performing filtration devices. The higher efficiencies obtained using PFE, BFE and VFE methods show that adding these supplemental particle penetration methods will not improve respirator certification.

Introduction

The Occupational Safety and Health Administration (OSHA) mandates a respiratory protection program in workplaces, which requires the use of National Institute for Occupational Safety and Health (NIOSH)-approved respirators[ to reduce the inhalation of contaminant aerosol particles. In the last decade, respiratory protection has become an increasingly important issue in healthcare as respirators have routinely become an integral component of infection control to protect workers from diseases caused by respiratory pathogens. Healthcare workers are advised to wear NIOSH-certified respirators during close contact with patients with an aerosol-transmitted disease such as tuberculous[ and measles[ and during aerosol generating procedures done on patients with influenza[ and Ebola.[

NIOSH tests the filtration efficiency of particulate-filtering, air-purifying respirators for certification purposes. NIOSH approves N-, R-, and P-series non-powered air-purifying respirators, each at 95, 99, and 99.97% filtration efficiency levels under 42 CFR Part 84.[ N95 filtering facepiece respirators (FFRs) are commonly used in industrial workplaces. To measure the filtration efficiency, N-series FFRs are tested at a flow rate of 85 L/min using a charge neutralized polydisperse sodium chloride (NaCl) aerosol with a count median diameter (CMD) of 0.075 ± 0.02 µm and a geometric standard deviation (GSD) of less than 1.86.[ NIOSH certification testing is considered as more stringent or worst-case method, because of the use of charge neutralized aerosol size close to the most penetrating particle size (MPPS) (∼0.050 µm for N-type respirators) at relatively higher flow rate (face velocity) to produce maximum penetration or conservative filtration efficiency.[

Surgical masks (SMs) are cleared by the Food and Drug Administration (FDA). FDA does not do any testing; but reviews the information supplied by the manufacturers in their 510(k) premarket application. Manufacturers submit test results for fluid resistance, filtration efficiency for polystyrene latex (PSL) and Staphylococcus aureus bacterial aerosol particles, differential pressure and flammability for SMs clearance. These devices are used by healthcare personnel during medical procedures to protect both the patient and the healthcare personnel from the transfer of infectious microorganisms, body fluids and particulate material.[ A SM is used as a physical barrier to body fluids and larger droplets in healthcare activities.

FDA also clears surgical N95 respirators for healthcare purpose. A surgical N95 FFR offers the protection of both an N95 FFR and a SM. These respirators are approved as N95 respirators by NIOSH first and then cleared by the FDA for fluid resistance and flammability properties. Because NIOSH certification of N95 respirators includes filtration efficiency and pressure difference tests, the FDA guidance document states that a NIOSH certification number may be submitted in lieu of the efficiency testing using PFE and BFE. Manufacturers submit only some N95 FFR models for FDA clearance as surgical N95 respirators.

Manufacturers obtain the efficiency data from third-party independent testing laboratories. For PFE testing unneutralized 0.1 µm polystyrene latex particles are used as per FDA guidance document[ at 0.5 to 25 cm/sec face velocities as recommended by the ASTM F2299 standard.[ BFE is measured using unneutralized S. aureus bacteria contained within an aerosol droplet with a mean particle size of 3 ± 0.3 µm diameter at a flow rate of 28.3 L/min as per FDA guidance and ASTM F2101 method.[ Both PFE and BFE are done to measure the performance requirements for materials used in the construction of medical face masks (ASTM F2100-11).[ These tests represent industry practices for characterizing the filtration performance of surgical mask material, not the entire mask. As indicated in the standards, these masks are not recommended for respiratory protection. However, the PFE and BFE for SMs are misconstrued by some as methods for determining filtration efficiencies acceptable for use in testing and clearance of respiratory protection devices. In fact, it is not uncommon for FDA 510(k) applications for surgical N95 FFRs to include BFE and PFE in addition to the NIOSH certification test data.[

One of the many challenges with the PFE and BFE test methods recommended by the FDA are that they are not clearly described in any single document, as shown in Table 1. This table refers to several ASTM standards, but none of the test methods is strictly followed when submitting a 510(k) application to FDA. For example, FDA recommends the use of a whole mask for testing with 0.1 µm unneutralized PSL aerosol particles for measuring PFE as described by the ASTM F1215-89 standard,[ which was withdrawn in 1999 and superseded by ASTM F2299 as recommended separately in ASTM F2100. On the contrary, the ASTM F2299 specifies testing SM material with charge neutralized PSL particles in the size range 0.1–5.0 µm at a face velocity in the 0.5–25 cm/sec range. In the case of BFE testing, the FDA guidance document recommends either the ASTM F2101 standard, Mil-M369454C or modified Greene and Vesley method.[ This further complicates comparisons across methods because, ASTM F2101 testing is done with SM material, whereas, Greene and Vesley method uses the entire mask. The FDA guidance document does not clearly specify testing the entire mask or an area of the mask material for BFE testing.

Table 1.

Comparison of filtration test methods.

Test	Source	Aerosol	Particle	Particle	Particle	Aerosol	Flow Rate	Test	Max	Sample Type
Method	Documents	Type	Size	Charge	Concentration	Detector	(Face Velocity)	Time	Efficiency	(Size)
NIOSH NaCl	42 CFR part 84	NaCl	0.075 µm CMD (GSD <1.86)	Neutralized	<200 mg/m3	Light Scattering photometer	85 L/min (Face velocity varies between respirators)	Maximum penetration	99.999%	Respirator (Entire mask)
FDA-PFE	1) FDA Guidance Document (SM 501(k)112) ASTM F 1215-89 (withdrawn)173) ASTM F2100144) ASTM F299912	Polystyrene latex spheres (FDAGuidance Document)	0.1 µm(FDA Guidance Document)	Unneutralized(FDA Guidance Document)	Generate 107 - 108 particles/m3 and dilute as needed(ASTM F2299)	Optical particle counter(ASTM F2299)	Face velocity- 0.5–25 cm/sec(ASTMF2299)	1-5 minInitial efficiency(ASTM F2299)	99.9%;Increase aerosol concentration to achieve greater efficiencies(ASTM F2299)	Surgical mask (Entire mask)(FDA Guidance Document)
ASTM-PFE	ASTM F229912	Latex spheres	0.1 to 5.0 µm (Mono-disperse aerosol; MPS	Neutralized	Generate 107 - 108 particles/m3 and dilute as needed	Optical particle counter	Face velocity- 0.5–25 cm/sec	1–5 min Initial efficiency	99.9%, Increase aerosol concentration to achieve greater efficiencies	Surgical mask (Mask material) (50–150 mm diameter circle)
FDA-BFE	1) FDA Guidance Document (SM 501(k))112) ASTM F2100143) ASTM F210113	Staphylo-coccus aureus (ASTM F2101)	3.0 ± 0.3 µm(MPS)(ASTM F2101)	Undefined(ASTM F2101)	2200 ± 500 viable particles per test (ASTM F2101)	Six-Stage Viable Particle Cascade Impactor (ASTMF2101)	28.3 L/min(Face velocity not defined) (ASTM F2101)	2 min aerosol exposure per test (ASTM 2101)	99.9% (ASTM F2101)	Surgical mask (Entire mask) (FDA Guidance Document)
ASTM-BFE	ASTM F210113	Staphylo-coccus aureus bacteria	3.0 ± 0.3 µm MPS	Undefined	2200 ± 500 viable particles per test	Six-Stage Viable Particle Cascade Impactor	28.3 L/min (Face velocity not defined)	2 min aerosol exposure per test	99.9%	Surgical mask (Mask material) (Test material area not defined; but should be reported)
VFE	Not a Standard test method	PhiX174 virus	3.0 ± 0.3 µm MPS (adapted from ASTM F2101)	Undefined	1700 – 2000 plaque forming units per test (adapted from ASTM F2101)	Six-Stage Viable Particle Cascade Impactor (adapted from ASTM F2101)	28.3 L/min (Face velocity < 4.7 cm/sec) (per Nelson Labs)	Not Defined	99.9% (adapted from ASTM F2101)	Entire mask or 10 × 10 cm mask material (per Nelson Labs)

The FDA guidance document and the ASTM standards overlook important factors necessary for measuring the filtration efficiency of SMs. The filtration efficiency of fibrous filter materials is controlled by factors including, aerosol charge, particle size distribution, face velocity and filter material charge.[ First, charge neutralized aerosol is known to produce maximum penetration (low efficiency).[ FDA refers the ASTM F1215-89 method (presently ASTM F1899) for PFE, but recommends the use of unneutralized PSL aerosol for testing. In the case of BFE, neither the FDA guidance document nor the ASTM F2101 has specified charge neutralization of the S. aureus aerosol. However, the aerosol used in the testing is assumed to be unneutralized. The use of unneutralized aerosol particles for filtration is known to overestimate filtration efficiency. Second, face velocity is another critical factor for filtration efficiency measurement.[ For the PFE testing, FDA guidance document recommends the ASTM F1215-89 method (presently ASTM F1899), which specifies testing at 5–25 cm/sec velocity. Filtration efficiencies at 5 and 25 cm/sec are expected to be different. The variation in PFE results may not be comparable between SMs. FDA guidance document recommends the ASTM F2101 method as one of the BFE testing methods. As per ASTM F2101 method, BFE is tested at 28.3 L/min flow rate, while the area of the filter material is not specified. Testing filter materials with varying surface areas is likely to produce different efficiency values. Above all, the particle size distribution is an important factor when measuring filtration efficiency. The PFE and BFE tests use particles of 0.1 µm (monodisperse) and 3 µm mean particle size (MPS), respectively, with no rationale behind those decisions. Many studies show that inert and biological aerosols are captured by similar mechanisms.[ The filtration efficiencies are different for various size particles. Particles closer to the most penetrating particle size should be considered for measuring a conservative filtration efficiency. The lack of specific test criteria to obtain a conservative filtration efficiency may produce inconsistent results. The wide variation in test conditions with the entire mask or a portion of mask material at different velocities using different size unneutralized aerosol particles is likely to produce a range of efficiencies for different masks, which is difficult to compare.

Individuals who are not well acquainted with particle filtration mechanisms, frequently request additional evidence on inert vs biological aerosol penetration through respiratory devices. Thus, there is still interest in evaluating respirator filter efficiency using test methods involving a biological aerosol. However, numerous studies show that fibrous filters capture inert and biological aerosols by similar mechanisms.[ Aerosol particle filtration is dependent on physical characteristics including particle size, shape, density, charge status, filter media charge, and face velocity.]8,29] Whether the particle is “living” or “infectious” plays no role in how well it will be collected by a filter.[ Once a particle is collected it will remain attached by electrostatic and van der Waals' forces,[ and although biological organisms have no capacity for moving through a filter on their own, a particle may be dislodged at higher air flow rates.[ Because of this information, Brosseau and Shaffer[ indicated that it not is necessary to test a respirator filter with a biological aerosol, but rather to focus on “worse-case” type test conditions. When properly selected and used, respirators tested using these types of filter tests should provide expected levels of protection against all types of workplace aerosols.

NIOSH published a Federal Register notice on the need for incorporating additional requirements and tests in the 42 CFR Part 84 respirator approval process.[ NIOSH requested evidence related to the performance of NIOSH-approved products, which are not FDA-cleared as medical devices, against alternative test methods including PFE and BFE and the exposure levels of aerosols in healthcare facilities. NIOSH also sought comparative results for testing with supplemental standard methods vs. test results obtained using the NIOSH NaCl test method. Comments to the docket[ revealed that no data exist on the comparative filtration efficiency of non-FDA cleared N95 FFRs and that healthcare workers also use non-FDA cleared, NIOSH-approved N95 FFRs (N95 FFRs) for protection against infectious aerosols. Surprisingly, no studies have been published directly comparing filtration efficiency test methods used for NIOSH certification of particulate respirators with those used for FDA clearance of SMs and surgical N95 FFRs.

To address this issue, six NIOSH-approved N95 FFR models, three FDA cleared and NIOSH-approved surgical N95 FFR models, and three SM models were evaluated for their filtration efficiency using NIOSH NaCl, PFE, BFE, and virus filtration efficiency (VFE) methods. We hypothesize that because of the test parameters (face velocity, charge neutralization, particle size, etc.), results (filter efficiencies) obtained using the NIOSH NaCl test conditions will be lower (“more conservative”) than those from the BFE, VFE, and PFE tests (Hypothesis #1). Also, we hypothesize that there will be no systematic difference in measured filtration efficiency for BFE and VFE between FFRs that are FDA-cleared and NIOSH-certified FFRs (i.e., surgical N95 FFRs), and FFRs that are only NIOSH-certified (Hypothesis #2).

Materials and methods

Test materials

Six NIOSH-approved N95 FFR models, three surgical N95 FFR models and three SM models were chosen from the United States Strategic National Stockpile or from respirator manufacturers known to have significant market share. The manufacturers and models in parentheses are: N95 FFRs - 3M (Model 8210), 3M (Model 9210), Moldex (Model 2200), Kimberly-Clark (Model 62126), Sperian-Willson (Model SAF-T-FIT), and US Safety (N95B240); surgical N95 respirators - 3M (Model 1860), 3M (Model 1870) and Kimberly-Clark (Model 46727); SMs - 3M (Model 1820), Kimberly-Clark (Model 47107) and Precept (15320). The N95 FFRs were labeled randomly as A, B, C, D, E, and F, the surgical N95 FFRs as G, H, and I, and the SMs as J, K, and L. None of the N95 FFRs or surgical N95 FFRs had an exhalation valve. Surgical N95 FFR models H and I were specifically chosen, because they appear to be identical to two N95 FFR models (B and E).

Area of test materials and face velocity or test material area, face velocity, and configuration

Table 2 shows the area, face velocity and configuration of masks used in the NIOSH NaCl, PFE, BFE, and VFE methods. Entire N95 FFR models (except model E) and surgical N95 FFR models (except model I) were tested in all four test methods. The area of other masks tested were different, because the nature of the fabric materials and the construction of some models did not permit the entire mask be tested. Also, it was difficult to fit some masks into the normal sample holders used for some test methods. The surface area of a typical FFR was approximately 150 cm2 as measured manually. Based on the area of the entire FFR and flow rates used for testing, face velocities 9.3 and 3.1 cm/sec were obtained for the NIOSH NaCl method, and for the other (PFE, BFE, and VFE) methods, respectively. SM models were not tested in their entirety, but samples of 100 cm2, 90 cm2, 45 cm2 and 45 cm2 areas were tested for the NIOSH NaCl, PFE, BFE and VFE measurements, corresponding to 14.2, 5.2, 10.5 and 10.5 cm/sec velocities, respectively. Filtration efficiency tests for N95 FFRs, surgical N95 FFRs and SMs were performed by a third party independent (TPI) laboratory. Five samples of each model were tested for filter penetration separately using the NIOSH NaCl, PFE, BFE, and VFE methods. From the mean penetration value, the percentage efficiency was calculated for each model.

Table 2.

Face velocity, and surface area and configuration of masks used in the NIOSH NaCl, PFE, BFE, and VFE test methods.

		Face Velocity (cm/sec)
Type	Model	NIOSH NaCl	PFE	BFE	VFE
N95 FFR	A, B, C, D, F	9.3(150 cm2)	3.1(150 cm2)	3.1(150 cm2)	3.1 (150 cm2)
	E	9.3(150 cm2)	3.1(150 cm2)	10.5(45 cm2)	10.5(45 cm2)
Surgical N95 FFR	G, H	9.3(150 cm2)	3.1(150 cm2)	3.1(150 cm2)	3.1(150 cm2)
	I	9.3(150 cm2)	3.1(150 cm2)	10.5(45 cm2)	10.5(45 cm2)
Surgical Mask	J, K, L	14.2(100 cm2)	5.2(90 cm2)	10.5(45 cm2)	10.5(45 cm2)

Numbers in parentheses show surface area of materials tested.

No shading - Entire mask (150 cm2); Light shading - Mask material (100 cm2)

Medium shading - Mask material (90 cm2); Dark shading - Mask material (45 cm2)

NIOSH NaCl filtration efficiency

Filter penetration for all devices was measured using the NIOSH NaCl aerosol method employed for certification of particulate respirators using an Automated Filter Tester (CERTITEST®, Model 8130, TSI, Inc., St. Paul, Minnesota). The samples were pre-conditioned at 85 ± 5% relative humidity and 38 ± 2.5°C for 25 ± 1 hr prior to measuring filter penetration. A 2% (wt/vol) NaCl solution was aerosolized, charge neutralized and then passed through the convex side of a test sample properly sealed and placed into a filter holder. The particle size ranges from 0.022–0.259 µm with a count median diameter of 0.075 ± 0.020 µm and a geometric standard deviation (GSD) of less than 1.86.[ The concentrations of NaCl aerosol upstream and downstream of the sample were measured at 85 L/min flow rate. The full NIOSH certification test typically takes approximately 90–100 min to load 200 mg of NaCl aerosol onto the respirator. In this study, we used an abbreviated method in which testing was stopped when the sample reached its maximum penetration level rather that continuing until the respirator was fully loaded with aerosol. Thus, the time to reach maximum penetration was different between different masks, but was typically less than 15 min. A previous study in our laboratory found that efficiencies obtained by measuring initial penetration (average of the first minute) of N95 FFRs were comparable to those done at maximum penetration at full loading conditions.[ The equipment was programmed to give the percentage filter penetration as shown below:

Penetration of each test sample was measured for 5 min or more time until maximum penetration was reached. For convenience and efficiency, doing the full loading test as done by NIOSH according to 42 CFR Part 84 during certification testing was not done in these experiments. From the percentage penetration, the efficiency was calculated as shown below:

Particulate filtration efficiency (PFE)

PFE of the different devices was measured using unneutralized 0.1 µm PSL particles as per the FDA guidance document.[ PFE testing was done using entire N95 FFRs and surgical N95 FFRs, and 90 cm2 surgical mask material. The test velocity was within the range 1–25 cm/sec as described in the ASTM 2299 method.[ PSL particles were suspended in water and the aerosol was generated using a particle generator (Model PG-100) (Particle Measuring Systems (PMS), Boulder, CO). The particle generator can be adjusted to generate the desired concentration of particles for testing, which are enumerated using the particle counter downstream of the test sample. The aerosol was passed through a drying chamber, diluted to the required concentration (10,000–15,000 particles per cubic foot) using HEPA filtered air, and then passed through the convex side of a test sample properly sealed and placed into a filter holder. The particles were not charge neutralized for testing, per FDA guidance.[ The test samples were preconditioned at 30–50% relative humidity (RH) at 21 ± 3°C, prior to testing.[ The concentrations of PSL aerosol upstream and downstream of the respirator were measured using a Model LASAIR II-110 Laser Particle Counter (PMS) at 28.3 L/min flow rate. The upstream and downstream counts were not measured simultaneously, because the equipment measures either upstream or downstream at any time. Upstream count was measured before and after measuring downstream count for each test material. Both upstream and downstream counts were measured for 1 min, 3 times, sequentially. PFE was calculated from the average of six upstream counts (C) and the average of three downstream counts (T) as shown below:

where

PFE results range from 1–99.99% as these values are the low (<1) and high (>99.99) detection limits. Accurate data for surgical mask K could not be obtained because of excessive particulate shedding detected by the particle counter.

Bacterial filtration efficiency (BFE)

The BFE of the filter devices was measured as described by the ASTM F2101 method. Penetration was measured using the bacteria S. aureus as the challenge organism. A suspension of S. aureus was aerosolized using a nebulizer to give a challenge level of 1700–2700 colony-forming units (CFU) per test as specified by the ASTM F2101 standard. The bacterial aerosol is a water droplet containing the bacteria and not an individual bacterial particle. The particles were not charge neutralized for testing. The test samples were preconditioned for 4 hr at 21±3°C and 85±5% RH, prior to testing. The aerosol sample was drawn through a test sample clamped into the top of a 6-stage Andersen sampler with agar plates for collection of the bacteria particles at a flow rate of 28.3 L/min for 1 min. The design of 6-stage Andersen sampler is based on the human respiratory tract, where all airborne particles greater than 0.65 µm are classified aerodynamically. The flow rate of 28.3 L/min is similar to human breathing flow rate to obtain deposition of particles in different stages of the Andersen sampler. A positive control without a test filter sample clamped into the system was used to determine the number of viable particles being used in each test. A negative control with no bacteria in the airstream was performed to determine the background challenge in the glass aerosol chamber prior to testing. In case of contamination, the testing system was cleaned thoroughly to reduce negative control CFU. The residual negative control (<1%) was subtracted from the test sample CFU. The positive control result was used to obtain the MPS of the test aerosol. The MPS was calculated from the particle sizes (P1-P6) and respective CFU counts (C1-C6) in the Andersen 6-stage impactor (Online supplementary Table 1), as shown below:

The particle sizes P1–P6 represent 50% cut-off diameter (Dp50) for stages 1–6, respectively, of the Andersen Sampler. The MPS was kept at 3.0 ± 0.3 µm for testing the filtration efficiency. The filtration efficiency was calculated from the number of positive control CFU and test sample CFU the as shown below: where

BFE results range from 1–99.9% as these values are the low (<1) and high (>99.9) detection limits which are based on the test parameters used and how the calculations are performed.

Viral filtration efficiency (VFE)

VFE is not recognized as a standard test method, but has been adapted by Nelson Laboratories[ from the ASTM F2101 method.[ It was included in this study because it is sometimes used by manufacturers in their marketing literature[ and in FDA 510(k) applications for N95 FFRs with an antimicrobial/antiviral agent (16). Penetration was measured using the bacteriophage phiX174as the challenge virus and Escherichia coli bacteria as the host. A suspension of phiX174 was aerosolized in a nebulizer and each test was performed with a challenge level of 1700-2700 plaque-forming units (PFU) with a MPS of 3.0 ± 0.3 µm for 2 min. The virus aerosol is a water droplet containing virus and not an individual virus particle. The MPS was calculated as shown for BFE method. The unneutralized aerosol sample was drawn through a test sample clamped into the top of a 6-stage Andersen sampler with agar plates inoculated with E. coli for collection. The flow rate was maintained at 28.3 L/min. The PFUs represent the number of viral aerosol particles or droplets. The total number of viral aerosols for test samples, and positive controls without a test sample, were obtained as described for the BFE method. A negative control with no virus in the airstream was performed to determine the background challenge in the glass aerosol chamber prior to testing. The filtration efficiency was calculated using the PFU similar to the method described for BFE. Similar to BFE, VFE results range from 1 to 99.9% as these values are the low (<1) and high (>99.9) detection limits which are based on the test parameters used and how the calculations are performed.

Table 3.

Filtration efficiencies for N95 FFR, surgical N95 FFR, and surgical mask models using the NIOSH NaCl, PFE, BFE, and VFE test methods.

		Efficiency (%)
			NIOSH NaCl		PFE		BFE		VFE
Type	Model	Sample Size	Mean	SD	Mean	SD	Mean	SD	Mean	SD
N95 FFR	A	5	98.87	0.20	99.88	0.16	99.90	0.00	99.90	0.00
N95 FFR	B	5	99.66	0.03	99.99	0.02	99.90	0.00	99.90	0.00
N95 FFR	C	5	98.15	0.21	99.74	0.11	99.62	0.24	99.80	0.12
N95 FFR	D	5	99.32	0.13	99.93	0.07	99.90	0.00	99.90	0.00
N95 FFR	E	5	99.31	0.18	99.94	0.05	99.90	0.00	99.90	0.00
N95 FFR	F	5	99.33	0.07	99.86	0.29	99.90	0.00	99.90	0.00
Surgical N95	G	5	98.93	0.20	99.97	0.02	99.80	0.12	99.88	0.04
Surgical N95	H	5	99.68	0.24	99.98	0.03	99.86	0.09	99.88	0.04
Surgical N95	I	5	98.27	0.37	99.84	0.05	99.90	0.00	99.90	0.00
SM	Ja	5	54.72	1.88	98.26	0.09	98.12	0.31	97.12	0.34
SM	Ka	5	88.40	1.48	—	—	99.80	0.10	99.88	0.04
SM	La	5	63.12	0.91	98.66	0.02	97.48	0.63	97.72	0.36

aSignificantly (p = <0.05) different from N95 FFRs and surgical N95 FFRs when tested using the NIOSH NaCl method.

Data analysis

Correlation coefficients for the filtration efficiency comparison between the test methods were obtained using SigmaPlot version 11 (Aspire Software International, Ashburn, VA). The median efficiency values of respirators for the test methods were compared using the Mann-Whitney Rank Sum test and Shapiro-Wilk normality test. All pair-wise multiple comparisons were done using Tukey Test. Similar statistical analysis was done for comparing the efficiency data for the different groups of devices. P values <0.05 were considered significant.

Results

Table 3 shows the mean and standard deviation (SD) percentage of filtration efficiency values for NIOSH-approved N95 FFR, FDA cleared surgical N95 FFR and SM models using NIOSH NaCl, BFE, VFE, and PFE methods. All six N95 FFR models showed >98.15% filtration efficiency using the NIOSH NaCl method. Surgical N95 FFRs are NIOSH-approved N95 FFRs for filtration efficiency. As expected, the efficiency levels (>98.27%) obtained for the three surgical N95 FFR models were similar to those (>98.1%) obtained for the six N95 FFR models. Five N95 FFR models (A, B, C, D, and F) and two surgical N95 FFRs models (G and H) were tested using entire masks, and these models were considered as “N95 FFRs” for efficiency comparison between the different methods. Figure 1 shows a graphical comparison of NIOSH NaCl method with PFE, BFE, and VFE methods for “N95 FFRs” tested, using the mean efficiency data in Table 3. The filtration efficiencies ranged from 98.15 to 99.68% for NIOSH NaCl method compared to 99.74–99.99% for PFE, 99.62–99.9% for BFE and 99.8–99.9% for VFE methods. Results showed that the median efficiency for “N95 FFR” models using the NIOSH NaCl method was significantly (p = <0.05) smaller than the efficiency values by the PFE, BFE and VFE methods.

Figure 1.

Comparison for filtration efficiencies measured using NIOSH NaCl method (NIOSH NaCl, open bars) with particle filtration method (PFE, ascending hatched bars), bacterial filtration efficiency (BFE, descending hatched bars), and viral filtration efficiency (VFE, cross-hatched bars) methods for N95 FFR models (A, B, C, D, and F) and surgical N95 FFR models (G and H). Five samples of each model were tested by the different methods. Error bars represent 1 standard deviation. *Significantly different from PFE, BFE, and VFE.

Efficiencies for the “N95 FFRs” models measured using the NIOSH NaCl method were also compared with the efficiencies for three SM models (J, K, and L). SMs showed NIOSH NaCl efficiencies ranging from 54.74–88.4% and the mean value was significantly (p = <0.05) lower than the mean efficiency value for “N95 FFRs” (Table 3). Efficiencies measured by the NIOSH NaCl method for SMs were compared with the values by the other methods. SMs showed PFE ranging from 99.74–99.99%, from 99.62–99.9% for BFE and 99.8–99.9% for VFE. The mean efficiency value by the NIOSH NaCl method was significantly (p = <0.05) lower than the values by the PFE, BFE, and VFE methods (Figure 2).

Figure 2.

Comparison for filtration efficiencies measured using NIOSH NaCl method (NIOSH NaCl, open bars) with particle filtration method (PFE, ascending hatched bars), bacterial filtration efficiency (BFE, descending hatched bars), and viral filtration efficiency (VFE, cross-hatched bars) methods for surgical mask models (J, K, and L). Five samples of each model were tested by the different methods. Error bars represent 1 standard deviation. *Significantly different from PFE, BFE, and VFE.

The efficiencies of “N95 FFRs” by the NIOSH NaCl method were correlated with the efficiencies obtained by PFE, BFE, and VFE methods. The comparison of NIOSH NaCl vs. PFE (r = 0.828) as well as NIOSH NaCl vs BFE (r = 0.819) showed some correlation (Figures 3A and 3B, respectively). In general, the models that showed higher efficiencies by NIOSH NaCl method also had higher efficiencies by PFE method. One FFR model (C) had lower efficiency for both NIOSH NaCl, and PFE and BFE methods. The comparison of NIOSH NaCl vs. VFE showed a slightly lower correlation coefficient (r = 0.779) (Figure 3C).

Figure 3.

Correlation of filtration efficiency for “N95 FFRs” as determined by the NIOSH NaCl aerosol method used for NIOSH certification process with FDA required particle filtration efficiency (PFE) (top) and bacterial filtration efficiency (BFE) (middle), and viral filtration efficiency (VFE) (bottom). Straight line shows the best fit for the NIOSH NaCl method and other methods. Five samples of each model were tested by the different methods.

Discussion

To address Hypothesis #1, the four filtration test methods were compared. For “N95 FFRs”, the efficiencies measured by the NIOSH NaCl method were lower than the PFE, BFE, and VFE methods. The NIOSH NaCl method is clearly more conservative compared to the other methods. This was validated by comparing the filtration efficiency obtained by the NIOSH NaCl method with the other methods. The NIOSH NaCl method yielded significantly (p = <0.001) lower efficiency levels than the PFE method for FFRs. This can be explained by the difference in the test conditions including, challenge aerosol size, charge status, and test velocity used between the two methods. Aerosol size is a critical factor for filtration efficiency measurement. The CMD (∼0.075 µm) of NIOSH NaCl aerosols used for NIOSH certification is much closer to the MPPS (0.05 µm) than the 0.1 µm size of the PSL particles used in the PFE test method. Penetration increases as the particle size approaches the MPPS. The closer a particle is to the MPPS, the higher the penetration will be.[ Second, the charge neutralized NaCl particles are expected to show lower filtration efficiency (or higher penetration) than the unneutralized PSL particles. The charge neutralization of challenge aerosol has been shown to increase the penetration (decreased efficiency) of particles.[ The higher face velocity (9.3 cm/sec) used in the NIOSH NaCl method is another factor that contributes to the lower filtration efficiencies in comparison to the values obtained by the PFE method at a lower face velocity (3.1 cm/sec). Overall, the results show that the filtration efficiencies measured using the NIOSH NaCl method are more conservative than the values obtained by the PFE method.

Similarly, comparisons between NIOSH NaCl method vs. BFE and VFE methods showed significantly (p = <0.001) lower filter efficiencies for NIOSH NaCl method than for BFE and VFE methods. The results can be explained by differences among the test methods. Both the BFE and VFE methods use ∼3 µm diameter size particles containing S. aureus and phiX174 bacteriophage, respectively, at a flow rate of 28.3 L/min (face velocity 3.1 cm/sec). The larger diameter size (∼3 µm) particles containing S. aureus and phiX174 bacteriophage is far removed from the MPPS (0.05 µm) of N95 FFR models and is expected to produce lower penetration or higher efficiencies than the values obtained by the NIOSH NaCl method. It should be noted that the VFE method used ∼3 µm droplets, even though the original size of phiX174 is 0.027 µm. Both, BFE and VFE methods use non-neutralized aerosols at 3.1 cm/sec velocity, which would likely produce relatively higher efficiency values. The test conditions including unneutralized larger aerosol size and lower velocity used in the BFE and VFE methods are relatively less challenging than those conditions used in the NIOSH NaCl method, and yielded higher efficiencies, as expected. The results from the study show that the NIOSH NaCl method is more challenging and produces lower efficiencies than the PFE, BFE and VFE methods.

Because these four methods measured the efficiencies of the same “N95 FFR” models, a relationship between these methods is possible, despite the differences between the test methods in terms of particle size, charge neutralization, and face velocities. The correlations between NIOSH NaCl (r = 0.828) vs. PFE, NIOSH NaCl vs. BFE (r = 0.819), and NIOSH NaCl vs. VFE (r = 0.779) indicate some association between the NIOSH NaCl and other methods. The lower correlations between the NIOSH NaCl method and other methods may partly be explained by the differences in the inherent measurement precision of the various filtration efficiency tests methods.

A more challenging test method can identify poorly performing products For example, all four methods showed efficiencies >98.15% for “N95 FFR” models. In the case of SM models, however, the NIOSH NaCl method showed lower efficiency values (54.72–88.40%) than the other methods (97.12–99.88%). The results are consistent with those reported previously.[ In that study, the efficiencies ranged from 10–96% for the nine SM models tested using the NIOSH NaCl method. As expected, the results show that the NIOSH NaCl method is more sensitive compared to the other methods. The lower efficiencies for SMs than for FFRs may partly be due to the higher velocity (14.1 vs. 9.3 cm/sec) used for testing. However, previous studies with approximately similar size FFRs and SMs showed significantly higher penetrations or lower efficiencies for SMs.[ Products with poor or mediocre filtration performance such as the SM models in this study can be consistently identified using only a more challenging method such as the NIOSH NaCl method used for NIOSH certification of particulate respirators. This confirms earlier reports from other laboratories,[ and ours on SMs,[ non-NIOSH certified dust masks,[ and cloth masks.[

The filtration efficiencies obtained by the different methods in the study may have implications for respirator use in healthcare environments. The results showed that the NIOSH NaCl aerosol method used for NIOSH certification is more challenging than the PFE, BFE, and VFE methods. The efficiencies measured using different methods showed that the NIOSH NaCl method is more conservative than the other methods. This indicates that there is no advantage or added benefit to be obtained by incorporating PFE, BFE, or VFE methods in the 42 CFR Part 84 certification process.

In general, both BFE and VFE showed similar results for “N95 FFRs.” The two methods also produced similar results for SMs. This can be explained by the use of similar diameter (3 µm) unneutralized aerosol. The BFE and VFE results for “N95 FFRs” did not show any significant difference indicating that VFE method is not different from BFE method. SMs also showed no difference in BFE and VFE values. VFE could be differentiated from BFE and made more challenging if testing were done using a method to create a smaller aerosol size near the MPPS. Hogan et al. reported that aerosol size distribution is controlled by the properties of the aerosolized liquid and the method of aerosolization, not by the physical size of the virus.[ For example, in one study, MS2 virus aerosols (∼0.05 µm) showed higher penetrations (lower efficiency) for N95 FFRs.[ Another study generated an aerosol CMD of ∼0.1 µm using viable H1N1 influenza to challenge N95 FFRs.[ Thus, if there was interest, more challenging VFE methods could be developed by a standards development organization, although a modified VFE with an aerosol challenge in the MPPS range would still be considered unnecessary for determining FFR efficacy as noted previously.[ The VFE method used in this study is adapted from the ASTM F2101 standard and is not being recognized by standard organizations. VFE method used in this study should not be considered useful for characterizing filter material efficiency. Unfortunately, some manufacturers report VFE for SMs to make marketing claims and it has been used in FDA 510(k) applications.

Surgical N95 FFR models H and I were specifically included in this study to address hypothesis #2 because they appear to be identical to two non-FDA cleared N95 FFR models (B and E). The models I and E are both flat folding respirators from the same manufacturer and visually appear to be identical except for color (one is orange and the other is white). Similarly, the models H and B are identical in appearance, except for the labeling and packaging. As shown in Table 3, both pairs exhibited similar filter efficiencies. There was no difference in measured filtration efficiency for BFE and VFE between FFRs that are FDA-cleared (H and I) and NIOSH-certified FFRs and FFRs that are only NIOSH-certified (B and E). Thus, additional tests involving a bioaerosol (e.g., BFE and VFE) included in some FDA 510(k) applications do not add any new information useful to a regulatory or certification agency. Furthermore, this data suggests that non-FDA cleared N95 FFRs would be a viable option for respiratory protection during surgical N95 respirator shortages for situations where fluid resistance[ and flammability resistance may not be necessary. The increased use of respirators in healthcare facilities can result in a shortage during a pandemic or an outbreak involving a respiratory pathogen.[ Moreover, N95 FFRs are generally less expensive than the surgical N95 FFRs. Manufacturers would prefer to get NIOSH certification and FDA clearance in a single-step process and use of a single filtration test by both agencies would be a positive step towards this objective.

Conclusions

The filtration efficiency for six models of NIOSH-approved non-FDA cleared N95 FFRs, three models of surgical N95 FFRs, and three models of SMs tested in the study showed that the NIOSH NaCl test method is more conservative and showed significantly lower efficiencies than the than the PFE, BFE, and VFE methods. Furthermore, PFE, BFE, and VFE suffer from the lack of precision and lack of well-defined and documented testing protocols. We conclude that addition of supplemental particle penetration methods such as PFE, BFE, and VFE to the requirements described in 42 CFR Part 84 will not provide any improvement in the current 42 CFR Part 84 respirator certification process.