Evaluating the safety and effectiveness of novel personal protective equipment during the COVID‐19 pandemic

Novel PPE, such as 3D printed face shields, must be compliant with regulatory requirements and a clinical evaluation protocol should be developed

Personal protective equipment (PPE) is critical in protecting hospital staff during the treatment of patients throughout the coronavirus disease 2019 (COVID‐19) pandemic. The increased usage of PPE worldwide, together with threatened manufacturing capability and disrupted supply chains, has resulted in reduced supplies of PPE reaching frontline workers, forcing the risk‐averse health care system to investigate alternative pathways for procuring PPE. Clinicians have sourced PPE from industrial suppliers and hardware stores.1 Reports of community groups reaching out to clinicians and hospitals, offering design and 3D printing capabilities to supply PPE, particularly face shields, are also widespread.2,3 In addition, health services are purchasing PPE from local or international suppliers, such as the South Australia stockpile of N95 masks,4 with lack of certainty around the compliance with the Therapeutic Goods Administration (TGA) regulations.

Banning novel PPE is reasonable when supply chains are robust; however, when they are strained, using these products becomes a real consideration for health services. In September 2020, the TGA released advice on three supply levels of PPE (standard, contingency and crisis), with flexibility around compliance in crisis situations.5 Guidance is needed to approach this situation with evidence and regard for the current regulatory environment. Rigorous assessment of PPE is critical, particularly given the high rate of COVID‐19 infection among health care workers globally and locally, including clusters in Melbourne, Victoria, and Ipswich, Queensland, possibly contracted in hospital tea rooms or when doffing PPE.6 Health services have little expertise or experience in assessing novel PPE. This article outlines the current issues and a suggested approach for managing novel PPE in a health care setting, illustrated by a case study. This emerging area requires significant thought and consistency to ensure safety of staff and consumers.

PPE products intended for use in Australian clinical health care settings meet the definition of a medical device if their intended use is for the prevention of transmission of disease between people.2 For example, a face shield used by a nurse performing COVID‐19 testing is considered a medical device, but the same shield used by a cleaner in a hospital kitchen is not. Medical devices are classified according to the level of risk they may pose to health care workers or patients. PPE generally falls in the lowest risk category of Class I non‐sterile, non‐measuring medical devices. PPE items require online listing on the TGA’s Australian Register of Therapeutic Goods by their legal manufacturer and compliance with the Essential Principles for medical devices. PPE items must:

• not compromise health and safety;
• be designed and constructed to conform to safety principles;
• be suitable for the intended purpose;
• provide long term safety;
• not be adversely affected by transport or storage; and
• provide benefits that outweigh any side effects.7

The PPE currently used in the Australian health care setting is listed in Box 1 with TGA requirements. Clinical evaluation of devices is necessary to demonstrate compliance with the fourth principle (provide long term safety). We use a case study to illustrate the approach taken by a health service to evaluate face shields.

Case study: 3D printing and evaluating face shields during crisis supply levels

Face shields are a form of PPE that protects the facial area and associated mucous membranes (eyes, nose, mouth) from splashes and sprays of bodily fluids. They form part of recommended PPE for COVID‐19.8 The efficacy of face shields is poorly characterised, but literature indicates they can reduce exposure to larger particles and contamination of respirator masks.9

We selected two open source, 3D printable designs based on collated feedback from the Australian COVID SOS interest group (https://twitter.com/covidsosaus), a collaboration of clinicians and engineers advocating for the needs of front line clinicians and providing stop‐gap solutions. At the time of our investigation in May 2020, there was no recommended standardised testing method for face shields in a clinical context (Box 1). To complete our assessment, we reviewed the international literature,9,10 and conducted a droplet protection efficacy evaluation of these face shield frames.

The two face shield headband designs were fitted with different lengths of clear visors (200 μ PVC clear binder cover) (Supporting Information, Appendix 1 online) and compared as a fixed unit to a disposable, commercial shield. The Melbourne School of Design, version 1 (MSD) face shield was selected for its light weight and ease of manufacturing using either 3D printing or laser cutting. The Prusa RC3 design (Prusa Research), endorsed by the Ministry of Health of the Czech Republic, has been widely disseminated as one of the first fully open source face shield designs.11

Methods to evaluate the safety and performance of face shields include cough simulators and spray bottles.9,10 The evaluation was performed using a simulator manikin set up to excrete 10 mL of fluid particles per spray, thereby mimicking a cough spray. Food colouring simulated bodily fluids.

As shown in Box 2, four static, reproducible, standing positions were marked on the floor. These reflected a range of health care procedures, such as the position assumed by an airway doctor standing behind and bending over the patient (position 1), an airway assistant (position 2), a theatre staff member standing on the side of the bed (position 3), and another staff member at the end of the bed (position 4). Five participants were fitted with scrubs, head and shoe covers, a surgical gown and one of five face shields (Supporting Information, Appendix 1 online), and were photographed after a simulated cough in each position. Participants washed their face with soap and water and donned a clean shield in between each position. The study obtained ethical clearance (Royal Brisbane and Women’s Hospital Human Research Ethics Committee: LNR/2020/QRBW/64373).

Results

The high resolution photographs (Supporting Information, Appendix 2 online) were independently assessed by ten reviewers (administrators who could not differentiate a commercial from a prototype shield). The pass criterion was no visible contamination (green droplets) on the face and forehead once the shield was removed.

Our results (Supporting Information, Appendix 3 online) showed that Prusa RC3 and MSD headbands, when used with a long visor, provide a level of physical protection against droplets comparable to commercial products. As such, they were deemed an acceptable alternative during crisis supply levels of face shields. Short visors are not recommended.

Our evaluation method provides valuable qualitative simulation testing before clinician evaluation in a low risk clinical environment; and may be considered a method of providing assurance to health care workers about the quality checks completed before local distribution of novel PPE. A week after the simulation testing, clinical acceptance was evaluated by a clinical advisory group of ten health care workers. Eighteen thousand headbands of the two designs were then crowdsourced over 3 weeks from Queensland makerspaces, universities, schools, businesses and community members using social media. Each batch was quality checked against the master samples, devices were released to non‐clinical areas to relieve commercial stocks, and technical documentation, a cleaning procedure and a conformity assessment were implemented before release to clinical areas.

Two out of the five participants in the spray test expressed concerns that non‐visible aerosolised particles could fall through the gap at the top of the headbands or stay trapped behind the visor. A subsequent review of the literature found that while some studies exist on the effects of aerosolised particles in an operating theatre environment,12 little is known about how a face shield impacts this exposure.13 This feedback suggests the need for further evaluation to assess efficacy against aerosolised particles, particularly in light of recommendations that face shields, without associated face masks, could help reduce community transmission.14

A suggested approach to selecting novel PPE in the Australian health care setting

Sourcing PPE from non‐traditional sources has been a hallmark of the COVID‐19 pandemic. Health services have been poorly prepared to evaluate the efficacy and safety of novel PPE. Health care workers understandably are using these non‐traditional devices as local supply issues and anxieties increase.

We have outlined the current TGA requirements for PPE to increase awareness of the relevant regulations. If novel PPE is to be deployed, it must be compliant with the TGA requirements and a clinical evaluation protocol should be developed. The resources and expertise required to develop these protocols are unlikely to exist routinely in most health services. A suggested flow chart for evaluating novel PPE is provided in Box 3.

The COVID‐19 pandemic has highlighted the lack of readily available evaluation processes to provide health services with the assurance that PPE sourced using novel methods, such as crowdsourcing or 3D printing, is fit for purpose and TGA compliant. In October 2020, Kursat Celik and colleagues, who also 3D printed face shields, conducted a review of international standards for industrial PPE. They highlighted that there is no universal standard applicable to face shields used in medical contexts, although international standards exist (including ANSI/ISEA Z87.1‐2020: American National Standard for Occupational and Educational Personal Eye and Face Protection Devices).15 In October 2020, the TGA did not state a mandatory standard for face shields, unlike for gowns, masks and gloves.2 Throughout the pandemic, health services have investigated internal capabilities for conducting occupationally relevant assessment, while qualitative and quantitative Australian Standards testing sites are being set up nationwide. Ultimately, collaboration to establish and utilise these new testing capabilities will be key to enabling swift responses to PPE manufacturing challenges.

Box 2 – Droplet testing set‐up showing the four positions and distances used for the simulated cough

---

Reproduced with permission from Metro North Hospital and Health Service.

Box 3 – A suggested approach to selecting novel personal protective equipment (PPE) in the Australian health care setting

---

ARTG = Australian Register of Therapeutic Goods; TGA = Therapeutic Goods Administration. * Health services may wish to consider conducting an evaluation of the device before procuring large quantities or if there are no applicable Australian Standards for the device. † Health services need to regularly check the TGA updates on guidelines on non‐compliant PPE in crisis supply levels.