Development and usability of a new subcutaneous auto-injector device to administer hydroxyprogesterone caproate to reduce the risk of recurrent preterm birth.

BACKGROUND: Current administration of hydroxyprogesterone caproate (HPC) by intramuscular injection is associated with limitations, including the potential for human error and contamination, patient anxiety, and increased risk of needlestick injury.
OBJECTIVE: To describe the design of an auto-injector for subcutaneous (SC) administration of HPC and the results of studies that evaluated the target user's understanding of the proper use of this device.
MATERIALS AND METHODS: A single-use, prefilled, fixed-dose, disposable auto-injector intended for the SC administration of HPC was developed, and its usability by health care providers was evaluated in 3 formative (N=32, 64 injections) and 3 validation studies (N=45, 90 injections). These studies consisted of one-on-one testing sessions performed in a simulated home environment. Analyses were based on observed use error or use difficulty during the performance of specific tasks, including those considered critical (associated with high severity harms).
RESULTS: In the formative studies, the majority of participants correctly administered an injection with the auto-injector, but prior training improved performance. Specific errors were noted, including holding the device at the injection site for a period inconsistent with its instructions for use (IFU). The IFU was modified to reduce potential occurrence of these errors. Use errors were subsequently observed on critical tasks in the first and second validation studies, including hold-time errors that were attributed to using visual cues rather than counting seconds. For the third validation study, the IFU was modified to focus on visual cues and all users were able to successfully perform the injection per the IFU.
CONCLUSION: An auto-injector device for SC administration of HPC for reduction in risk of recurrent preterm birth was successfully developed through iterative design and validation testing. The device design provides high usability and acceptance of this device by health care professionals.

Introduction

The incidence of preterm birth in the USA, defined as delivery prior to 37 weeks’ gestation, was 9.6% in 2015, representing the first increase since 2007,1 with a continued increase to 9.8% in 2016.2 Preterm birth is the leading cause of neonatal mortality in the US3 and is associated with an increased risk of long-term complications relative to full-term birth.4–6

One of the most significant risk factors for preterm birth is previous pregnancy history, ie, women who have had a prior preterm birth have a 2.5-fold greater risk than women with no such prior history.7,8 One treatment that has demonstrated efficacy in clinical trials to reduce the risk of recurrent preterm birth is the use of hydroxyprogesterone caproate (HPC, also known as 17-OHP and 17P),9,10 based on the suggested ability of progestogens to support gestation and inhibit uterine activity.11 The use of HPC as an intervention to reduce the risk of recurrent preterm birth has been recommended in guidelines by the major US obstetric associations (The American Congress of Obstetricians and Gynecologists, Society for Maternal-Fetal Medicine, and American College of Nurse-Midwives).12–14 A formulation of HPC that is approved by the US Food and Drug Administration (FDA) is currently marketed as Makena®, AMAG Pharmaceuticals, Inc., Waltham, MA, USA (available as both multidose vials and single-use preservative-free vials), which is an injectable synthetic progestin indicated to reduce the risk of preterm birth in women with a singleton pregnancy who have a history of singleton spontaneous preterm birth.15 In a large, controlled clinical study conducted by the National Institute of Child Health and Human Development, administration of HPC significantly reduced the rate of recurrent preterm birth by 32% among women at high risk for recurrent preterm birth with a singleton pregnancy.10

Administration of HPC has historically been as an intramuscular (IM) injection in the upper outer quadrant of the gluteus maximus muscle using a conventional syringe with a 1½ inch (>35 mm) 21-gauge needle.15 Furthermore, the administration regimen requires that the health care professional first draw the drug from a vial using a larger 18-gauge needle and then switch needles to administer the dose with the smaller 21-gauge needle.15 Slow injection of this medication in a viscous, oily vehicle intramuscularly (over 1 min or longer) is recommended in the approved prescribing information.15

Several factors led to reconsideration of this method of administration. Conventional injection can present risks and challenges to the health care provider, such as the potential for human error and contamination when drawing up the dose in the syringe, patient anxiety in terms of “needle phobia,” as well as a risk of needlestick injury. It has been estimated that the incidence of needlestick injuries among health care workers is ~384,000 cases annually in the hospital setting,16 and may be as high as 800,000 when all health care settings, including home health care visits, are considered.17 Lastly, while there are multiple factors that contribute to the choice of route of administration, when patient preference is considered between the IM and subcutaneous (SC) routes, patients generally prefer the SC to the IM route.18

An auto-injector is a device that completely or partially replaces the activities involved with parenteral drug administration with a conventional syringe and needle. Such devices are increasingly being developed for use in the clinical setting or home environment for treatment of acute and chronic conditions. Potential advantages that may be expected with an auto-injector include reduction in patient anxiety from “needle phobia” since the patient does not see the needle; a reduction in needlestick injuries resulting from a hidden needle with a shielded needle tip; reduction of errors in drawing up the dose consistently; prevention of accidental drug contamination while drawing up the viscous drug or changing the needle; convenience and efficiency to the health care provider; and performing a standardized administration in which the needle is inserted to a specific depth, ensuring that the full dose is delivered every time.19–21

A novel auto-injector was developed for SC dosing of HPC to enhance ease of administration by health care professionals and potentially increase patient adherence to treatment. This design incorporated a smaller 27-gauge, ½ inch needle, which was based on injection into the SC compartment in the back of the upper arm as opposed to the deeper IM space, as well as use of a needle shield that prevents the patient from seeing the needle and reducing the risk of inadvertent needlestick injuries. Additionally, the auto-injector has the advantage that it is a prepackaged, single-dose product, providing greater fill accuracy than can be achieved by manual filling. This method of dosing HPC has been shown to produce comparable systemic exposure, as expressed through area-under-the-curve values, in reference to the traditional IM administration of HPC.22 The auto-injector was approved in the USA in February 2018 by the FDA.

The application of knowledge of human capabilities and limitations, also known as usability or human factors studies, is a clinically relevant component in the development of safe and reliable medical devices such as auto-injectors. The application of usability methods throughout the design cycle is required by regulatory authorities,23 and standards have been developed to guide design and evaluation of such devices.24,25 Usability testing during development is typically divided into 3 stages: 1) early-stage formative studies that are conducted with the aim of providing user feedback to iteratively refine the device design and instructions for use (IFU); 2) late-stage formative tests to confirm that the device is suitable for its intended use and likely to pass the usability part of design validation; and 3) usability validation, which is carried out to provide objective evidence that the intended use has been achieved and that the device can be reliably and safely used by the intended user population.26

While auto-injectors are typically designed for self-administration,19,27,28 the specified user for the HPC auto-injector in the prescribing information is the health care professional who will be administering this drug to the patient on a weekly basis in conjunction with high-risk pregnancy visits.15 Therefore, the goals of this article are to describe the iterative processes of research and design of the auto-injector as well as the usability studies that evaluated the target user’s understanding of product use, and the mitigation of risk to an acceptable level for health care professionals and patients.

Materials and methods

Design of HPC auto-injector for usability

Development and implementation of a novel auto-injector device for SC administration of methotrexate in patients with rheumatoid arthritis19 provided the basis for design of a similar device that would meet user requirements for administration of HPC. The development and usability testing of this device were conducted in accordance with current guidelines on the application of Human Factors Engineering, including FDA guidance.23–25

During the design process, user requirements associated with the use of the auto-injector were identified as design inputs. The auto-injector was designed to meet each of these needs. Table 1 summarizes the identified user needs and how the design was developed to satisfy them. These requirements included no preparation other than removal of a safety cap; a short injection time via a small gauge needle; and drug product contained in a prefilled syringe that provides the preset dosage and a sterile barrier. Feasibility studies were performed to determine the optimal parameters for the HPC injection. A fine gauge needle was desired to minimize pain associated with needle insertion; however, this required a powerful auto-injector spring to deliver the required dose volume of 1.1 mL of the viscous, oil-based HPC formulation in a reasonable amount of time. The use of a suitably designed injector allowed the needle to be reduced in size from 21 gauge to 27 gauge and reduced the delivery time from 1 min to <20 s compared with the IM injection.

Table 1

User needs and their resolution during device design

User need	How user need was met by device design
Device requires no setup other than removing the safety cap	The safety cap has a needle shield capture feature that pulls the needle shield off as the safety cap is removed. Safety mechanism is integrated into the cap. There is no additional safety cap to remove
Indicate, at least by visual means, that it is ready for injection, and when ready to deliver a dose, device is different from its state when the dose has been delivered; the difference should be visible	Syringe content is only visible when the device is ready for injection; when the dose has been delivered, the window becomes occluded
Device will allow user to view the entire contents of the syringe prior to injection	Clear window allows for viewing syringe contents
The needle should be hidden from the user throughout the injection process	Spring-loaded needle guard retracts during injection and automatically re-extends upon removal from the injection site
The device should minimize the potential for needlestick injury	Once the injection is complete, the device locks out, preventing further exposure of the needle
Device labeling should provide instructions for the safe use of the device	Device labeling and IFU provide graphics- and text-based instructions to facilitate correct use of the device

Abbreviation: IFU, instructions for use.

Since administration of HPC is once weekly over a period that may be as long as 21 weeks, the need for repeat injections requires altering administration locations (into the back of the left or right upper arm) at weekly intervals. The intended users of the auto-injector are health care professionals, and all users are expected to have been previously trained to deliver SC and IM injections. The setting for use is a clinical environment, such as the provider’s office or at the patient’s home during a home health care visit.

An initial design concept was created that included a draft IFU. A user task analysis was performed using these materials to assess each step of the injection process. A user task analysis is a cognitive walk-through of the injection process that examines the following for each step in the injection process:

What is the goal of the step?

What information is available to the user?

What decisions the user needs to make at each step?

What actions are needed (physical steps, information gathering, interpreting, decision making)?

What can go wrong and the associated harm?

The user task analysis ensured that the user interface of the device and the printed instructions work together to provide enough information to successfully use the device. A usability Failure Modes and Effects Analysis risk assessment was performed to identify features and use steps that required modification to reduce the potential for patient harm. The potential hazards shown in Table 2 were identified for drug-device combination auto-injector products from FDA database searches, specifically the Manufacturer and User Facility Device Experience and quarterly reporting of the Adverse Event Reporting System, as well as Antares’ and Design Science’s experience in usability testing for auto-injector devices. The table also shows how these hazards were mitigated during development of the current device. Once risk reduction changes were implemented, the device and instructions were then tested in formative usability engineering studies to assess the potential for use errors (UEs) and use difficulty (UD) by represented user groups.

Table 2

Identified hazards and their mitigation during development of the auto-injector for subcutaneous administration of hydroxyprogesterone caproate

Risk with similar products	Mitigation with current device
Inappropriate device choice for a specific drug product, relative to the drug’s viscosity, dosing, or patient population	Addressed by auto-injector design and assessed in all tasks in the study
Unit of measure confusion, including units being inconsistent with dosing directions, units being abbreviated or including trailing zeros, device markings being uncommon for the device type, device markings being illegible or obscured when the drug is added to the device, and device not being able to measure all possible doses	Addressed by design: auto-injector with a single fixed dose that is not adjustable by the user
Unusual or unexpected device operation	Assessed in all tasks in the study via observation
User injury (eg, unintentional needlestick injuries)	The needle remains concealed until ready to use; further assessed in all tasks in study
Incorrect dosing (ie, drawing incorrect volume of fluid for intended dose)	Addressed by design: auto-injector with a single fixed dose that is not adjustable by the user
Improper techniques (eg, premature liftoff)	Assessed in study tasks
Syringe reuse	Addressed by design (ie, device designed to be for single use only: safety guard locks out)

Usability assessment

The usability assessment studies consisted of 3 formative studies and 3 validation studies. The auto-injectors used in these studies were intended to be representative of the commercial product and included on-device labeling and packaging. These devices were also filled with a placebo of similar viscosity to the commercial product and contained an actual needle. The testing devices were provided to participants in cartons that resembled the commercial packaging, and each carton contained 1 auto-injector and 1 IFU. Since all injections were performed on simulated patients, approval by an Institutional Review Board or Ethics Committee was not required.

Usability studies consisted of a single, one-on-one testing session per participant that lasted ~45 min and took place in a simulated home environment, characterized by moderate lighting (~ 200 lux) and visual and audio distractions (ie, a television on at a moderate volume, 45–55 dB). This environment was felt to provide the highest level of potential distraction and most rigorous test, relative to a health care professional’s office setting. Participants representing health care professionals who would use the device in the clinical setting, were either trained or untrained with regard to use of the device, and provided a subjective assessment of the device, IFU, device labeling, and carton labeling. The training (formative studies only) consisted of a 30-min session in which the moderator described the auto-injector components, demonstrated correct use of the device, and allowed the participant to demonstrate the injection back to the moderator. There was a minimum of 24 h between training and testing sessions to simulate the time between a user’s training and the first administered injection in actual use (ie, training decay period).

The analyses in the usability studies were based on observed cases of UEs or UDs during the performance of the specific tasks related to use of the device. A UE was defined as an action (or lack of action) that leads to a result that is not intended or expected by the user (ie, a mistake). A UD was defined as a case of struggling to some extent to complete the intended action; however, a UD is always resolved. Specific tasks associated with high severity harms during use of the injection device were considered critical for appropriate performance.

Labeling design changes introduced in each formative study were evaluated in a subsequent validation study until the acceptance criteria had been met. Acceptance criteria were considered to be met when none of the observed UEs and operational difficulties present an unacceptable risk to the safety of the user or the patient, and none of the safety-related UEs can be further reduced.

Formative studies

The 3 formative studies were performed at Design Science testing labs (Philadelphia, PA, USA) and required completion of 2 simulated-use scenarios and knowledge assessment tasks. Injections were administered into an injection pad placed on a simulated patient. Throughout each session, the study moderator recorded participant behavior and asked follow-up questions regarding use of the auto-injector. In the first 2 formative studies (N=17 and N=8, respectively), approximately half of the participants were trained on use of the device.

The second formative study was designed to assess the effectiveness of IFU and labeling changes from the first formative study, and to determine if there were any new errors associated with the device before validation testing. The third formative study included one-on-one sessions with 7 representative users who completed 2 simulated-use scenarios and knowledge assessment tasks without training, although all participants had access to the IFU in each task.

Validation studies

Participants in the 3 validation studies were required to be licensed pharmacists, physicians, and/or registered nurses who were trained to administer SC and IM injections, with no more than half of the participants having previous experience injecting HPC IM using the currently marketed product. The first validation study took place at Design Science testing labs, and participants (N=15) completed 2 simulated-use scenarios and knowledge assessment tasks (n=30 injections) without any training, although all participants had access to the IFU in each task; no injections were made into patients for this study; all injections were simulated by using manikins.

The second validation study (N=15), conducted at usability testing labs in New York City, was performed to confirm the first study and to incorporate into the evaluation changes made to the IFU as a result of the first validation study. Similarly, the third validation study (N=15), which was performed at Design Science testing labs, incorporated changes made to the IFU and device label as a result of the second study. Participants in the second and third validation studies completed use scenarios with critical tasks that were required to demonstrate the safety and effectiveness of the auto-injector and labeling (Table 3).

Table 3

Use scenarios performed in the second and third validation studies of the hydroxyprogesterone caproate auto-injector

Use scenario	Tasks	Successful performance
Simulated use Scenario 1 (reading IFU optional) and Scenario 2 (reading IFU mandatory)	• Inspect packaging• Inspect the auto-injector for damage• “Check the expiration date”• “Inspect the medication liquid”• “Choose a proper injection site”• “Wash hands”• “Clean the injection site with alcohol”• “Remove the cap”• “Position the device”• Inspect viewing window position• “Hold the arm to be injected”• “Push down to initiate drug injection”• “Hold the device in place until window is occluded”• Count slowly to 3 (Validation 2) or check viewing window (Validation 3)• “Remove the device”• “Dispose of the used device into a sharps container”	Participant delivers full volume of the medication into the injection pad and completes all critical tasks
Scenario 3: labeling knowledge assessment	• “Answer a series of questions using the on-device and carton labeling”	Participant correctly answers all questions using the on-device and carton labeling
Scenario 4: IFU knowledge assessment	• “Answer a series of questions using the IFU”	Participant correctly answers all questions using the IFU

Note: Critical tasks are enclosed within quotes and must have been completed successfully to demonstrate the safety and effectiveness of the auto-injector and labeling.

Abbreviation: IFU, instructions for use.

Results

Auto-injector design

The auto-injector that was developed to meet user needs is a single-use, prefilled, fixed-dose, disposable device intended for the SC administration of HPC (Figure 1). The body includes on-device labeling that identifies the product, dose volume, lot number, and expiration date. The body also includes a viewing window, which allows inspection of the medication prior to use of the device, and is fully occluded after the injection has been completed. The auto-injector contains a nominal 1 mL long prefilled syringe with a 27-gauge staked needle. It features an automated delivery of the drug to the SC tissue once triggered by pushing the device on skin. The tasks followed by the user to perform an injection are demonstrated in the IFU and chiefly involve the actions of removing a protective cap, placing the device onto the skin, pushing down to start the injection, observing a click, holding in place until the full injection is delivered, and disposing of the device. The cap includes a safety seal, which is broken when the cap is removed. The needle shield is also removed with the cap. Therefore, the needle end of the device is exposed after the cap is removed. The unshielded needle remains concealed within the needle end. When the needle end is depressed, the needle is exposed, and when fully depressed, the medication is expelled. The needle end returns to its original location after injection and is locked in place for prevention of accidental needlestick injuries. Thus, the patient receiving the injection does not see the needle prior to, during, or after the injection.

Figure 1

Auto-injector for subcutaneous injection of hydroxyprogesterone caproate.

Note: Images shown are for illustrative purposes only and are not representative of the final drug-device combination product.

The results of the first formative evaluation indicated that the majority of participants were able to correctly administer an injection with the auto-injector. Overall performance improved between the first and second simulated-use trials, with reduction in the number of observed UEs between the 2 trials. Training improved performance, with fewer UEs observed in the trained participants; untrained participants committed a total of 37 UEs and 4 UDs across both simulated-use scenarios, whereas trained participants committed a total of 10 UEs and 1 UD. Several safety-related UEs were identified and were considered related to inadequate product labeling. While no modifications were implemented for the device design, labeling changes were made to emphasize the SC administration route, clarify the wording to improve user understanding of the hold time, and improve the figures indicating device position for injection (Table 4).

Table 4

Summary of changes in the injection instructions over evolution of the IFU

Designation	Users, injections	Critical IFU hold instructions	IFU image
Formative 1	17 (9 trained, 8 untrained), 34	Hold device down after the click while counting from 1 to 20 s to allow all the medication to be delivered. After holding for 20 s, remove the auto-injector.
Formative 2	8 (4 trained, 4 untrained), 16	Continue to hold down after the click. “Slowly” count for 20 s after the click to allow all of the medication to be delivered. After holding for 20 s, remove the auto-injector.
Validation 1	15 (untrained), 30	A click will occur when the injection begins. Hold for 20 s. Remove the device.
Validation 2	15 (untrained), 30	While holding against the arm, watch the viewing window until it is fully blocked (completely orange), continue to hold and slowly count to 3. Remove the auto-injector.
Formative 3	7 (untrained); 14	While holding against the arm, watch the viewing window until it is fully blocked (completely orange), continue to hold and slowly count to 3. Remove the auto-injector.
Validation 3	15 (untrained); 30	While holding against the arm, watch the viewing window until it turns orange. Verify viewing window has turned completely orange before removing from injection site.

Abbreviation: IFU, instructions for use.

In the second formative study, none of the trained participants experienced any UEs. However, critical UEs occurred in the untrained group, including not holding the arm with opposite hand when administering the injection (n=3) and inadequate holding time of the device at the injection site (n=2). To reduce the potential for occurrence of these errors, the administration step with regard to holding the patient’s arm and understanding the hold time was further clarified (Table 4).

In the third formative study, 2 participants did not hold the auto-injector at the injection site until the injection was complete, resulting in wet injections. These participants misinterpreted images in the IFU, believing that they were being instructed to remove the auto-injector from the injection site after 3 s, rather than injecting the full dose and waiting 3 s before removing the auto-injector.

Based on findings from this formative study, the figure in the IFU was modified to show progression of the orange plunger across the viewing window, and the instructions to hold for an additional 3 s were replaced with instructions to verify that the orange plunger has filled the viewing window (Table 4).

Demographic characteristics of the health care professionals enrolled in the validation studies are shown in Table 5. The age range was generally similar across the studies, and participants were primarily nurses, with a higher ratio of females to males in studies 1 and 3, and few had previous experience with HPC administration.

Table 5

Characteristics of health care provider participants in the 3 validation studies

Characteristic	Study 1 (N=15)	Study 2 (N=15)	Study 3 (N=15)
Sex, n
Male	3	7	4
Female	12	8	11
Age (years), median (range)	34 (28–64)	48 (29–65)	37 (27–62)
Occupation, n
Nurse	12	13	15
Physician	2	1	0
Pharmacist	1	1	0
HPC experience, n
Experienced	4	1	0
Naive	11	14	15
Handedness, n
Right	13	15	12
Left	2	0	3

Abbreviation: HPC, hydroxyprogesterone caproate.

In the first validation study, several UEs were observed on critical tasks. In particular, in 10 of the 30 simulated injections, the participant did not hold the device on the injection site for the required time. While this resulted in 1 incomplete dose (ie, a wet injection), the remaining 9 simulated injections resulted in a complete dose despite holding for less than the required delivery time of 14 s. Most of these hold-time UEs were attributed to the participant utilizing the visual cue of looking at the viewing window instead of the hold time or incorrectly counting to 20 s (instructions to hold for 20 s were to account for typically fast counting to ensure the majority held for at least 14 s). The other UEs included 1 participant who did not hold the arm with his free hand in his first injection, and 1 participant who did not rotate injection sites between injections. In both cases, the participants stated that they did not see the instructions in the IFU. Other errors were attributed to the “simulation” nature of the exercise, ie, 1 participant who did not clean the injection site properly before either simulated injection stated that she did not pay attention to this step because it was not a real patient, and another who did not check the expiration date stated that she assumed that the injector was not expired due to the simulated testing environment; both affirmed that these errors would not occur when administering a real injection.

To address the main UEs, the IFU was modified following the first validation study to focus on the viewing window instead of a 20-s hold time, as the window occlusion corresponds with the delivery time. The specific instructions were written to press and hold the auto-injector against the skin until the viewing window was fully blocked, and continue to hold to the count of 3 (Table 4).

In the second validation study, performance improved between the first and second injections (Table 6), and there were only 2 instances in which the participant did not hold for the complete injection, both of which occurred during the first task. During the second task, there were no UEs or UDs observed for holding the upper arm, pushing the device down, and holding until the window is occluded. To further reduce the occurrence of incomplete injections, the figure in the IFU was modified to illustrate the text instructions introduced in the previous validation study (Table 4).

Table 6

Results of simulated use in the second and third validation studies

Steps	Number of participants with events
	Second validation study (N=15)						Third validation study (N=15)
	IFU optional			IFU mandatory			IFU optional			IFU mandatory
	UE	UD	Correct	UE	UD	Correct	UE	UD	Correct	UE	UD	Correct
Open package	–	2	13	–	1	14	–	–	15	–	–	15
Inspect auto-injector for damage	–	–	15	–	–	15	1	–	14	–	–	15
Check expiration datea	–	–	15	1	–	14	2	–	13	–	–	15
Inspect medication through window	1	–	14	1	–	14	–	–	15	–	–	15
Select appropriate injection site	2	–	13	3	–	12	–	–	15	–	–	15
Wash handsa	2	–	13	2	–	13	1	–	14	–	–	15
Clean injection sitea	3	–	12	1	–	14	1	–	14	–	–	15
Remove capa	–	1	14	–	–	15	–	1	14	–	1	14
Do not touch safety guard prior to injectiona	1	–	14	–	–	15	–	–	15	–	–	15
Place device and position it 90° to injection site	1	–	14	1	–	14	–	–	15	–	–	15
Inspect viewing window position	N/A	N/A	N/A	N/A	N/A	N/A	–	1	14	–	–	15
Hold upper arma	1	–	14	–	–	15	–	1	14	–	–	15
Firmly push device down until click occursa	–	5	10	–	–	15	1	2	12	–	1	14
Hold device in place until viewing window is occludeda	2	–	13	–	–	15	–	1	14	–	–	15
Count slowly to 3	–	–	13b	–	–	15	N/A	N/A	N/A	N/A	N/A	N/A
Check viewing window	N/A	N/A	N/A	N/A	N/A	N/A	–	–	15	–	–	15
Remove devicea	–	–	15	–	–	15	–	–	15	–	–	15
Dispose of device into a sharps containera	–	–	15	–	–	15	–	–	15	–	–	15

Notes:

Indicates a critical step;

two participants removed the auto-injector before the injection was complete and did not have the opportunity to count slowly to 3. – indicates a zero (0).

Abbreviations: IFU, instructions for use; N/A, not applicable; UD, use difficulty; UE, use error.

In the third validation study, all participants were able to correctly administer an injection with the auto-injector. Overall performance improved between the simulated-use trials (ie, reduction in the number of observed UEs between Trial 1–IFU Optional and Trial 2–IFU Mandatory) (Table 6). All UEs and UDs observed decreased between these scenarios; 5 of the 6 UEs during scenario 1 occurred at critical steps. Specifically, these errors included failure to check the expiration date, wash hands, clean the injection site, and inject drug. Three of the observed UEs were attributed to test artifacts that were artificially caused by the testing environment or simulated scenarios, and therefore do not represent potential harm to the patient. The remaining 3 UEs were attributed to the device, labeling, or user perception. These included not initially pressing down on the device to activate the auto-injector for the first injection, not inspecting the auto-injector for damage on the first injection, and not checking the expiration date before administering the first injection because the on-device labeling does not instruct the user to check the expiration date.

Discussion

A series of usability studies demonstrated successful development of an auto-injector device for SC administration of HPC. The device met user requirements and demonstrated advantages over conventional IM administration that included ease of device use and should result in enhanced safety as a consequence of the device design (self-shielded concealed needle). In these studies, each successive test resulted in alterations to the device labeling and/or IFU to address sources for UEs; there were no modifications needed for the device design. The most common critical UE in earlier studies was ensuring the user held the device in place for long enough to deliver the complete dose. The iterative changes after each usability trial resulted in a general decrease in injection errors, with no observed injection errors by the third validation study (Figure 2). A complete summary of these changes is presented in Table 7 and further discussed in the following section.

Figure 2

Number of injection UEs by users over evolution of the IFU.

Abbreviations: IFU, instructions for use; UEs, use errors.

Table 7

Summary of changes made to labeling and IFU based on the formative and validation studies

Finding	Change
First formative study
Route of administration error	Made “Subcutaneous” more prominent throughout IFU and labeling.
Position of injection error	Updated figures in IFU to improve clarity of injection position and health care practitioner hand position.
Device angle error	Improved IFU to clarify device position.
UE for several preparatory steps, including inspecting device for damage, inspecting medication, and checking expiration date	Created a preparation heading, and made inspection and preparation numbered steps on the IFU.
One user did not see the second page	IFU is entirely on one side of 1 sheet rather than 2 pages.
Hold-time errors	Clarified wording in IFU to improve user understanding of hold time.
Users touched needle end	Changed IFU wording from “safety guard” to “needle end” and made warning more prominent.
Second formative study
Hold the front of the upper arm with opposite hand	Removed specific holding instructions to allow for any hold of the arm for stability to ensure a full injection.
Hold device on the injection site for 20 s	Simplify instructions to push, click, and hold (ie, push until you hear a click, and hold for 20 s).
Several participants opened the device packaging from the “wrong” side (ie, the side where it is difficult to access the IFU)	Included label on “wrong” side of packaging to “Open other end” and seal the end with a clear sticker. Also, included a thumb cutout on the “correct” side of the packaging.
Third formative study
Wet injection	Changed figure to show progression of orange plunger across the viewing window. Replaced instruction to hold for an additional 3 s with instruction to verify that the orange plunger has filled the viewing window.
First validation study
Hold device at injection site for 20 s	Modified the instruction to focus on the viewing window instead of the 20-s hold time. The new instructions state to position the auto-injector at the site with the viewing window in sight. Press and hold auto-injector against the skin until the viewing window is fully blocked, and continue to hold and slowly count to 3.
Rotate injection sites	Added a bullet in front of the instruction to rotate injection sites.
Second validation study
Incorrect injection location	Added bullet in front of existing text “Only use the back of either upper arm for injection site” instruction.
Additional hold time	Show depressed auto-injector needle end against skin and add a “plus” symbol between the auto-injector and timer to indicate that the auto-injector should be held until the injection window was fully blocked and then slowly count to 3 before removing the device from the injection site.
Third validation study
Hold-time viewing	While holding against the arm, watch the viewing window until it turns orange. Verify viewing window has turned completely orange before removing from injection site.

Abbreviations: IFU, instructions for use; UE, use error.

In the discussion of UEs detected in these studies, health care professionals rarely noted the label or IFU content as the source of their misapprehension of proper procedure. Rather, users uniformly cited prior practices (negative transfer errors), beliefs, or memory lapses. In addition, there were several artifact errors that arose due to the fact that a simulation is not the same as actual practice.

Acceptance criteria were considered to be met by all 3 validation studies, and the device was found to have a high level of usability. The final IFU as tested in validation study 3 was considered the optimal wording, as it produced no injection UEs. However, residual risks may exist that are inherent with injection devices and therefore cannot be completely mitigated by design or labeling changes. These risks include negative transfer, which is present when there are similar products familiar to the users.

An inherent risk with any injection is inadvertent administration of IM drug into the SC layer, or SC drug into the IM layer. The shorter needle length of the auto-injector mitigates this risk, as does proper injection technique.

Not rotating injection sites is a risk that is inherent with all injections and cannot be completely mitigated with design or labeling changes. Not rotating injection sites may cause more discomfort to the patient on subsequent injections, but does not by itself lead to a harm of not receiving the correct dose from the device. The auto-injector mitigates this risk as much as possible by requiring all users to be health care professionals, and by instructing users to rotate injection sites in the IFU. Since treatment with HPC is not chronic, but consists of weekly injections, up to a maximum of 21 total injections per patient, there is not likely to be a serious cumulative effect at the injection site in the absence of site rotation. In actual use, the patient can also provide the health care professional with feedback regarding the previous injection location or any discomfort, further mitigating the frequency of this risk.

Touching the safety guard represents a risk to the health care provider, since the safety guard is intended to protect the user from accidental needlestick injuries. However, the severity of the potential harm is minor because the needle is not contaminated before the injection, and the safety guard locks into place after the injection.

A final residual risk inherent with all injection devices is that of incomplete injection or missing the dose due to malfunction or incorrect use. Since HPC is not an acute therapy, has a long half-life and plasma levels drop off very slowly,29 the therapeutic effect of these errors is reduced. Such errors observed in the present study were resolved after the participant read the IFU in the second simulated-use scenario, demonstrating that the IFU is effective in mitigating the risk of this error. Subsequently, the device was successfully used in humans in the study that demonstrated bioequivalent drug exposure between SC administration using the auto-injector and the standard IM administration.22

Limitations

Limitations of this study include a limited number of users, and while the high proportion of nurses could potentially be criticized as selection bias, it should be noted that HPC is mostly administered by nurses or medical assistants rather than physicians. The simulation nature of these studies may also be considered a limitation, including that not all users were familiar with this drug. However, FDA guidance recommends simulation testing as an acceptable method for assessing the safe and effective use of an auto-injector device.23 Additionally, the auto-injector and IFU were not tested for direct use by patients, since the device is labeled for use by health care professionals.

It should also be noted that auto-injectors may not be appropriate for all patients. As a new mode of administration, the health care professional is required to learn an alternative means to deliver medication in addition to conventional syringe and needle injections. Malfunctions with auto-injectors are possible as they are more complex, being comprised of multiple components. However, as described earlier, the evaluated auto-injector completed extensive device verification and validation testing to ensure reliable use.

Conclusion

Successful development of an auto-injector device for SC administration of HPC was achieved through iterative design and testing; the use instructions and labeling was then validated through a series of usability studies. Although residual risks of using an auto-injector will always be present, a simplistic user interface and modifications of labeling and IFU through these studies provided mitigation as much as reasonably possible for use by health care professionals for administering HPC with this SC device. This risk mitigation resulted in high usability and acceptance of this device that represents a novel design for viscous drug delivery and which provides health care professionals with convenience, ease of use, and needle safety during administration of HPC for reduction in risk of recurrent preterm birth.