Human factors validation study of 3 mg sumatriptan autoinjector, for migraine patients.

BACKGROUND: Migraine pain relief is reported by more than 50% of patients who receive low dose (3 mg) of sumatriptan. Currently, there is no two-step autoinjector of low-dose sumatriptan available on the market for acute migraine treatment. To fulfill this need, a fully assembled, single-dose, subcutaneous autoinjector (sumatriptan 3 mg; product-code DFN-11) was developed. The device allows for injection with a simple two-step, push-to-inject process and provides feedback of the injection activation, progress, and completion.
OBJECTIVE: To determine if DFN-11 autoinjector can be used correctly and safely by migraine patients. METHODS AND PARTICIPANTS: A human factors validation study was conducted with 45 migraine patients (30 oral-only medications users; 15 injectable sumatriptan users) who performed one unaided simulated injection. Two days prior, half the oral participants were briefly trained. All others were only given the device to inspect and written instructions to review. No injections were performed during the initial session. All participants received written instructions at the injection session.
RESULTS: All participants (45/45; 100%) performed the injection without any errors. Objective measures included device removal from packaging, cap removal, expiration date check, inspection of fluid in window, identification of allowable injection site, proper device positioning, dose confirmation, and device disposal. All participants (45/45; 100%) reported no difficulty administering the injection and no concerns about using the autoinjector during a severe migraine onset.
CONCLUSION: The results showed that the DFN-11 autoinjector can be used with safe handling without patterns of confusion, failures, high-risk errors, wet injections, or patient safety risks. The DFN-11 autoinjector was validated to be used correctly and safely by migraine patients, whether they were injection experienced, unexperienced, trained, or self-trained.

Introduction

In the US, ~17% of all women and 6% of all men suffer from migraine.1,2 Migraine headaches are extremely painful, debilitating, and can lead to severe impairment and reduce cognitive function.3,4 When suffering a migraine attack, it has been found that a person’s reaction time, reasoning, attention, working memory, and visual–spatial processing capacity decrease when compared to those not experiencing a migraine attack. In addition, the effect of migraine has also been found to impair hand–eye coordination when compared to baseline performance without a migraine attack.5,6 Recent studies in migraine patients have found that patients are well aware of these disabilities and can evaluate the impact of migraine attacks on their performance.4,7

Triptans and nonsteroidal anti-inflammatory drugs are the most commonly prescribed drugs for management of migraine pain.8,9 Specifically, sumatriptan has been found to be effective in alleviating migraine symptoms.10 Administration of this medication may be via oral, intranasal, transdermal, or subcutaneous (SC) injection.11,12 The latter provides the fastest response action.13,14 Considering the aforementioned decline in cognitive ability and physical performance during a migraine, it is a reasonable requirement that medication delivery be as quick and simple as possible, while still maintaining safety for the user.

Given how debilitating a migraine can be, there is need for a quick and efficient way to deliver medication by migraine sufferers. Autoinjectors are intended to be easy to use and are designed for self-administration by patients or untrained people. In comparison to a (vial and) syringe for SC injections, autoinjectors are designed with less steps, while significantly reducing needle exposure risk before and after the injection.15,16

Pain relief was reported by 57% of migraine patients when 3 mg sumatriptan was administered SC and with lower rates of triptan-related adverse events than when higher doses (eg, 6 mg) were administered.17 A 6 mg sumatriptan autoinjector is available on the market; however,18 to administer a SC 3 mg dose requires SC injection preparation from a vial, which is inconvenient in home settings. Therefore, we developed DFN-11, a two-step autoinjector, to allow users to experience a safe and simple process utilizing a low, but effective to many, dose of sumatriptan (3 mg).

The intent was to design a device that 1) is easy to use during cognitive impairments, 2) uses clear visual and audible feedback indicators to support users, 3) contains nonthreatening features that conceal the needle pre- and postinjection, 4) reduces the probability of needle stick injury, and 5) introduces friendly form factors that are comfortable to hold and pleasing to the eye.

The single-use autoinjector adopted for the delivery of sumatriptan for DFN-11 is prefilled with medication and only requires the user to perform two steps to administer its contents subcutaneously: 1) remove the cap and 2) press down on the skin to activate the injection. All steps in the procedure require gross motor movements only. No assembly, button press, or needle exposure is required. This simple two-step procedure is intended to address the cognitive and physical impairments encountered during a migraine.

The goal of the presented, simulated use, validation study was to determine the usability and safety of the DFN-11 autoinjector in migraine patients, the intended user population.

Methods and participants

Ethics

The authors did not seek institutional review board approval for this study as it was not required by the US Food and Drug Administration for this simulated-use human factors study. All participants signed an informed consent

Objectives

The objectives of this human factors validation study were to 1) evaluate the DFN-11 design, ease of use, intuitiveness, and risks of the autoinjector, 2) determine whether the device can be correctly, safely, and effectively used by the intended user population without patterns of preventable use errors that would result in harm to the user, and 3) confirm the device labeling and instructions for use (IFU) to support users in mitigating high risks and use contexts.

Study device

The DFN-11 autoinjector was designed to provide a simple, convenient, and safe means for at-home self-administration of sumatriptan by migraine patients. The SC injection is administered into the thigh or back of the arm (deltoid area).

The DFN-11 autoinjector (presented in Figure 1) is a fully mechanical, hand-held, single-dose, prefilled, disposable device. It is designed for manual needle insertion, automatic delivery of the drug, manual needle withdrawal, and automatic needle guarding. The autoinjector utilizes a prefilled syringe containing 0.5 mL of the medication and utilizes a thin 29 G needle with a sharp five-bevel edge to minimize injection pain and discomfort.19,20 Prior to use of the autoinjector, the syringe needle is shielded from the user by a needle guard to hide the needle and prevent accidental contact. Once the user removes the autoinjector cap and presses the autoinjector on the injection site (depressing the needle guard manually inserts the needle to the controlled insertion depth), the autoinjector will automatically dispense the entire contents of the prefilled syringe using a spring mechanism. Visual indicators in the inspection window (red plunger rod movement) and audible/tactile indicators (clicks) inform the user of proper use of the autoinjector, including the start and end of the injection.

Figure 1

The DFN-11 disposable single-use autoinjector used in the study.

Following the injection, as the user manually lifts the autoinjector from the injection site, the needle guard automatically locks over and shields the needle to prevent accidental needle sticks.

Instructions for use

A single-sided 11×17-in poster with complete IFU was provided to each participant with the device. The IFU was designed to communicate critical procedural information including important warnings and learning of the device to mitigate associated risks. The IFU guided the users through the injection procedure including preparing for administration, performing the two-step injection, and proper disposal of the device.

Ancillary supplies and injection sites

In order to perform the injection, participants used the final production line DFN-11 autoinjector with active drug-filled syringes. The IFU was available at the participants’ discretion. During the injection, participants were provided access to alcohol swabs, cotton balls, and a US Food and Drug Administration-cleared sharps container. Participants with injection experience were randomly assigned an injection site of thigh or upper arm. Injection-experienced participants were able to self-select their injection site (thigh or upper arm). Injection pads were used and placed over the participant’s injection site to simulate real use and handling of the device.

Participants and groups

This study included a total of 45 patients as participants. The study population of the DFN-11 autoinjector was composed of patients who reported having been diagnosed with migraines. This included two subgroups: 1) injection naïve: migraine patients who only take oral medications (n=30) and 2) injection experienced: migraine patients who administer an injectable version of sumatriptan (eg, StatDose) (n=15). These user groups and sample size (Table 1) allowed the study to address the variance in learning and transfer effects that may occur between the different types of users who possibly would be prescribed the DFN-11 autoinjector. The sample size is also consistent with the suggestions by the US Food and Drug Administration draft guidance issued on February 3, 2016, entitled Applying Human Factors and Usability Engineering to Optimize Medical Device Design.21

Table 1

Definition of user groups with exclusion criteria

Group	Condition	Particpants (n)	Exclusionary criteria for all groups
Injection-naïve users	Trained	15	Excluded if did not experience migraines.
Injection-naïve users	Self-trained	15	Excluded if did not currently, or in the recent past, take any medication for the treatment of migraine.
Injection-experienced users	Self-trained	15	Excluded if <18 years of age.Excluded if was a health care provider.

Conditions

To validate the device under representative and intended use contexts, half (n=15) of the injection-naïve patients in the study were provided with brief training similar to that provided by a health care provider. Our aim was to demonstrate that, with minimal training, patients are able to self-inject and are capable of successfully delivering unassisted injections with this autoinjector in a safe and effective manner.

In order to demonstrate that self-trained patients can also deliver an unassisted injection using the autoinjector and associated instructions, all injection-experienced users (n=15) and half of injection-naïve users (n=15) received no formal training prior to their first unaided injection. Participants in this group were given up to 20 minutes to self-educate using the device and instructions. However, participants were not allowed to perform any part of the injection procedure during this time.

For injection-naïve users, this procedure represented a worst-case scenario where someone with no injection experience was required to use an injection device for the first time, with no training whatsoever and only the instructed materials.

Facilities

This study was conducted in the Interface Analysis Associates (IAA) custom usability laboratory in Saratoga, CA, USA, and the usability testing laboratory owned and operated by Fieldwork, Inc. in San Francisco, CA, USA. The room arrangement was the same in both facilities, designed to represent a home-like setting. The room contained a table, chairs, and lowered ambient lighting. During each session, IAA employees recorded participants’ interaction, performance, behaviors, verbal responses, and subjective feedback for each trial in real time. All sessions were video recorded.

Procedure

All participants were scheduled to attend two sessions, 2 days apart. The first session was 20 minutes in length, and the second session was 30 minutes. At the beginning of the first session, all participants signed an informed consent. Participants then read a brief introduction containing unbiased verbiage outlining the study purpose and context.

Participants assigned to the trained condition (n=15) received brief (~10 minutes) training from the moderator consisting of a verbal walkthrough of the injection procedure and were given time to review the IFU on their own. Participants then demonstrated the full injection procedure (without actually injecting) under supervision of the moderator. Feedback and any needed procedural corrections were provided to the participant following the supervised injection.

Participants assigned to the self-trained condition (n=30) did not receive any formal training during their first session. Instead, these participants were asked what they would do, if anything, to familiarize themselves with the device and procedure. If the participant stated that they would not read the instructions, or would only read the instructions immediately prior to performing the procedure for the first time, the first session was concluded. If the participant stated that they would read the instructions immediately after receiving the prescription or in advance, prior to injecting the medication, the participant was given time to familiarize themselves with the device, IFU, and injection procedure. During this period, participants were allowed to hold the device; however, they were instructed not to interact or manipulate it in any way.

All participants returned 2 days later for their second session involving unassisted use. Session 2 tasks were identical for both trained and self-trained participants. First, participants were asked to identify the allowable injection sites.

Next, participants were instructed to perform an unassisted injection into an injection pad placed over their respective injection site, without the moderator or any test personnel present in the room. Participants were given access to, but not instructed to, use the IFU during the unassisted injection. Additionally, the lights in the test room were dimmed during this unassisted injection to mimic the lowered ambient lighting a patient would most likely operate under during a migraine episode. Any difficulty opening the packaging, removing the autoinjector from the packaging, and using the autoinjector to deliver the injection was recorded by study staff observing the session and confirmed by video following the session.

After the injection, participants answered a series of subjective postinteraction questions, focusing on identifying any difficulties they experienced while preparing or administering the injection. Next, to ensure that participants understood critical steps throughout the procedure, participants were instructed to read selected passages from the IFU and, in their own words, describe the procedure outlined in that step. Participants were also asked if they had any difficulty understanding the IFU content.

Measures

During each session, we observed the behaviors and performance of each step in the injection procedure and recorded the success and failures, errors, confusions, and other indices of mal-interaction that could result in incorrect use of the device. After each task or knowledge probe, participants were actively probed to provide a subjective narrative on ease of use and any difficulties or concerns in using the device.

The performance measures collected included opening the carton, removing the device from the carton, inspecting the medication, cleaning the injection side, removing the cap, injecting at the proper angle (90°), activating the device, administering the full dose, lifting up prematurely (assessed by the presence of liquid on the injection pad postinjection), length of time held (in seconds) after the second click, confirming that the injection was complete, and any needle pricks.

The behavioral measures included verbal comments made by the participant during the study (when applicable) and their reactions to the device. Behavioral measures also included use of reference materials by the participant during the dosing trial.

The subjective measures included feedback related to participants’ interaction with the device and IFU. These data were solicited from participants after performing the dosing trial.

All failures, errors, and significant difficulties observed during the study were followed up by a subjective debrief with the participant at the conclusion of their trial. At the conclusion of each injection, each participant was asked to provide a subjective assessment of their experience.

Results

Participant demographics

Representative migraine patients were recruited across the two user groups, according to the screening criteria defined in Table 1. Across the 45 participants, 87% (39/45) were female and 13% (6/45) were male, with 73% (33/45) taking oral medications and 33% (15/45) taking an injectable medication (some participants reported having experience in using both oral and injectable migraine medications). The participants varied in highest level of education achieved: high school diploma: 19/45 (42%), trade school: 2/45 (4%), associate degree: 14/45 (31%), bachelor’s degree: 7/45 (16%), and master’s degree: 3/45 (7%). Injection-experienced participants were composed of those with current or past experience using a prefilled syringe or autoinjector for the treatment of their migraines. The mean age of the participants was 47 years, with a range of 22–70 years.

Training and self-trained session 1 findings

For trained participants, there were no observations (0/15, 0%) of significant difficulties in simulating the injection procedure (without actually injecting) or understanding the IFU during their first session.

In the self-trained condition, all participants, including the 15 injection naïve and 15 injection experienced (30/30, 100%), stated that they would read the IFU in advance to prepare and familiarize themselves with the autoinjector and injection procedure.

Unaided injection trial findings

Across all conditions, 100% (45/45) of participants successfully administered a full dose using the DFN-11 autoinjector without patterns of failures or use errors. During the procedure, 96% (43/45) of participants referenced the IFU. The two participants who did not reference the IFU were in the self-trained group. All participants (45/45, 100%) stated they did not have any difficulty administering the injection or any concerns about using the autoinjector during a severe migraine. Table 2 presents a summary of all performance measures recorded during the study.

Table 2

Summary of results – all performance measures by condition and group

Metrics	Trained	Self-trained		Overall (n=45)
	Injection naïve (n=15)	Injection naïve (n=15)	Injection experienced (n=15)
Failure to administer full dose	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to remove cap	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to place device at 90° angle	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to activate device (depress needle guard)	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to comprehend need to check expiration date (Step 1B)	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to comprehend need to inspect fluid in window (Step 1C)	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to comprehend need to confirm dose (Step 4A)	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to identify allowable injection sites	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to remove device from packaging	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to hold device against site for at least 1 second after second click	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to comprehend need to clean injection site (Step 2B)	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to comprehend need to wait 5 seconds after delivery (Step 3B)	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)
Failure to comprehend device disposal (Step 4B)	0/15 (0%)	0/15 (0%)	0/15 (0%)	0/45 (0%)

Knowledge probes and reading comprehension findings

Overall, all participants (45/45, 100%) correctly answered the knowledge probe identifying the allowable injection sites. Additionally, all participants (45/45, 100%) correctly answered all IFU comprehension questions related to steps 1B (inspecting expiration date), 1C (inspecting drug solution), 2B (cleaning injection site), 3B (waiting 5 seconds following second click), 4A (ensuring red plunger rod has completely filled window following injection), and 4B (recapping and disposing of a used device). No participants (0/45, 0%) stated having any difficulty understanding any of the previously mentioned IFU sections. Knowledge probe and reading comprehension questions were administered independently, following the unaided injection trial.

IFU review findings

The IFU performed very well and was found to be clear and effective. Of those who referenced the IFU during the injection trial, 100% (43/43) of participants stated that the IFU provided them the information and guidance they needed. Two self-trained participants, one with oral medication experience and one with injection experience, did not refer to the IFU during their injection. Seven participants offered noncritical suggestions to improve the clarity of certain IFU sections.

Packaging findings

There were no observations of any significant difficulties in the participants’ ability to open the packaging and remove the autoinjector. Overall, 100% (45/45) of participants successfully opened the packaging, removed the autoinjector, and showed no difficulty in removing the device from the packaging.

Failures overview

In order to determine the rate of failures observed in this study, we first calculated the total number of potential opportunities for participants to commit a critical or essential task failure. Across all eight critical and essential measures, we calculated 360 potential failure points for all participants (n=45) during the study. Overall, we observed no failures (0/360) across all participants’ high-risk and medium-risk tasks. This correlates to a failure rate of 0.0%.

Subjective impressions

At the conclusion of the session, subjects were asked to verbally provide their impressions of the device and any concerns about using the device during a severe migraine. Table 3 presents a representative sample of participants’ subjective impressions of the device and/or procedure.

Table 3

Selected participants’ reactions to the concluding inquiry: “What do you think of the device?”

User type	Group	Reaction (context)
Injection naïve	Trained	It was really simple (use of the device).
Injection naïve	Self-trained	That really is easy (use of the device). Removing the end and being able to inject right away, it seems like a very streamlined process. It’s not complicated and it’s all compact in one thing. In midst of a migraine, this is way easier than the other methods I have used.
Injection naïve	Self-trained	I think it’s easy, even a kid could do it (use of the device).
Injection naïve	Self-trained	It’s really simple (use of the device).
Injection experienced	Self-trained	I wish that my doctor would prescribe that (the device).
Injection experienced	Self-trained	Its nice, easy, and simple (the device). Depending on the migraine, the pain could be really bad, but this makes it easy. Sometimes I need to turn off the lights, and I could still see this device.
Injection experienced	Self-trained	This is way easier than what I used before (used Imitrex®a previously).

Note:

Imitrex is a registered trademark of the GlaxoSmithKline group of companies.

Discussion

The results showed that across both groups, injection-naïve and injection-experienced (100%, 45/45) patients successfully prepared and injected the full dose using the DFN-11 autoinjector by 1) removing the device from its packaging, 2) preparing the injection site, 3) removing the cap, 4) pressing the device down to activate the injection, 5) holding the device in place for 5 seconds, 6) lifting and removing the device after completion, and 7) disposing of the device in a sharps container.

The design of the autoinjector was highly received by patients as they found the device to be easy, simple, and clear. Injection-experienced patients liked that it was less steps compared to their current device and that it was simple and intuitive for those without injection experience. Furthermore, patients were confident that they could successfully and safely use the autoinjector to deliver an injection during a migraine episode without any difficulty in the future. All patients who referenced the IFU during their unassisted injection trial (43/43, 100%) stated that the IFU provided them the information or guidance they needed. Furthermore, the IFU was proven to be clear and informative, as all patients (45/45, 100%) were able to use the IFU to answer all six knowledge probes questions correctly and without difficulty. We find that the IFU supports the user during the learning process and injection procedure to help complete a safe and successful injection.

Overall, the DFN-11 autoinjector and IFU were successfully utilized by all study participants without use errors. In addition, the results showed no differences between the trained and self-trained conditions. All injection-naïve users and injection-experienced users were able to complete the injection procedure in a safe and effective manner with varying training. Based on these results, we find that the device and procedure are validated to be safe, learnable, and easy to use.

Study limitations

This study was conducted in a controlled laboratory environment. The nature of this study required participants to inject into an injection pad, simulating their skin. All steps were taken to ensure that the laboratory reflected a home environment with dim lights, though participants were not tested while undergoing a migraine attack.

Conclusion

The results of DFN-11 device validation study showed that the DFN-11 autoinjector can be used with safe handling without patterns of confusion, failures, high-risk errors, wet injections, or patient safety risks. Patients’ reactions to the device were very positive and noted that they found the autoinjector to be easy, simple, and clear. Additionally, patients appreciated the streamlined procedure and the intuitiveness of the autoinjector, especially for those without injection experience. The DFN-11 autoinjector was validated to be used correctly and safely by migraine patients, whether injection experienced, inexperienced, trained, or self-trained.