Using Patient Feedback to Optimize the Design of a Certolizumab Pegol Electromechanical Self-Injection Device: Insights from Human Factors Studies.

INTRODUCTION: We incorporated patient feedback from human factors studies (HFS) in the patient-centric design and validation of ava®, an electromechanical device (e-Device) for self-injecting the anti-tumor necrosis factor certolizumab pegol (CZP).
METHODS: Healthcare professionals, caregivers, healthy volunteers, and patients with rheumatoid arthritis, psoriatic arthritis, ankylosing spondylitis, or Crohn's disease participated in 11 formative HFS to optimize the e-Device design through intended user feedback; nine studies involved simulated injections. Formative participant questionnaire feedback was collected following e-Device prototype handling. Validation HFS (one EU study and one US study) assessed the safe and effective setup and use of the e-Device using 22 predefined critical tasks. Task outcomes were categorized as "failures" if participants did not succeed within three attempts.
RESULTS: Two hundred eighty-three participants entered formative (163) and validation (120) HFS; 260 participants performed one or more simulated e-Device self-injections. Design changes following formative HFS included alterations to buttons and the graphical user interface screen. All validation HFS participants completed critical tasks necessary for CZP dose delivery, with minimal critical task failures (12 of 572 critical tasks, 2.1%, in the EU study, and 2 of 5310 critical tasks, less than 0.1%, in the US study).
CONCLUSION: CZP e-Device development was guided by intended user feedback through HFS, ensuring the final design addressed patients' needs. In both validation studies, participants successfully performed all critical tasks, demonstrating safe and effective e-Device self-injections. FUNDING: UCB Pharma. Plain language summary available on the journal website.

Introduction

Anti-tumor necrosis factors (anti-TNFs) can be an effective treatment option for a number of chronic inflammatory diseases, including rheumatoid arthritis (RA), ankylosing spondylitis (AS), psoriatic arthritis (PsA), and Crohn’s disease (CD).

Certolizumab pegol (CZP) is an Fc-free, PEGylated anti-TNF [1], delivered subcutaneously, often by self-injection. Self-administration of subcutaneous injections can be beneficial to patients, as this can increase independence, and offer greater flexibility in the timing of injections [2, 3]. However, suboptimal treatment adherence can reduce the clinical benefits of anti-TNF therapy [4]. Treatment adherence is influenced by multiple factors. Many patients with RA, AS, PsA, or CD face barriers to safe self-injection, including needle phobia, problems with manual dexterity, pain linked to joint swelling in the hands, and a lack of confidence in the self-injection process [5-8]. As a number of factors affecting treatment adherence are associated with treatment delivery [3], optimizing device design can provide an opportunity to improve adherence, enhance the patient experience, and aid disease management. Self-injection devices can be designed specifically to meet patients’ unmet needs and provide flexibility and control for treatment administration, with additional features such as injection reminders to aid scheduling, promoting ease of use [2, 3].

Currently, CZP is administered via a prefilled syringe (PFS) or prefilled pen (PFP) [1]. Patients, together with healthcare professionals (HCPs), can select a device best suited to their needs, taking into account factors such as a visible versus hidden needle, and the control of injection speeds using manual versus automatic injection devices. For example, in a study in patients with RA, participants compared several designs of PFPs and identified the ability to hold the grip securely, reassurance of medication delivery, an easy activation method, and a cap that is easy to remove as the most important design features for self-injection devices [9].

Human factors studies (HFS) are designed to collect feedback from intended users on device design and user interface, and to eliminate or mitigate any potential use-related hazards [10]. HFS are used to observe patients’ interactions with devices and iteratively improve device design by listening to patient feedback. A review of medical device usability testing methods highlighted the value of HFS as part of a patient-centric design process, with user-feedback-led design more likely to result in patient satisfaction and correct use of the final device [11]. Furthermore, these studies form part of the USA FDA’s requirements to validate the usability and safety of medical devices [10, 12, 13], and help to demonstrate conformance with requirements of EU and international regulatory authorities, such as the European Parliament and Council of the European Communities’ medical devices directives and the International Electrotechnical Commission [13-15].

Here, we report the results from formative and validation HFS conducted across the UK and USA to support the development of ava®, an electromechanical injection device (e-Device) designed for self-injection of CZP. Through interaction with patients across several indications, healthy volunteers, HCPs, and caregivers, we sought to ensure the development and the safe and effective use of the new e-Device and hence to further widen the device choices available for patients receiving CZP therapy. Responses from intended users were used to develop, improve, and validate the final design of the CZP e-Device, including but not limited to the device itself, its packaging, and instructional materials for use of the e-Device. The ava® e-Device was recently approved for CZP administration in the EU [1].

Methods

A comprehensive series of 13 HFS (11 formative studies, 1 EU validation study, and 1 US validation study) were used to guide the development of the e-Device and validate the final version for use. The studies enrolled a mixture of HCPs, caregivers, healthy volunteers, and patients.

The initial prototype CZP e-Device was designed in collaboration with OXO (New York, NY, USA), with use of feedback from RA patients and HCPs. Users recommended a simple and intuitive graphical user interface (GUI), leading to the development of on-screen, step-by-step self-injection guidance. This initial prototype was designed to automatically direct the user through the entire injection process, including inserting the dose-dispenser cartridge (DDC), performing the injection, and DDC disposal. A skin sensor on the device was included so injection could be initiated only when the e-Device was placed in full contact with the skin, and to ensure that the needle would be fully retracted and hidden from the user when not in use. A scroll wheel allowed the choice of the speed of the injection (four speeds between 7 and 18 s), with additional control provided by a pause button to stop the injection at any time.

This OXO-designed prototype e-Device was then refined and validated with use of feedback from 13 HFS, performed by independent human factors specialists. Eleven formative HFS were used to iteratively refine the e-Device; the first two formative studies were led by SmartDesign (Barcelona, Spain), the third, fourth, and ninth formative studies were run by UCB Celltech (Slough, UK), and the remaining formative and validation studies were performed by Medical Device Usability Ltd (Cambridge, UK). The EU and US validation HFS included intended users to assess the final design.

According to the label recommendations, CZP regimens are initiated with an initial loading dose consisting of 400 mg at weeks 0, 2, and 4 followed by a maintenance dose of either 200 mg every other week or 400 mg every 4 weeks [1]. For the HFS involving injections, no medications were administered. Instead, simulated self-injections were administered with use of an artificial skin pad attached to the patient’s upper thigh or abdomen. The validation studies’ test environments simulated either a home environment (for patients and caregivers) or an office environment (for HCPs).

Simulated injections were performed in nine of the 11 formative studies and both of the validation studies. After each simulated injection, a moderator asked open-ended questions to understand the participants’ experiences. Use errors and feedback were recorded by observation of participants during the simulated injections or by knowledge tests; these included asking who would set up the e-Device, what dose of medication the participant would inject, and what the participant should do to confirm that the dose has been successfully delivered. Following each formative study, participant questionnaire feedback was used to continually improve the design of the e-Device and user interface.

Participants

HCPs, caregivers, healthy volunteers (both injection-naïve and injection-experienced healthy volunteers), and patients with RA, CD, AS, or PsA were enrolled in the formative and validation studies; participants reflected the intended users of the device and were recruited via phone and e-mail screeners. Participants aged 18 years or older were included if they were caregivers who administered subcutaneous injections for a patient with RA, AS, PsA, or CD; HCPs currently in practice for at least 2 years, and treating at least one RA, CD, AS, and/or PsA patient, administering a total of four or more injections per month; and patients with an active diagnosis of RA, AS, PsA, or CD as confirmed by a physician letter, and currently receiving any medication for these conditions. There were no further eligibility criteria for healthy volunteers other than no involvement with the development of the e-Device. Full eligibility criteria for each individual study are described in Table S1.

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1964, as revised in 2013. Informed consent was obtained from all participants for their being included in the studies. In compliance with the FDA human factors guidelines [12], 15 participants from each distinct user group (HCPs, caregivers, RA patients, PsA patients, AS patients, and CD patients) were recruited for the US validation study. For the EU validation study, 15 participants with RA and 15 HCPs were recruited [13].

Formative Studies

Eleven formative studies (studies 1–10 and study 12) were conducted between March 2011 and December 2015 in the UK (London, Slough, Cambridge, and Stockport) and the USA (Baltimore, MA; Houston, TX; Los Angeles, CA; Englewood, NJ; and Miami, FL). The first formative study used the OXO-designed prototype. The following three formative studies iteratively updated the design of this prototype, incorporating feedback from healthy volunteers, HCPs, and patients with RA. The final e-Device design developed following the fourth formative study was used in the seven remaining formative studies. Intended users provided feedback following the HCP setup and simulated use of the e-Device in response to moderator questions, and healthy volunteers assessed the ease of interpreting injection logs. Details of all e-Device components evaluated are shown in Table 1.

Table 1

Evaluation of electromechanical injection device (e-Device) user interface elements

DDC dose-dispenser cartridge, HCP healthcare professional, IFU instructions for use

aNo critical user tasks depend on using the log. For this reason, the injection log was not evaluated in study 13.

Validation Studies

An EU and a US validation study were undertaken to validate the safe and effective use of the final CZP e-Device, but with CZP replaced by sorbitol in the DDCs. All simulated scenarios reflected the expected use of the e-Device. Twenty-two tasks considered critical for safe and effective administration were evaluated in both validation studies (Table S2), either by observation during simulated injection on an artificial skin pad or by a knowledge test. Predefined criteria determined whether a participant successfully completed the tasks associated with safe and effective use of the CZP e-Device. Successful task completion was recorded if the participant could achieve the successful outcome of that task, or could answer a knowledge question correctly. For a task to be classified as successful, the participant had to complete it within three attempts. Task outcomes were reported as “successes” (completed the task without mistakes), “successes with difficulties” (completed the task within three attempts), or “failures” (did not succeed within three attempts). Any critical task failures were assessed by a moderator in a posttask interview to determine whether the failure was caused by/linked to the design of the e-Device.

EU Validation Study

The EU validation study (study 11) was undertaken in the UK after the first ten formative studies had been completed, and the design was improved and finalized with use of patient feedback. The objective was to verify that untrained participants could understand the instructions for use (IFU), GUI, and e-Device training materials to perform all critical tasks safely and effectively without prior training.

Patients performed simulated CZP loading dose self-injections using an artificial skin pad attached to the intended injection site (abdomen/thigh) and determined the next scheduled injection date. HCPs were asked to set up CZP loading dose and maintenance dose regimens on the e-Device, and perform simulated patient injections. After the simulated injection, each participant had a 120-min interview with a human factors specialist.

US Validation Study

The US validation study (study 13) used the final e-Device design; participants were not given any training and had only the associated packaging, labeling, and instructional materials for guidance. The study was undertaken to demonstrate that all intended users could effectively and safely use the device, and that HCPs could both effectively and safely set up the e-Device, and show others how to use it. Each participant attended two separate 90-min interviews with a human factors specialist, with a simulated injection at each interview, and a 2-week period between the interviews, reflecting the most commonly expected dosing interval for CZP.

Results

Patient Disposition and Baseline Characteristics

Overall, 283 participants (HCPs, caregivers, healthy volunteers, and patients with RA, CD, AS, or PsA) were enrolled in the formative studies (Tables S3, S4) and validation studies (Table 2). Participants were representative of intended users of the e-Device. Across all studies, 260 participants performed at least one simulated self-injection with the e-Device (Fig. 1). No needlestick injuries occurred during any study, and all participants could successfully and safely complete an injection procedure.

Table 2

Validation study participants

AS ankylosing spondylitis, CD Crohn’s disease, CG caregiver, HCP healthcare professional, NA not applicable, NR not reported, PsA psoriatic arthritis, RA rheumatoid arthritis

a“White” is composed of “white”, “white other”, “white British”, and “white American”.

b“Other” is composed of “African American”, “Southeast Asian/Middle Eastern”, “Indian/Pakistani/Middle Eastern”, “Mexican/Hispanic/Latino”, “black/African/Caribbean”, “other (not stated)”, “black”, and “Asian”.

cLevels of disease severity were self-reported by patients.

dInjection responses are not mutually exclusive.

Fig. 1

Participant injection summaries for a formative studies and b validation studies. AS ankylosing spondylitis, CD Crohn’s disease, CGs caregivers, HCPs healthcare professionals, IE injection-experienced, IN injection-naïve, PsA psoriatic arthritis, RA rheumatoid arthritis

Formative Studies with Early Prototypes

The first four formative studies with prototypes involved 17 healthy volunteers, 5 HCPs, 1 caregiver, and 28 RA patients (Table S3; studies 1–4). These studies tested the simulated injection, interpretation of the IFU, and use of the DDC. Step-by-step instructions were provided by the GUI to guide patients in the correct use of the e-Device throughout the injection process, and a training cartridge was available for participants to practice the injection procedure. Key findings included the existence of use errors with DDC insertions and recognition that the training cartridge aided the operation and understanding of the device, and would be valuable for inclusion with the final e-Device. Changes to the device’s design, including alterations to button locations, GUI screen features, and task descriptions, were made to prevent these use errors (all outcomes and subsequent design feature changes are summarized in Table 3).

Table 3

Key findings of formative studies

CGs caregivers, DDC dose-dispenser cartridge, e-Device electromechanical injection device, GUI graphical user interface, HCP healthcare professional, IFU instructions for use

Formative Studies with Fully Functioning Devices

The final design of the e-Device was used in the remaining seven formative studies and is presented in Fig. 2. These studies included 18 healthy volunteers, 25 HCPs, 35 RA patients, 7 AS patients, 8 CD patients, 8 PsA patients, and 11 caregivers (Table S4; studies 5–10 and 12). In addition to features analyzed in studies 1–4, these studies tested device charging, injection logs, and GUI alarms. Key findings included user difficulties interpreting the injection log and HCP difficulties with the device setup; these findings and their resulting design modifications are summarized in Table 3.

Fig. 2

a Final certolizumab pegol (CZP) electromechanical injection device (e-Device) and dose-dispenser cartridge (DDC), and b key features of the e-Device. GUI graphical user interface, HCP healthcare professional. (From [1])

Thirty participants were enrolled in the EU validation study (Table 2): 15 patients with RA and 15 HCPs who treat RA patients. Overall, 60 simulated injections were performed. Among the HCPs tested, only one failure to set up the e-Device correctly was recorded, which was due to poor understanding of English (Fig. 3a). Despite some difficulties, all other HCPs could set up the e-Device correctly.

Fig. 3

a 1/15 EU HCPs failed to set up the e-Device with MD1; this was due to poor understanding of English; this HCP was not involved in the remaining e-Device set-up tasks, resulting in n=14 HCPs in the LD then MD2, MD1, and MD2 task groups. b 1/15 US HCPs failed to set up MD1 due to confusion over the meaning of the pharmacy label; the HCP correctly set MD but also included the LD 3 times before succeeding with the task on the 4th attempt. HCP: healthcare professional; LD loading dose; MD1 maintenance dose 1 (200 mg every other week); MD2 maintenance dose 2 (400 mg every 4 weeks)

All 30 participants in the EU validation study successfully completed all the tasks required to safely and effectively administer a full loading dose within three attempts. Failures were reported in four of 285 critical tasks (1.4%) performed by HCPs and eight of 287 critical tasks (2.8%) performed by patients. Failure to correctly dispose of used DDCs in the sharps bin (critical task number 20, CT20) accounted for seven of the failures (five patients and two HCPs); all seven participants stated this was due to their assuming the DDC would be reused within the test scenario and that they would use a sharps bin in daily practice. Other reasons for failure included the following: one patient and one HCP stated they would inject a syringe which had cloudy contents (CT06), two patients did not correctly describe how to clean the e-Device (CT21), and one HCP had a poor understanding of English, resulting in a failure to configure the e-Device (CT02). Successes with difficulty reported by HCPs (n = 26) and patients (n = 14) were most commonly due to an inability to find the information in the IFU (Fig. 4). The device was temporarily positioned upside down by one HCP and one patient, but this was quickly rectified and did not impact on drug delivery (Table 4); these were therefore classified as successes with difficulty. Patients commented positively on the user-friendliness of the final GUI (“I like that it shows you how to inject”; “Told me what to do when I had a difficulty”) and general ease of understanding the GUI instructions (”Bright screen with easy writing”; “Good diagrams for people where English is not first language”).

Fig. 4

Summary of critical task failures and difficulties (EU validation study). DDC dose-dispenser cartridge, e-Device, electromechanical injection device, HCP healthcare professional, RA rheumatoid arthritis

Table 4

Key findings of validation studies

GUI graphical user interface, HCP healthcare professional, NA not applicable

Ninety participants were enrolled in the US validation study (Table 2). Overall, difficulties were reported in 57 of 5310 critical tasks (1.1%). No remaining use errors occurred that were unexplained or unexpected (Table 4). There were two critical task failures (less than 0.1%): one HCP failed to set up the e-Device with a maintenance dose of 200 mg three times, and one patient held the device upside down four times before completing the task correctly after reading the IFU. In both cases, the participants were aware that they were not performing the task correctly, and eventually completed it successfully; as both took three or more attempts to complete the task, these tasks were classified as failures. Critical tasks involving difficulties or failures are summarized in Fig. 5.

Fig. 5

Critical task failures and difficulties [US validation study]. AS ankylosing spondylitis; CD Crohn’s disease; DDC dose-dispenser cartridge; CG caregiver; HCP healthcare professional; PsA psoriatic arthritis; RA rheumatoid arthritis

Discussion

Although efficacy and safety are central considerations when one is selecting treatment options, patient attitudes toward different methods of administration are an important consideration as well. Patients often have a preference for specific devices or administration methods that are influenced by personal priorities such as preferring fewer injection steps or being reassured that the injection was completed successfully [16-18]. Aligning the injection device to patients’ preferences provides an opportunity to improve the treatment experience, which has the potential to promote treatment adherence [3]. The importance of patient choice when selecting treatment options for chronic inflammatory disease was highlighted in two ethnographic studies examining the patient experience of self-injection that interviewed self-injection device users, and concluded that patients with more treatment options and a deeper disease understanding were more likely to have a better self-injection experience [19].

The HFS reported here used feedback from healthy volunteers and intended users to develop a new patient-centric e-Device for self-injection of CZP. HCPs configured the dosage regimen and intended users followed on-screen guidelines to perform their injections. Both the screen design and the GUI text were optimized in response to user feedback during 11 formative studies. Key design changes included moving the screen viewing angle to reflect typical use, altering the GUI color scheme to improve legibility, and text simplification.

In the final US validation study, there were two critical task failures (less than 0.1% of all critical tasks). However, user feedback suggested that had this happened outside a study environment, users would be fully aware that they had not completed the task successfully and the GUI would guide them toward safe use. Patients holding the device in the incorrect orientation is a known problem with self-injection devices [9, 20]. All US validation study participants demonstrated safe use, and no participant activated the device upside down. In all cases where participants initially placed the device upside down, the GUI and the user manual instructed them to correct the orientation of the device before beginning the injection procedure, allowing the injection to be completed successfully.

Rates of adherence to biologic therapy can be suboptimal; in a retrospective study of 53,477 self-injecting patients with RA, AS, or PsA, only 37% were adherent during their first year of use of a new anti-TNF agent [21]. RebiSmart® (Merck, Zug, Switzerland) is an electronic injection device for use by multiple sclerosis patients. It was the first electromechanical autoinjector for multiple sclerosis patients [22], and has reported adherence rates of 95% across a median drug exposure period when the device is used for 979 days [23-25]. There is evidence suggesting that technologized injection devices may increase the rates of adherence to use of biologics. For example, a study assessing the golimumab SmartJect® autoinjection device in RA patients reported that 91.7% of patients completed 6 months of treatment [26].

An electromechanical autoinjector currently in development for use with etanercept, another tumor necrosis factor inhibitor used to treat chronic rheumatological conditions, highlighted the importance of incorporating patient feedback into the design of an injection device [16]. A real-world study comparing patient preferences of an etanercept PFP with the electromechanical autoinjector reported that the etanercept PFP was preferred by patients, as it had fewer steps and patients experienced less injection site discomfort or pain [16]. In previous studies with the anti-TNF adalimumab, many RA patients reported a preference for a PFP over a PFS, due to the use of a hidden needle in the PFP [27]. The CZP e-Device avoids many limitations often associated with self-injection devices through the incorporation of patient feedback during each stage of device development. Patients with pain and reduced dexterity can find it difficult to grip devices securely [8, 28]. The e-Device was designed to reduce these difficulties, incorporating a large, nonslip hand grip and large buttons. The IFU was updated in response to early patient concerns with IFU legibility.

Response to anti-TNF treatment can be reliably predicted from patients’ disease activity following 3 months of therapy [29], and so treat-to-target guidance suggests that treatment should be adjusted at least every 3 months until the desired treatment target is reached [30]. However, this guidance does not differentiate between no response to treatment for nonefficacy reasons or nonadherence. The CZP e-Device automatically generates an electronic injection log, providing quantifiable compliance and adherence data that can be used by both patients and HCPs as a disease management tool to improve treatment decisions.

HFS are limited by their simulated nature; full clinical studies are required to assess common concerns with self-injection such as pain levels and self-injection anxiety. In addition, participants may also perform the self-injection process differently when being observed in an HFS environment rather than in day-to-day life at home or in the office, even when these locations are simulated within the HFS; one EU validation study patient stated that the difficulties this patient had may be related to study anxiety: “Under the circumstances I was probably panicking, unfamiliar with everything, lots of information. At home [I] would be fine.” There are also several limitations associated with the design of the e-Device. The e-Device is battery operated and is relatively large compared with alternative devices such as a PFP or a PFS, which may affect travel and general portability. However, the e-Device needs to be used only one or two times a month, so this is likely to cause only minimal disruption to patients.

Conclusions

User involvement in medical device design and testing provides a valuable means of ensuring that devices accurately meet patients’ needs and desires, and helps to minimize any potential reasons for injection failure. HFS are particularly important, as intended user feedback can be both collected and acted on in accordance with strict US and EU guidelines to validate a device’s usability and safety [10, 12, 13].

In this study, intended user feedback was collected across 11 formative studies in the UK and the USA and was used to develop and improve the patient-centric design of the e-Device, and minimize use error. The EU and US validation HFS confirmed that intended users of the final, optimized e-Device could safely and effectively perform self-injections with the e-Device without training.

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 381 kb)