Comparison of Usability, Accuracy, Preference, and Satisfaction Among Three Once-Weekly GLP-1 Receptor Agonist Pen Devices.

Glucagon-like peptide 1 (GLP-1) receptor agonists are relatively new additions to the treatment options available for type 2 diabetes. These agents provide glycemic control by increasing glucose-dependent insulin secretion, decreasing inappropriate glucagon secretion, slowing gastric emptying, and increasing satiety (1). The GLP-1 receptor agonists are an attractive treatment option because they are associated with weight loss and have a low risk of hypoglycemia. However, their use is limited in some patients because of gastrointestinal adverse effects (nausea, vomiting, diarrhea), cost, and their subcutaneous route of administration (1).

There are currently six GLP-1 receptor agonists approved for use in the United States: exenatide (Byetta), 2005; liraglutide (Victoza), 2010; exenatide XR (Bydureon), 2012; albiglutide (Tanzeum), 2014; dulaglutide (Trulicity), 2014; and lixisenatide (Adlyxin), 2016. Many differences exist within this class of medications, including efficacy and safety profiles, dosing schedules, and preparation and administration requirements, making product selection and use potentially confusing for both health care professionals (HCPs) and patients.

Dosing schedule is one important consideration when comparing GLP-1 receptor agonists. Whereas three of the GLP-1 receptor agonists are available in multi-use, disposable pen devices and are given either twice daily (exenatide) or once daily (liraglutide and lixisenatide), the other three GLP-1 receptor agonists are once-weekly formulations available in single-use, disposable pen devices (albiglutide [2], dulaglutide [3], and exenatide XR [4]). Once-weekly products may offer the advantage of better adherence and ease of use compared to once- or twice-daily medications. In clinical studies, adherence to treatment and treatment satisfaction significantly increased with once-weekly GLP-1 receptor agonists compared to once- or twice-daily GLP-1 receptor agonists (5–7).

In comparing efficacy and safety of the once-weekly agents, meta-analyses and head-to-head trials demonstrate that dulaglutide showed the greatest reduction in A1C compared with other once-weekly GLP-1 receptor agonists, whereas albiglutide showed the lowest reduction (8–10). Data on body weight show a significant reduction for both dulaglutide and exenatide XR and no significant weight reduction with albiglutide (8). Gastrointestinal side effects were seen most often with dulaglutide, followed by exenatide XR and then albiglutide. The three agents did not significantly increase the risk of hypoglycemia (8–10).

Ease of use is another particularly important consideration when comparing the GLP-1 receptor agonists because complexity of medication administration requirements has been linked to poor adherence, lower patient satisfaction, and inaccurate dosing (11). Preparation and administration requirements vary significantly among the GLP-1 receptor agonist pen devices.

Only one study has compared user accuracy and satisfaction of GLP-1 receptor agonists in patients with type 2 diabetes (12). The study compared the usability of and user preference for lixisenatide, exenatide, and liraglutide pen devices. All three devices are multi-dose, disposable pens and are administered either once or twice daily. The study results indicate that lixisenatide allowed faster task completion and had more successful user performance. To date, no study has compared the usability, accuracy, or user preferences of the three once-weekly GLP-1 receptor agonist pen devices. The objective of this study was to compare the accuracy of and user preference for three once-weekly, single-use GLP-1 receptor agonist pen devices—albiglutide, dulaglutide, and exenatide XR—when used by HCPs, as represented by pharmacist interns.

Methods

Study Design

This work was an open-label, task-based, interview-based pilot study comparing three GLP-1 receptor agonist single-use, disposable, once-weekly pen devices. The study design and methods were adapted based on the user-based study comparing lixisenatide, exenatide, and liraglutide pen devices (12). Assessments were conducted with 30 participants. Usability and accuracy performance outcomes were time taken to complete the administration demonstration and percentage of required administration steps demonstrated correctly. User satisfaction and preferences were evaluated by participant surveys. This study was conducted at one major university medical campus and was approved by the investigational review board. Informed written consent was obtained from all subjects before the start of any study procedures.

Participants

Participants in the study were pharmacist interns who were recruited via campus email invitations. To be included in the study, participants must have been enrolled in the Doctor of Pharmacy program at the investigator’s institution. To ensure some basic foundational understanding of diabetes, participants must have successfully completed the core diabetes management portion of their didactic program. This population was selected because 1) patients often do not have access to education from a certified diabetes educator when starting a new diabetes medication, 2) pharmacists are easily accessible to patients receiving a new prescription medication for diabetes, and 3) pharmacists receive specific training and are required by law to counsel patients on new prescription medications. Students with diabetes, individuals working in a diabetes-specific practice setting, and students who currently take a GLP-1 receptor agonist were excluded. A sample size of 30 participants was needed for sufficient power to show a significant difference in the performance measurements.

Study Procedures and Assessments

The study was conducted between September and November 2015 and involved a single site visit for each participant, taking ∼45 minutes per visit to complete. Each participant completed the study in a private room with a laptop computer. The study was completed using demonstration pen devices provided by the product manufacturers. For each pen device, the participant viewed the manufacturer’s instructional video. Subsequently, the participant demonstrated and verbalized how to use the device in a patient education simulation. The study investigator served as the patient and the evaluator and completed an evaluation of each demonstration according to pre-established criteria for required administration steps (Table 1). The investigator documented the time required to demonstrate using each device and recorded whether each demonstration step was demonstrated or verbalized correctly, incorrectly, or not completed. After that, the participant completed a six-item usability questionnaire on which they were asked to rate how easy or difficult it was to perform certain tasks for that product using a 5-point Likert-type scale with 5 being very easy, 4 fairly easy, 3 neither easy nor difficult, 2 somewhat difficult, and 1 very difficult. The process was repeated for the next two products. The order of products was randomized using 3 × 3 Latin squares to avoid learning bias. After completing all three demonstrations, participants completed a 5-item overall preference questionnaire using a ranking scale with 1 indicating most preferred and 3 least preferred. All sessions were observed and assessed live by one study investigator to ensure consistency. All sessions were also recorded and reviewed by the study investigator immediately after the session to ensure accurate data collection.

TABLE 1.

Required Pen Demonstration Steps: Evaluation Criteria

	Albiglutide	Dulaglutide	Exenatide XR
Preparation	1. Take out of refrigerator	1. Take out of refrigerator	1. Take out of refrigerator
	2. Wash hands	2. Wash hands	2. Keep at room temperature for 15 minutes
		3. Remove gray base cap	3. Wash hands
		4. Press against skin and turn towards green button to unlock	4. Look at liquid in the window and make sure it’s clear
Attaching needle and reconstitution	3. Make sure pen has #1 in the window, and don’t use if there isn’t	NA	5. Keep pen upright and attach needle by twisting and pushing down until tight
	4. Hold pen up and twist clockwise until you feel and hear a click		6. Don’t remove the plastic cap
	5. You will see #2 in the window		7. Twist the bottom until you hear a click and the green label disappears
	6. Rock pen side to side five times in a windshield motion		8. Tap 80 times
	7. Keep upright in a cup for 15 minutes		9. Rotate every 10 taps
	8. Wash hands again		10. Make sure the solution is uniformly cloudy with no clumps
	9. Rock side to side five times again		11. If it’s not, tap longer and firmer
	10. Inspect liquid to make sure it’s clear and doesn’t have particles
	11. Inspect liquid, since it will look yellow and have large air bubbles
	12. Peel tab off needle and push needle until you hear click and feel it in place
	13. Don’t twist
	14. Tap cartridge two to three times to bring air bubbles to the top
	15. Keep pen upright
Administration	16. Twist clockwise and feel and hear click	5. Push green button until you hear a click	12. Once mixed, twist until orange label disappears and pull cap off needle
	17. You will see #3	6. Wait for a second click, which means drug is delivered, and dispose of pen into FDA-approved sharps container	13. Inject needle into skin at 90-degree angle
	18. Injection button will also pop out		14. Push button down until you hear click
	19. Remove outer cap		15. Hold 10 seconds to get full dose and then remove
	20. Remove inner cap		16. Dispose of needle in FDA-approved sharps container
	21. Push needle slowly into abdomen and push until you hear click
	22. Hold thumb down for 5 seconds
	23. Pull needle out and put right away into FDA-approved sharps container
Total number of required demonstration steps	23	6	16

FDA, U.S. Food and Drug Administration.

Statistical Analysis

Continuous variables were compared using the one-way analysis of variance (ANOVA) test. Categorical variables were compared using the Fisher exact test. User satisfaction ratings were compared using the Mann-Whitney U test. Statistical significance level was set at 5%, so that P <0.05 denoted a statistically significant difference between groups. The Tukey and Bonferonni adjustments for multiple testing were also applied.

Results

Participant Characteristics

All 30 participants were between the ages of 21 and 35 years. Eight participants (26.7%) were male. Twenty-eight participants (93.3%) were third-year pharmacy students, and two (6.7%) were second-year pharmacy students. All participants had successfully completed the core diabetes management portion of their didactic program, which included lectures on the pharmacology and therapeutics of GLP-1 receptor agonists but no specific training on the once-weekly devices or administration. All participants self-reported that they had normal dexterity. Twenty-eight participants (93.3%) self-reported that they had never used an injectable pen device on themselves or administered the device to another person. One participant who answered “yes” used an epinephrine autoinjector once, which requires an injection technique that is different than that of the GLP-1 receptor agonist pen devices. The other participant administered an insulin glargine pen to a relative twice.

Usability and Accuracy Performance Outcomes

Average demonstration time was shorter with dulaglutide compared to exenatide XR and albiglutide (1.37, 2.30, and 3.53 minutes, respectively; P <0.001 for both, Figure 1) and shorter for exenatide XR compared to albiglutide (P <0.01, Figure 1). Based on manufacturers’ instructions, the number of required administration steps for dulaglutide, exenatide XR, and albiglutide was 6, 16, and 23, respectively (Table 1). Accuracy in demonstrating these steps (i.e., the average percentage of steps demonstrated correctly) was 84.4% with dulaglutide, 83.8% with exenatide XR, and 75.9% with albiglutide (P <0.01 for dulaglutide vs. albiglutide, P <0.05 for exenatide XR vs. albiglutide, and P >0.05 for dulaglutide vs. exenatide; Figure 2).

FIGURE 1.

Usability performance outcome: time taken to complete each pen demonstration. P <0.001 for dulaglutide versus albiglutide and exenatide XR. P <0.01 for exenatide XR versus albiglutide.

FIGURE 2.

Accuracy performance outcome: user accuracy in demonstration steps. P <0.01 for dulaglutide versus albiglutide. P <0.05 for exenatide XR versus albiglutide. P >0.05 for dulaglutide versus exenatide XR.

Errors or omissions during the demonstration process were common. Only 13 participants (43.3%) demonstrated all six dulaglutide steps correctly; one participant (3.3%) demonstrated all 16 exenatide XR steps correctly, and zero participants (0%) demonstrated all 23 albiglutide steps correctly. The most common error for dulaglutide was step 1: wash hands (n = 5). The most common steps done incorrectly or omitted for exenatide XR were step 4 (look at the liquid in the window to make sure it is clear [n = 8]), step 6 (don’t remove the plastic cap [n = 10]), and step 15 (hold plunger down for 10 seconds to get the full dose [n = 8]). The most common steps done incorrectly or omitted for albiglutide were steps 2 and 8 (wash hands [n = 26]), step 3 (make sure pen has #1 in the window and don’t use if there isn’t [n = 9]), step 6 (rock pen side to side five times [n = 14]), step 10 (inspect the liquid to make sure it’s clear [n = 10]), step 11 (inspect liquid, as it will look yellow and have large air bubbles [n = 19]), step 14 (tap cartridge to bring air bubbles to the top [n = 16]), step 13 (don’t twist the needle on [n = 10]), and step 18 (injection button will pop out [n = 13]). All participants (100%) demonstrated the dulaglutide steps in the correct order compared to 83.3% of participants for exenatide XR and 66.7% of participants for albiglutide.

User Satisfaction Survey

User satisfaction survey data are summarized in Figure 3. There was no significant difference between albiglutide and exenatide XR regarding participant satisfaction with reconstituting the pen device (average Likert scale score 3.38 for albiglutide vs. 3.35 for exenatide XR, P = 0.9808). Dulaglutide was excluded from that comparison because it does not require reconstitution before injecting. Dulaglutide was easier to prepare for injecting than exenatide XR or albiglutide (average Likert scale scores 4.70, 3.57, and 2.87, respectively; P <0.001 for both). Dulaglutide was easier to hold stable when injecting compared to exenatide XR or albiglutide (average Likert scale scores 4.83, 4.47, and 4.30, respectively; P <0.05 for both). Participants found it easier to push down the injection button for dulaglutide compared to exenatide XR or albiglutide, but these differences did not reach statistical significance (average Likert scores 4.8, 4.57, and 4.33, respectively; P = 0.076). Dulaglutide was the easiest to determine that the injection had been delivered compared to exenatide XR or albiglutide (average Likert scores 4.87, 3.66, and 3.57, respectively; P <0.001 for both). Participants found it easier to recall administration steps for dulaglutide compared to exenatide XR and albiglutide (average Likert scale scores 4.40, 3.17, and 2.27, respectively; P <0.001 for both) and easier for exenatide XR compared to albiglutide (P <0.05).

FIGURE 3.

User satisfaction survey: usability rating of each pen device. Usability rating scale: 1 = very difficult, 2 = somewhat difficult, 3 = neither easy nor difficult, 4 = fairly easy, 5 = very easy. +P <0.001 for dulaglutide versus albiglutide and exenatide XR. &P <0.05 for dulaglutide versus albiglutide and exenatide XR. #P <0.05 for exenatide XR versus albiglutide. @P >0.05 for dulaglutide versus albiglutide and exenatide XR and albiglutide versus exenatide XR. P >0.05 for all other comparisons.

User Preference Survey

Participant preferences regarding pen devices are summarized in Figure 4. Twenty-eight participants (93%) preferred dulaglutide, one participant preferred exenatide XR, and one participant preferred albiglutide in overall preference (P <0.001 for dulaglutide vs. exenatide XR or albiglutide). Dulaglutide was also the device that 28 participants (93%) found easiest to learn how to use (P <0.001) and believed would be the easiest to teach others how to use (P <0.001). A total of 29 participants (97%) selected dulaglutide as the device that gave them the best confidence of correct and complete dose delivery (P <0.001). Lastly, 28 participants (93%) would most likely recommend dulaglutide to other health care professionals (P <0.001).

FIGURE 4.

User preference survey. P <0.001 for dulaglutide versus albiglutide and exenatide XR for all questions. P >0.05 for all other comparisons.

Discussion

All GLP-1 receptor agonists are administered via injection pen devices. Studies demonstrate that pen devices can increase adherence, treatment satisfaction, perceived social acceptance, and quality of life and provide more accurate dosing than vials and syringes (13–16). Studies focused on insulin therapy show that most patients prefer pen devices over traditional vial and syringe methods (16). However, pen devices have varying features and designs that could help or hinder ease of use. The ease of use of the device should be a key factor in the decision-making process when choosing a specific medication within a drug class because the complexity of the medication administration is a well-established barrier to adherence and proper use. Evidence suggests that a high priority related to patient satisfaction with pen devices is ease of self-administration (11). Features such as an easily depressed push button injection minimize the treatment burden associated with type 2 diabetes and lead to higher patient satisfaction and increased adherence (11).

In this study, dulaglutide was significantly easier to use and preferred by participants compared to albiglutide and exenatide XR. Subjects ranked dulaglutide to be the easiest to prepare, to inject, and to recall all steps compared to the other two devices. Dulaglutide also took the least amount of time to complete the demonstration steps. Overall, participants preferred dulaglutide the most. These findings are not surprising given dulaglutide does not require reconstitution or needle attachment and has significantly fewer administration steps.

The satisfaction differences between albiglutide and exenatide XR were more subtle. Exenatide XR was easier for participants to recall all the steps compared to albiglutide, and this was one notable difference between the two. All other survey differences were not statistically significant.

There are important clinical implications of the accuracy findings of this study. Accuracy in demonstrating administration steps, as measured by percent of steps demonstrated correctly, was better with the dulaglutide and exenatide XR pens than with the albiglutide pen and was not significantly different between dulaglutide and exenatide XR. However, when evaluating the data based on actual number of steps, errors or omissions in demonstration steps were common with the albiglutide and exenatide XR pens. Both medications required reconstitution and needle attachment and required more steps than dulaglutide. Albiglutide requires 23 steps, whereas exenatide XR requires 16 steps in the reconstitution and administration process. Although omitting some demonstration steps such as washing hands may be unlikely to result in anything clinically significant, errors with other steps such as holding the plunger down for 10 seconds or inspecting the liquid to make sure it is clear could significantly affect the dosing accuracy, efficacy, or safety of the medication.

The results of this study add to or reinforce findings of the majority of other recent studies. A study comparing adherence, persistence, and treatment patterns among patients with type 2 diabetes newly initiated on GLP-1 receptor agonist therapy found that patients starting dulaglutide had significantly higher adherence and persistence, with lower discontinuation rates than exenatide XR or liraglutide during a 6-month follow-up period (17). Another study evaluated patient preferences for treatment features, safety, and efficacy of dulaglutide versus liraglutide among patients with type 2 diabetes in Japan (18). Significantly more participants preferred the dulaglutide profile compared to the liraglutide profile (94.5 vs. 5.5%; P <0.001). In this study, as well as in a previous study conducted in the United Kingdom, participant responses indicated that dosing frequency and type of delivery system were the two most important characteristics of a treatment option (18,19). A preference study in Germany and the United Kingdom found that, among patients experienced with injecting GLP-1 receptor agonists, key drivers of treatment preference were side effects, efficacy, dosing frequency, and required preparation (20). However, a recent multinational study of injection-naive patients indicated that side effects, efficacy, and dosing frequency were the most important attributes influencing preference, whereas device size, needle size, and required preparation were least important (21). This last study highlights the fact that patient preference and treatment adherence are complex and multifaceted.

There were limitations to this study. First, manufacturer videos were different in length as well as educational approach, which could affect the video’s educational efficacy. For example, the video on albiglutide was the longest because it included drug warnings at the beginning, which could affect a viewer’s attention span and learning capabilities. Others provided administration education first and then discussed warnings. This study accounted for this difference by only showing participants the instructional portions of each video. However, this process could be a learning barrier for patients if they were instructed to watch the manufacturer video at home. Also, all assessments were made based on a single experience with each pen device. Accuracy and usability could possibly be improved with repeat use of the products, particularly with repeated use of the instructional videos.

Another limitation was in the selection of our participant pool. We did not evaluate the ease of use and satisfaction of type 2 diabetes patients. Instead, we looked at the ease of use and satisfaction of future pharmacists. Pharmacists play a paramount role in educating patients on how to use these devices. The majority of patients receive diabetes care from primary care providers, and few patients are educated on device administration by a diabetes educator (22). Therefore, we believe that assessing the ability of future pharmacists to accurately demonstrate how to use these devices is valuable. Pharmacists are legally responsible for providing patient education on new medications and are the most accessible HCPs to provide medication-related education. We believe that the ease of educating someone on how to use a product would directly correlate to how easy a patient perceives the use of the device. Because these students did not have specific training on these devices before the study, they may have had a similar baseline understanding of the devices as patients would be expected to have. Alternatively, these professional students may have generally higher health literacy levels than many patients, yet still demonstrated less-than-optimal accuracy and ease of use with some of the once-weekly devices. Thus, our results illustrate the challenges that new users face regarding proper use of such devices.

Our study is also unable to provide information on ease of use or preference in patient populations with impaired dexterity because the participants all had normal dexterity. Finally, the open-label design could have led to bias and the sample size was small.

A future study directly comparing the three once-weekly GLP-1 receptor agonists in terms of accuracy, patient satisfaction, and adherence from a patient perspective would be beneficial, and evaluating accuracy and ease of use from the perspective of other HCPs would be useful. Although not a direct limitation of this study, it would also be interesting to evaluate how patients receive education about GLP-1 receptor agonist devices in a real-world setting and how this education affects patient use, adherence, and persistence.

Since completion of this study, the manufacturer of albiglutide has announced that they will be discontinuing its production because of limited prescribing. The lack of use of the product may be, at least in part, directly related to our findings of low user satisfaction with the device and further underscores the importance of easy-to-use pen devices and the need for future studies comparing and evaluating new products as they come to market (23).

Conclusion

More complex medication administration devices can be associated with lower user accuracy, participant satisfaction, and adherence. In this study, among three once-weekly GLP-1 receptor agonists, dulaglutide was associated with faster use with less demonstration errors compared to exenatide XR and albiglutide. Dulaglutide was also associated with higher user satisfaction and preference ratings. User accuracy, ease of use, and participant preference should be considered when selecting a specific treatment option for type 2 diabetes. Larger studies comparing once-weekly GLP-1 receptor agonist devices among different participant groups and in different settings are needed.