Evaluating the usability and safety of the semaglutide single-dose pen-injectors through summative (human factors) usability testing.

AIMS/
INTRODUCTION: A single-dose, shield-activated pen-injector for each of the three approved dose variants (0.25, 0.5 and 1 mg) of once-weekly subcutaneous semaglutide has been developed to improve usability. This analysis presents findings from the summative usability testing process for the single-dose semaglutide pen-injectors, including the pen-injector four-pack cartons and instructions for use.
MATERIALS AND METHODS: A total of 60 adults representing four user groups were included: patients with/without pen-injector experience, non-pharmacist healthcare professionals and pharmacists (each n = 15). Participants carried out four tasks: (i) pen-injector carton retrieval; (ii) first simulated injection; (iii) pen-injector retrieval; and (iv) second simulated injection. All participants carried out task 1, and patients and non-pharmacist healthcare professionals took part in tasks 2-4 (n = 45). The number and types of use errors, close calls and operational difficulties were evaluated, and participants subjectively rated the ease of each task on a scale of 1 (difficult) to 7 (easy).
RESULTS: No potentially serious use errors and only one non-serious use error were reported. Eight participants committed use errors with no potential for harm, one participant committed an unclassified use error, one participant encountered a close call with no potential for harm and one participant experienced an operational difficulty. Mean ease-of-use ratings were 6.7 (task 1), 5.9 (task 2), 6.6 (task 3) and 6.9 (task 4).
CONCLUSIONS: All three dose variants of the semaglutide single-dose pen-injector were considered easy to use (subjective feedback scores near 7) and not associated with any serious use errors, even when participants received no training before study participation.

Introduction

Pen‐injectors have been used in diabetes management since the 1980s , . These were originally used for insulin and are now also available for other injectable diabetes therapies, including glucagon‐like peptide‐1 receptor agonists (GLP‐1 RAs) and glucagon rescue therapy.

Once‐weekly subcutaneous semaglutide (Ozempic®; Novo Nordisk A/S, Bagsvaerd, Denmark) is a GLP‐1 RA that was first approved in 2017 for the treatment of adults with insufficiently controlled type 2 diabetes, as an adjunct to diet and exercise , . Three dose levels are approved: 0.25 mg (starting dose), and 0.5 and 1 mg (maintenance doses). Once‐weekly semaglutide has shown statistically and clinically significant reductions in HbA1c and bodyweight in patients with type 2 diabetes, including in randomized controlled trials of Japanese populations. In January 2020, the US Food and Drug Administration (FDA) approved a label expansion for once‐weekly semaglutide. This label expansion was for the reduction in risk of major adverse cardiovascular events in adults with type 2 diabetes with established cardiovascular disease, based on results from the Semaglutide Unabated Sustainability in Treatment of Type 2 Diabetes 6 cardiovascular outcomes trial (which included 3,297 participants, 8.3% of which were Asian) .

Once‐weekly semaglutide is launched as a multidose pen‐injector, containing four once‐weekly doses, in all countries where it is approved. To improve ease of use, a single‐dose, shield‐activated pen‐injector for each dose has been developed (Figure 1). On March 12 2020, Japan’s Ministry of Health, Labor and Welfare approved these single‐dose pen‐injectors. As required in the development of a safe, effective and easy‐to‐use device, usability engineering was applied throughout the product design process , , . A key aspect of usability engineering applied was usability testing, which is required by many regulatory authorities , , , and recommended by the Japanese Pharmaceuticals and Medical Devices Agency. In usability tests, users are asked to carry out specific, critical tasks (defined by the US FDA as user tasks which, if carried out incorrectly or not at all, would or could cause serious harm to the patient or user, or compromise intended medical care) while interacting with the device in simulated‐use scenarios .

Figure 1

Use of the single‐dose, shield‐activated pen‐injector for semaglutide. (a) Pull the pen cap straight off the pen and (b) push the pen firmly against the skin until the yellow bar has stopped moving.

Usability tests can be divided into three categories , . First, early‐stage formative testing is carried out in early device development to collect user feedback to refine device design and instructions for use (IFU). Second, late‐stage formative testing, sometimes known as “pre‐summative” usability (human factors pre‐validation) testing, is carried out in the middle‐to‐end of device development to confirm that the device and test methodology are ready for summative (i.e., human factors validation) usability testing. Third, late stage testing carried out at the end of development, also known as “summative” usability testing, is carried out to show that the intended users can use the device safely and effectively in the intended environments . Summative usability testing also serves to: (i) identify residual use errors that could result in serious harm (defined as “medical consequences”); (ii) assess risk management measures; (iii) expose use errors or other user interaction difficulties; and (iv) collect participants’ subjective feedback regarding handling of the device.

We present findings from the summative usability testing of the single‐dose semaglutide pen‐injectors in adult patients with type 2 diabetes, non‐pharmacist healthcare professionals (HCPs) and pharmacists. This was preceded by pre‐summative formative testing in adult patients with type 2 diabetes, elderly patients with type 2 diabetes, non‐pharmacist HCPs and pharmacists. The aims of the summative usability testing process were to confirm that intended users (i.e., adult patients with type 2 diabetes and relevant HCPs) could use the semaglutide pen‐injectors and associated IFU correctly and easily, and that intended users are not vulnerable to potentially harmful use errors that could lead to injury or suboptimal therapy.

Methods

Summative usability testing procedures and participants

Novo Nordisk A/S contracted Emergo by UL to carry out the summative usability testing. The testing focused on the differentiation and handling of the semaglutide single‐dose pen‐injectors, as well as their labels, IFU and cartons in a simulated environment intended to reflect actual real‐life use. The summative usability testing plan was reviewed and approved by an institutional review board, and aligned with guidance from the US FDA . Testing was carried out in the USA at two sites, at L&E Research’s facilities in Tampa, Florida, and Raleigh, North Carolina, from 7 to 24 January 2019.

The summative usability testing process included participants representing four intended user groups: adult patients (aged ≥18 years) with type 2 diabetes (split into pen‐injector‐experienced and pen‐injector‐naïve user groups) and HCPs (split into pharmacist and non‐pharmacist user groups, with the latter group comprising certified diabetes educators, inpatient nurses or physicians). Based on FDA guidance , , a sample size of 15 users in each distinct group (60 in total) was deemed sufficient to detect ≥90% of relevant use errors per group in this semaglutide pen‐injector summative usability testing process.

Participants were compensated for their participation, and could end the test at any point without losing compensation. Adult patients with type 2 diabetes were included if they either currently used or could be candidates to use one or more antidiabetes medications (GLP‐1 RAs, orally administered drugs or insulin) and could carry out their own injections. HCPs were included if, in their daily work, they prescribed or dispensed pen‐injectors, or taught others how to carry out injections with pen‐injectors. Exclusion criteria for all participants were: (i) inability to read English; (ii) mental or extreme physical incapacity; (iii) any disease or condition that might interfere with participation; (iv) inclusion in the previous 6 months in any product evaluation of an injection device involving hands‐on use; or (v) occupation in clinical research in the medical device or pharmaceutical industries, or in a field that might influence results, such as packaging development.

The protocol was approved by a suitable institutional review board (Allendale Investigational Review Board, CT, USA), conforming to the provisions of the Declaration of Helsinki. Participants provided informed consent and were appropriately instructed regarding test activities. Participants completed a background questionnaire and then the moderator (test administrator) read the test session introduction. Participants were subsequently informed about the test duration and planned tasks.

Testing was carried out in a room with lighting and noise levels similar to a home or a physician’s office. The room contained: a table for the participant, moderator and data analyst; a refrigerator; an injection pad (referred to as the “injection cushion” in the participant materials); and a transparent tray containing pen‐injectors (covered by opaque paper until the participant began the task). Each session was video recorded and observed by Novo Nordisk representatives through a one‐way mirrored window or by camera.

Tasks

Depending on their user group status (patient, HCP [non‐pharmacist] or pharmacist), participants were asked to carry out a subset of the planned sequential tasks listed in Table 1, and their performance was recorded. To simulate a worst‐case scenario, no participant received training before the test.

Table 1

Sequential task instructions for each of the four intended user groups

Tasks for patients (pen‐injector‐naïve and ‐experienced)
Task 1	You have been prescribed Ozempic® 0.25 mg/0.5 mg/1 mg (one dose per printed card). Please open the refrigerator and select the right carton
Task 2	Please give one dose into the injection cushion using the Ozempic® pen
Task 3	Select the pen you just took a dose with from the tray
Task 4	Please give one dose into the injection cushion using the Ozempic® pen
Tasks for non‐pharmacist HCPs
Task 1	Retrieve the Ozempic® carton containing 0.25 mg/0.5 mg/1 mg pens from the refrigerator (one dose per printed card)
Task 2	Please give one dose into the injection cushion using the Ozempic® pen
Task 3	Select the 0.25 mg/0.5 mg/1 mg Ozempic® pen from the tray (one dose per printed card)
Task 4	Please give one dose into the injection cushion using the Ozempic® pen
Tasks for pharmacists
Representative prescription order handed out stating Ozempic® 0.25 mg/0.5 mL or 0.5 mg/0.5 mL or 1 mg/0.5 mL
Task 1	Open the refrigerator and select a product according to the prescription order †

Tasks are written as they appeared on the instruction card during the testing process. An illustration of the single‐dose pen‐injectors is available in Figure 1.

Prescription order included brand name, drug name, concentration of drug, number of pens per box, volume in mL per pen and National Drug Code number.

Tasks included in the summative usability testing process were selected based on the intended use and a prior risk analysis of the single‐use pen‐injectors carried out by Novo Nordisk. This risk analysis process involved dividing the use cases and user steps into individual tasks and steps by a task analysis, with each task assigned to the relevant user groups. These were then evaluated by a use error risk analysis to assess whether each task was necessary to validate the safety and effectiveness of the pen‐injectors. Finally, the tasks deemed critical by the risk analysis were included in the final task list in the summative usability testing process.

At the start of each task in the summative usability testing, the test moderator asked each participant to read aloud a written task instruction presented on a printed card, after which participants were asked to summarize the task in their own words to ensure their understanding of the objective. If participants incorrectly summarized the instructions, then the moderator corrected them on the objective before they carried out any tasks. Before the tasks, patients, but not HCPs (including pharmacists), were allowed to independently review the semaglutide single‐dose carton and contents.

Tasks 1 and 3 involved selecting the correct product. In task 1, participants were asked to choose the correct product carton (semaglutide single‐dose pen‐injector and comparator product cartons available; comparators included other GLP‐1 RA products and insulin products) from a range available in the refrigerator (3–6 products for patients, and 3–8 for HCPs [including pharmacists]). For patients, the refrigerator contained one of each type of carton, and they were not stacked, whereas for HCPs (including pharmacists), the refrigerator contained stacks of three cartons of each type. In task 3, each patient and non‐pharmacist HCP (pharmacists did not participate in task 3) had access to a tray containing the semaglutide single‐dose pen‐injector and comparator pen‐injectors (5–6 products for patients, and 6–7 for non‐pharmacist HCPs), and were asked to choose the correct product.

Tasks 2 and 4 involved giving a simulated injection into the injection pad using the pen‐injector, and were completed by patients and non‐pharmacist HCPs only.

Task performance definitions

Participants’ performance during each task was recorded as a task success or task failure. Task failure was recorded when a participant carried out a task with an action or lack of action that potentially could lead to harm or not receiving the prescribed therapy. If assistance was required from the moderator during a task, this was also recorded as a task failure. Task success was recorded when a participant carried out a task in a way that led to receiving the prescribed therapy and did not lead to harm.

Types of study finding

The moderator and the data analyst present during the testing ascertained patients’ performance according to three main study finding types defined in alignment with FDA guidance : (i) use errors; (ii) close calls; and (iii) operational difficulties (Table 2).

Table 2

Definitions of study findings

Category	Definition †	Classification	Example
Use error	User action, or lack of action, different from that expected by the manufacturer, which caused a result that was: Different from that expected by the user, AND Not caused solely by device failure	Potentially serious (possibly associated with a serious AE) Non‐serious (potentially associated with a non‐serious AE) No potential for harm	Selecting and administering fast‐acting insulin instead of the intended semaglutide pen‐injector, resulting in harm or potential harm Selecting the incorrect carton from the refrigerator, resulting in harm or potential harm Removing the pen‐injector from the injection site prematurely
Close call	Situations when a participant almost committed an error, but noticed in time to avoid it, OR Cases in which a participant committed a use error but detected it and performed corrective measures before the error became consequential	Potentially serious Non‐serious No potential for harm
Operational difficulty	User repeatedly attempted to complete a task and showed apparent confusion that this could cause a potential use error	NA	NA
Unclassified use error	An error not captured in any of the prespecified categories in this table	NA	NA

AE, adverse event; NA, not applicable.

All definitions were aligned with US Food and Drug Administration guidance .

Data collection

The types of data collected were: (i) use errors, close calls, and operational difficulties; (ii) instances of moderator assistance; (iii) participant comments before, during or after task performance; (iv) participant responses to questions during the initial, post‐task and post‐test interviews; (v) participant subjective rating of ease of use; (vi) comments by moderator and data analysts regarding events; (vii) time stamps associated with events, such as use errors; (viii) technical complaints and adverse events; and (ix) photos and video recordings.

Participants were asked to rate the ease of carrying out each task on a scale of 1 (difficult) to 7 (easy).

All data were collected confidentially and simultaneously by the moderator and the data analyst during the test session.

Data analysis

Demographic and background information for each user group was summarized. On completion of test sessions, test data were consolidated and analyzed. This was carried out by counting the number of each type of observed use error, close call and operational difficulty that occurred during the test session, identifying the conditions and root cause(s) of these findings based on test data and the professional judgement of the Emergo by UL test team. The number of times participants required assistance to complete the tasks was also analyzed.

Results

Participant demographics

The summative usability testing process included 60 participants representing four user groups (each n = 15). All participants took part in task 1 (n = 60), and only patients (including those who were pen‐injector‐naïve and ‐experienced) and non‐pharmacist HCPs took part in task 2–4 (n = 45).

A total of 15 patients were pen‐naïve, and 15 patients were pen‐experienced; five patients had experience with the Ozempic® multidose GLP‐1 RA pen‐injector, five with the FlexTouch® insulin pen‐injector and five had “other” pen‐injector experience. All 15 non‐pharmacist HCPs were pen‐injector‐experienced (pen‐injector experience data were not collected for pharmacists). The mean age (years) of each group was 61 for patients, 49 for non‐pharmacist HCPs and 42 for pharmacists (Table 3).

Table 3

Participant characteristics in the summative usability testing process

	Summative usability testing process
	Patients (n = 30)	Non‐pharmacist HCPs (n = 15)	Pharmacists (n = 15)
Age, years, mean (range)	61 (32–82)	49 (31–68)	42 (28–72)
Sex
Male	16	5	3
Female	14	10	12
Vision †
Normal	2	6	8
Corrected with contacts/glasses for reading	16	3	–
Corrected with contacts/glasses for distance	5	3	5
Corrected with contacts/glasses for distance and reading	6	3	2
Color‐blind	2	–	–
Hearing
Normal	26	15	15
Corrected with hearing aids in both ears ‡	4	–	–
Dexterity
Normal	21	13	15
Arthritis, both hands	4	1	–
Arthritis, right hand §	2	–	–
Neuropathy, both hands	1	–	–
Neuropathy, right hand	1	–	–
Numbness, right index finger	1	–	–
Tremor in both hands	–	1	–
Highest education level completed		NA	NA
High school	3
Some college	11
Associates degree	4
Undergraduate degree	11
Advanced degree	1
Pen‐injector experience			NA ¶
Ozempic® PDS290 GLP‐1 RA pen‐injector	5	–
FlexTouch® pen‐injector	5	–
Other pen‐injector	5	–
Pen‐naïve	15	–
Experience injecting with pen‐injector	–	15
Occupation	NA		NA
Registered nurse		6
Physician		5
Certified diabetes educator		4
Years in professional practice, mean (range)	NA	19 (5–47)	16 (3–48)
Work environment ¶	NA
Hospital		11	2
Primary care clinic		2	3
Long‐term care clinic		–	2
Endocrinology clinic		1	–
Family medicine clinic		1	–
Outpatient clinic		1	1
Retail		–	10

All values are counts, unless otherwise specified.

GLP‐1 RA, glucagon‐like peptide‐1 receptor agonist; HCP, healthcare professional; NA, not applicable.

One patient reported more than one vision correction/impairment.

There were no participants who had hearing impairments that were corrected in one ear only.

There were no participants who had arthritis in their left hand only.

Data on pen‐injector experience of pharmacists were not collected, because their tasks did not include an injection.

Some participants reported more than one work environment.

Use performance in the summative usability testing process

Potentially serious use errors

No potentially serious use errors were observed. Furthermore, none of the participants required moderator assistance or committed close calls leading to potentially serious or non‐serious use errors (Figure 2).

Figure 2

Instances of use errors and close calls during the summative usability testing process. The instances of use errors and close calls were equal to the number of participants who committed each task failure type. All use errors and close calls were committed by patients except: †two committed by non‐pharmacist healthcare professionals; and ‡committed by a non‐pharmacist healthcare professional.

Non‐serious use errors

One of the 60 participants selected the incorrect semaglutide single‐dose pen‐injector carton from the refrigerator (task 1, carton retrieval). This non‐serious use error was made by a patient without pen‐injector experience, who selected the semaglutide single‐dose 1‐mg rather than 0.25‐mg carton from the refrigerator. The root cause attributed to this use error was the similarity between the semaglutide single‐dose 0.25‐mg and 1‐mg cartons; that is, both have images of the same shape pen‐injectors, blue casing, gray cap, brand name and label design. The main differences are label color and text.

Use errors with no potential for harm

Eight out of 45 participants made use errors with no potential for harm while carrying out tasks 2–4 (pharmacists did not complete these tasks). Of these eight participants, three patients and two non‐pharmacist HCPs had pen‐injector experience. Five participants retrieved the semaglutide single‐dose pen‐injector of the incorrect dose from the tray (task 3, pen‐injector retrieval of the same dose taken from the refrigerator in task 1), with four selecting the 0.25‐mg dose rather than the 1‐mg pen‐injector, and one selecting the 0.25‐mg dose rather than 0.5‐mg pen‐injector. These errors were attributed to the following root causes: (i) similar pen‐injector appearance; that is, the semaglutide single‐dose 0.25‐, 0.5‐ and 1‐mg pen‐injectors are all visually similar; (ii) reliance on the user to determine that the pen‐injector has multiple variants (i.e., 0.25, 0.5 or 1 mg); (iii) habit, particularly for participants with injection‐device experience; that is, not checking the dosage on current pen‐injectors used at home; (iv) obstructed view of the dosage window; that is, covering the dosage window while handling the pen‐injector; and (v) misunderstanding of the task prompt.

Of the eight participants committing a use error with no potential for harm, four removed the pen‐injector from the injection site before the pen‐injector had completed the injection, indicated by the yellow bar in the pen‐injector window ending its motion (task 2, first injection). This type of error was attributed to the following root causes: (i) negative transfer, in which participants transfer their experience with their current mode of injection (other pen‐injector or syringe) to the tasks with the semaglutide single‐dose pen‐injector; (ii) reliance on the user to refer to the IFU for injection instructions; for example, two participants only skim‐read the IFU and did not realize the purpose of the yellow bar filling the dosage window (indicating dosage completion) in the semaglutide single‐dose pen‐injector; (iii) reliance on participants to hold the pen‐injector so that they can see the yellow bar that indicates the single injection is complete; (iv) inconspicuous instruction text; for example, “lift the pen slowly” or “dosing takes about 5–10 s” (these non‐emphasized instructions might have resulted in some participants perceiving that once the yellow bar has stopped moving, they should retract the pen‐injector immediately, and this might have resulted in these participants prematurely retracting the pen‐injector or overlooking the need to wait 5–10 s before removing the pen‐injector from the injection site); and (v) one of the IFU illustrations was interpreted as meaning that the three illustrated actions should occur simultaneously, rather than sequentially. However, all participants who made an error on their first injection attempt carried out the injection correctly on their second attempt.

Unclassified use errors

One out of 45 participants, who was pen‐injector‐experienced, made an unclassified use error while carrying out the tasks. This was an attempt to reuse the semaglutide single‐dose pen‐injector from the first injection (task 2) during the second injection (task 4). This error was attributed to the following causes: (i) negative transfer, in which participants transfer their experience with their current mode of injection (other pen‐injector or syringe) to the tasks with the semaglutide single‐dose pen‐injector; (ii) reliance on users to refer to the labeling to determine the pen‐injector is single‐use; for example, one participant reused the pen‐injector, because they did not realize the pen‐injector was single use; and (iii) reliance on participants to hold the pen‐injector so that they can see the yellow bar that indicates the single injection is complete.

Close calls with no potential for harm

One out of 60 participants encountered a close call with no potential for harm while carrying out the tasks. This was a patient who was pen‐injector‐experienced. The participant did not initially remove the pen‐injector cap before injecting (task 4, second injection). This was attributed to the participant’s pen‐injector experience and preoccupation with a different preparation process for another pen‐injector that they had used previously.

Operational difficulties

One participant out of 60 experienced a single operational difficulty, which was determining where the needle was located, because the needle is not visible before injection (task 2, first injection). This operational difficulty was experienced by an HCP (non‐pharmacist) with pen‐injector experience. The following root causes were attributed to this use error: (i) negative transfer; that is, current experience with pen‐injectors with visible needles; (ii) reliance on users to refer to IFU for understanding pen‐injector parts; and (iii) the inner metal piece of the cap of the pen‐injector, which might appear to store the needle.

Summative usability testing process subjective feedback for IFU

In the subjective feedback analysis (from a scale of 1 [difficult] to 7 [easy]), ratings ranged from a mean of 5.9 (task 2, first injection) to 6.9 (task 4, second injection). Findings from the subjective feedback analysis are summarized in Figure 3.

Figure 3

Post‐task ease of task performance ratings (scale of 1 [difficult] to 7 [easy]). Error bars indicate the minimum and maximum ratings for each task. Task 1 was carried out by all participants (n = 60), whereas tasks 2–4 were completed by patients and non‐pharmacist healthcare professionals only (n = 45).

Discussion

This summative usability testing process was not associated with serious use errors in any of the four user groups tested, which included: (i) pen‐injector‐experienced patients; (ii) pen‐injector‐naïve patients; (iii) non‐pharmacist HCPs; and (iv) pharmacists. The present study supports a conclusion that the semaglutide single‐dose pen‐injectors are easy to use and are not associated with serious use errors. For the use errors that did occur, there was a trend for the root causes of these errors to be attributed to the visual similarity across the pen‐injectors and across the cartons for different doses of semaglutide, or to previous experience with other pen‐injectors. It should be noted that of the 13 patients showing use errors, nine were pen‐injector‐experienced, which indicates that even experienced patients should be advised to read the IFU when they first use a new pen‐injector. However, all participants who carried out injections administered the second injection correctly, indicating that the semaglutide pen‐injectors are easy to learn to use.

Findings from the subjective feedback analysis (on a scale of 1 [difficult] to 7 [easy]) implied that participants generally considered the pen‐injectors, IFU, and cartons easy to handle and differentiate from other dosage variants, as well as visually similar insulin and GLP‐1 RA products on the market, although these data should be interpreted with caution given that some participants made errors during the testing process. Mean scores were close to the highest value of 7 (easy), with the ease of use of the carton retrieval rated as 6.7, the first injection as 5.9, the pen‐injector retrieval as 6.6 and second injection rated as 6.9. The achievement of these high scores, despite the fact that participants did not receive any face‐to‐face training, is notable because, generally, HCPs have a relatively short amount of time available to train patients to self‐inject. This means that improving usability of pen‐injectors might reduce the burden on HCPs’ time without compromising users’ safety. This is particularly important in Japan, which has one of the highest rates of patient visits per physician globally, and injections are often carried out by HCPs . In addition, with the increased use of telemedicine for diabetes as a result of the COVID‐19 pandemic , the ease of use of these single‐dose pen‐injectors might enable some patients, who have previously required a clinic visit, to self‐inject in the home setting.

The results from the present study are consistent with those from a previous similar summative usability testing process that assessed the safety and effectiveness, ease of use and training requirements for the liraglutide 3.0‐mg pen‐injector . In that study, 234 participants (half of whom received IFU along with video‐based training) were asked to carry out six tasks followed by post‐task interviews on task difficulty. All participants interpreted the IFU correctly, they committed no potentially serious use errors and only one non‐serious error (a needle‐stick injury).

Among the strengths of the present study, the test method worked well, in the form of a simulation study to evaluate the critical task needs. Additional strengths were: (i) the inclusion of a selection of products in the differentiation testing, including both insulin and GLP‐1 RA products, to ensure that all components of the pen‐injectors for semaglutide were sufficiently different from each other and other marketed products; and (ii) the inclusion of a second injection, which in all cases was carried out correctly, because participants who failed to carry out the first injection correctly were able to realize what they had done incorrectly.

When interpreting the results of the present study, the inherent differences between participant performance observed in simulated environments and how users would carry out tasks in actual real‐life use should be considered. For example, injection into a pad, although intended to simulate an actual injection, deprives participants of some critical tactile feedback (e.g., needle piercing of skin, pain sensations) that might influence how they perform the injection into their body. The test setting was limited, because actual use might present other performance‐affecting environmental factors that the test method did not introduce. Additionally, the repetitive nature of carrying out two differentiation tasks (selecting the correct pen‐injector/carton from other, similar products) and two injections over a short (60 min) duration might result in some participants being more or less attentive to the task.

Performance anxiety caused by observed participation in a paid research activity could be hypothesized to lead some participants to be intensely focused on carrying out the task correctly, thereby potentially improving their performance. Alternatively, the setting might lead some participants to experience performance anxiety, which could adversely affect performance. In addition, because of the Hawthorne effect, some participants’ behavior might have been altered by the knowledge that they were being observed . Finally, treatment effectiveness (clinical improvement) cannot be assessed when injections are performed into an injection pad.

In summary, as medical devices become generally more complex and are used increasingly in home and public environments, it is necessary for untrained patients and caregivers to be able to use them safely . Therefore, a rigorous testing process is required for developing new medical devices, to improve design and help mitigate risks to patient safety. From this summative usability testing process in which participants received no training in the use of the semaglutide single‐dose pen‐injectors, we observed that all three variants of the pen‐injector: (i) were easy to use; (ii) could generally be differentiated from each other and other similar marketed products; and (iii) were not associated with any serious use errors. From a summative usability perspective, as per the 2016 US FDA guidance on applying human factors and usability engineering to medical devices , as well as Japan’s Industrial Standards’ T62366‐1 document , the semaglutide single‐dose pen‐injectors can be considered to be safe, because there were few use errors, and none were associated with risk of serious harm.

Disclosure

DCK has acted as a consultant for Dexcom, EOFlow, Fractyl Laboratories Inc., Lifecare Inc., Novo Nordisk, Roche Diagnostics and Thirdwayv. Emergo by UL (SB, AD, EE) received funding for this study from Novo Nordisk, as well as funding for other studies by Novo Nordisk. MQ, TS and SS are employees of Novo Nordisk A/S; MQ and TS own stock in the company.