Evaluating Patient Safety And Ease Of Use Of A Novel Connection-Assist Device For Peritoneal Dialysis.

INTRODUCTION: It is estimated that there are currently over 3 million patients receiving dialysis treatment worldwide. With effective pre-dialysis counselling, a majority of patients choose the home-based therapy peritoneal dialysis (PD) but only approximately 11% of prevalent dialysis patients use this modality. Connection-assist devices can overcome the challenges posed by decreased manual dexterity and/or visual acuity, and can allow more patients to be treated with home-based therapies. As part of the CE marking authorization, a connection device has been evaluated for safety and ease of use in a usability study.
METHODS: Fifteen patients and nine carers volunteered in this study, ranging from 23 to 86 years in age and from 0.3 to 24 years in experience in the PD therapy. The operating cycle consisted of eight tasks, each having several handling steps. The data analysis focused on the task effectiveness and the subjects' subjective feedback from the NASA task load index (N-TLX) questionnaire and semi-structured interviews.
RESULTS: Of 1248 handling steps performed in total, 38 use errors were observed and discussed with the subjects. This equates to 97% of all handling steps being performed safely and easily. In all six dimensions of the N-TLX, more than 50 percent of subjects rated the task load 50 points or less on the scale. Regarding the handling of the device, 13 of 15 of the patients and 8 of 9 of the carers gave positive feedback. DISCUSSION: Safety and ease of use was demonstrated by evaluating task effectiveness (97% SU), interviews and N-TLX. Additionally the study provided valuable individual user feedback, which will inform the final design of the system for PD. The majority of patients and carers gave positive feedback regarding use and handling of this connection device. Innovative connection devices in general promise to reduce the barriers to using this home-based dialysis treatment.

Introduction

End-stage renal disease is a severe chronic illness that requires treatment with renal replacement therapy (RRT) in the form of dialysis or renal transplantation. It is estimated that there are currently over 3 million patients receiving dialysis treatment worldwide.1 The number of prevalent patients receiving RRT will continue to increase due to aging,2 diabetes3 and hypertension.4

The two dialysis modalities available are hemodialysis (HD) and peritoneal dialysis (PD). Both modalities have similar survival rates.5 Peritoneal dialysis is associated with improved health related quality of life and reduced health resource costs compared to HD and is therefore promoted by most healthcare systems.6

With effective pre-dialysis counselling a majority of patients choose PD as their dialysis modality7 but only approximately 11% of prevalent dialysis patients use this modality.1Many factors contribute to this disparity including access to PD technology, patient factors and PD modality survival. PD is a patient or carer delivered home-based therapy and a key feature of PD is the requirement for patients and/or their carers to interact with PD equipment. This can be a limiting factor in frail, elderly or disabled patients.

To evaluate the interaction of patients with current PD treatment technology a need finding study with eye tracking technology indicated that the use of PD in frail individuals, irrespective of age, can be facilitated by connection-assist devices.8 This outcome is supported by literature.9–11

Connection-assist devices can overcome the challenges posed by decreased manual dexterity and/or visual acuity, and can allow more patients to be treated with home-based therapies.9,10

The growth in the RRT population is primary in the elderly age group, a population with a high frailty and co-morbidity burden. Facilitating PD in this group of patients requires improved PD technology with greater accessibility tailored to patient needs.

It is proposed that by making PD treatment simpler, more standardized and better protected from infections, patient access to PD could be increased significantly.

Following evaluation of specific barriers to the use of PD including feedback from established PD patients, a novel connection-assist device has been developed.

As part of the CE marking authorization, the connection device has been evaluated for safety and ease of use in a study at University Hospitals Birmingham NHS Foundation Trust involving 24 patients and carers.

Materials And Methods

The evaluated device is designed to assist peritoneal dialysis patients performing their daily dialysis treatment either for APD or for CAPD (see Figure 1, ). The dialysis consumables from Baxter are suitable for this device. The device facilitates the connection and disconnection of the dialysis line to the PD catheter transfer set. The connections are performed inside the device in a protected area, and the patient does not need to touch the transfer set or the dialysis line while they are unprotected. In addition, the breaking of the dialysis line frangible and the clamping of the dialysis line are replaced by pushing a button.

Figure 1

Connection-assist device as used in the study and later in the market with inserted dialysis lines and transfer set.

The operating procedure of the device consists of multiple tasks including preparing and loading the transfer set and dialysis line into the device, performing the handling steps related to the PD therapy with the device and finally unloading the used material from the device (Figure 2). Each task comprises several handling steps. During the study, the operating procedure with the device was simulated and the subjects were connected to a dummy catheter, which was attached to an apron.

Figure 2

Percentage of use errors (UE) in each task and ratio of steps conducted with ease and with failed steps.

This study recruited established PD patients and carers involved in the delivery of PD. Carers included healthcare professionals and lay carers.

For the patient group, the study was conducted at the Birmingham NIHR/Wellcome Trust Clinical Research Facility (CRF). This clinical research facility was fully staffed and equipped to support patients. For the carers, the study was conducted at the Medical Device Testing and Evaluation Center, Birmingham.

The study design was divided into three one hour long parts. Firstly, a training session introduced the operation of the device. Secondly, a break of one hour allowed relaxation and recovery. Thirdly, a test was carried out to determine whether the subject could use the device independently with the aid of a quick start guide and the device manual.

At the beginning of the training, the subjects were welcomed and thanked for their participation. They were asked to read and sign the consent form if they agreed. The training consisted of two cycles in which the moderator guided the subject and three cycles in which the subject guided the moderator through the complete handling of the device.

After the break, the subjects were asked to carry out a complete handling cycle themselves. Subjects could use the quick start guide and the device manual as help. After the unsupported handling cycle, subjects were asked to complete the NASA Task Load Index (N-TLX) questionnaire.12 In the meantime, the moderator left the room to obtain information on observed use errors. In a final semi-structured interview, the subjects were asked about the reasons for the observed use errors.

The final part was a thank you for the participation.

For each subject, one moderator and one observer were involved in the study. The moderator was the interface between the subject and the device both in training and in testing. This person was employed by an external agency to avoid any influence on the subject. The observer saw the handling with the help of a live feed from a separate room. He evaluated the handling steps in the categories of “safe use” and “use error” according to IEC 62366–1 (2015). For each use error, the observer described the observed situation from his point of view.

In addition to evaluating the effectiveness of the task performance, the subjects were interviewed in a semi-structured way. Here the observations of technique errors were discussed with the subjects in order to identify the specific causes. In this interview, the subjects were also asked for their feedback on the general ease of use of the device.

In addition, the subjects were asked about the task load in six different dimensions in the standardized N-TLX. Task load is a hypothetical construct that represents the cost incurred by a human operator to achieve a particular level of performance. The dimensions of the N-TLX are mental, physical and temporal demands as well as performance, effort and frustration in the task. Each dimension is evaluated on a 20-point scale. This scale is then transferred to the task load index, which is a scale between 0 and 100 points. To classify the results of the N-TLX, Eitrheim and Fernandes (2016) state “Workload levels below 50 were perceived as acceptable.”14

Results

The characteristics of the study subjects are shown in Table 1. The mean age of the patients was 54.0 years including one third being older than 65 years (mean 73.0 years). With a mean of 47.9 years, the age of the carers was higher than expected. Subjects varied both in age and experience distribution. In the patient group, experience in treatment varied from beginners to very experienced. For the carers the experience level varied between 1 and 24 years. In addition to end-stage renal disease, two of the patients suffered from arthritis and Reynard’s syndrome as comorbidities.

Table 1

Demographics Of The Subjects. Collected Data Is Number, Gender Ratio, Distribution Of Age And Experience In The Peritoneal Dialysis Therapy Treatment

	n	Percentage Female	Distribution Of Age	Mean (Min/Max) Age [y]	Mean (Min/Max) Experience [y]
Patients	15	40%	33% (> 65 y)	54.0 (23/86)	1.8 (0.3/5)
Carers	9	89%	89% (> 40 y)	47.9 (23/66)	9.2 (1/24)

Note: Comorbidities: Arthritis, Reynaud’s syndrome.

Each of the 24 subjects performed 52 handling steps resulting in 1248 evaluated handling steps. In total 38 use errors were observed and discussed with the subjects. This equates to 97% of all handling steps being performed safely and easily.

The error rate in the single tasks varied between zero and five percent (see Figure 2). In the task of preparation and insertion, use errors occurred in 5 percent of the handling steps performed. The observed use errors were, for example, insufficient condition checks and insufficient cleaning of the device in the preparation task or incorrect insertion of material into the device. The discussion of these use errors in the interview also revealed country-specific differences in UK compared to Switzerland in the handling cycle in standard PD treatment such as disinfection or use of gloves.

In all six dimensions of the NASA Task Load Index, more than 50 percent of subjects rated the task load 50 points or less on the N-TLX scale (see green color in Figure 3). Results varied in all dimensions across the scale. For the dimension effort, 50 percent of subjects rated the task load 50 points or less. For the remaining dimensions of mental demand, physical demand, temporal demand, performance, and frustration, this acceptable task load level is achieved for 63, 75, 71, 67, and 71 percent of subjects, respectively (see Figure 3).

Figure 3

Results from the NASA task load index questionnaire. It shows the rating distribution of the subjects in six dimensions. The NASA task load index is on a scale from 0 to 100 points, represented by color in this figure.

With regard to ease-of-use, the subjects gave their opinion in a semi-structured interview. Among the patients, 13 out of 15 gave positive feedback on the handling of the device and confidence in its handling. For the caregivers, the score was 8 out of 9. Additionally, 8 out of 15 patients offered feedback that they believed that the device would result in a reduction of infection risk.

Discussion

Of 1248 handling steps performed, only 38 contained errors, which is equivalent to 3%. The most error-related handling steps were the preparation of the medical device system and the loading of the material, as shown in the results. The country-specific differences indicate the need to adapt the printed material such as manuals and quick start guides to the needs and practices of the patients being treated. The difficulties in loading the material illustrate the need to increase training of this important step and to develop the printed material to achieve this.

Regarding the task load, the majority of subjects rated lower than 50 points on the scale of the N-TLX questionnaire, which is regarded as acceptable. The main task load comes from the effort required to use the device. This may be due to a lack of experience with the device. Subjects practiced using the device for the first time in this study, which represents relatively brief exposure and training. Standard PD training requires between several days and two weeks. The results of the other remaining dimensions show a perceived low level of frustration, temporal and physical demand. This indicates a low task load.

This conclusion is supported by the high level of positive feedback on the user-friendliness in the interviews. For some patients, this reduction in the level of stress also appears to be due to the perceived increase in infection prevention by the device.

Since the handling cycle was a simulation instead of a real therapy treatment, there are some limitations in this study. In order to overcome the feeling of simulation and to create as high a level of reality as possible, different approaches were used. Firstly, originally packaged and new dialysis consumables were used. Secondly, the subjects were connected to the device system via an original catheter attached to an apron. Thirdly, in the unassisted test run, the subjects could only use the quick start guide and the device manual as support.

A further limitation is the break of one hour between training and test. In later use, patients have about three to four hours between two fluid exchanges. The Food and Drug Administration stated in its 2016 Applying Human Factors and Usability Engineering to Medical Devices guide: “In some cases, giving the subjects a break of an hour (e.g., a ‘lunch break’) is acceptable”.13–15 In this study, subjects were invited to have a cup of tea, a small lunch or to watch television during the break. This was intended to decay retention of training.

Regarding the evaluation of the task effectiveness by the observers, in some cases it was difficult to assess whether control steps were carried out with the given third-person perspective from the live feed. Interviews were necessary to clarify these open questions together with the subjects. In some cases, the subjects could not remember making certain mistakes or were convinced that they had not made them. This made it difficult to find the underlying causes in these situations.

Another limitation was the patient group characteristic for age with a mean of 54.0 years being rather on the younger side. One third of this group was older than 65 years with a mean of 73.0 years.

Finally, the study of 24 subjects may seem insufficient to reflect thousands of potential future users of the device. Faulkner et al (2003) showed that a number of 20 users detected at least 95% of all use errors.16 In addition, subjects in this study showed a wide variety of age and experience in PD therapy in order to increase the probability of finding all potential use errors.

Conclusion

User groups of patients and carers participated in this simulation handling study. Subjects varied in age and experience thus representing a broad range of potential users.

For the subjects of this study, safety and ease of use was demonstrated by evaluating task effectiveness (97% SU), interviews and N-TLX. Additionally the study provided valuable individual user feedback, which will inform the final design of the system for PD.

The majority of patients and carers gave positive feedback regarding use and handling of the device and more than half believed the device would result in a reduction of infection risk.

Based on the findings of this and future studies, the connection device will be continuously evaluated and improved together with patients and carers. Aiming for even higher variation in age, experience and comorbidities of the patients promise additional insights.

In general, innovative connection devices can have the potential to reduce the barriers for patients that want to start or want to continue with home-based RRT. The barriers particularly concern frail, elderly or disabled patients, and help by such an assisting device may expand access to PD in this growing population.