Development and Pilot-Testing of Key Questions to Identify Patients' Difficulties in Medication Administration.

PURPOSE: The development and testing of key questions suitable to identify patients' difficulties with medication administration.
MATERIALS AND METHODS: We used a consecutive five-step process to draft key questions regarding 43 aspects of medication administration that can be difficult for patients who manage a complex drug treatment: Step 1) Identification of potentially error-prone characteristics of drug treatment (such as certain dosage forms) and initial draft of key questions. Step 2) Assessment of how comprehensible the questions are for patients. Step 3) Pre-testing of exemplary key questions with patients and monitoring of patient's actual medication administration behavior. Step 4) Evaluation by general practitioners of how well the questions may be integrated into actual patient visits. Step 5) Final approval of the questions in an expert panel. Thereafter, we pilot-tested exemplary questions with 36 patients (43 tests). In the course of this pilot-testing, the patients' answers to the key questions were tested against both their actual behavior during medication administration and against their answers to more general questions regarding potential difficulties with medication administration.
RESULTS: More than half of the key questions (N = 24/43) were revised at least once during the development process. During the pilot-testing, 55.8% of the pilot-tests (N = 24/43) revealed medication administration difficulties. It was observed that the key questions identified significantly more difficulties (N = 17) than the general questions (N = 8; P = 0.021, positive predictive value = 94.4% vs 88.9%). In one case, both a key question and a general question identified difficulties, which, however, was not confirmed during the drug administration demonstration, indicating a false positive rate of 5.3% in both cases.
CONCLUSION: We developed key questions aimed at detecting administration errors with a high specificity and a significantly higher sensitivity than general questions, suggesting that the resource-intensive demonstration of medication administration can be reserved for the detection of rarer and uncommon administration errors.

Plain Language Summary

The administration of drugs can be difficult for patients. Patients often do not know that they make errors or do not want to admit that they are having problems. We therefore developed specific questions to identify patients’ difficulties with respect to medication administration in a five-step development process that involved patients, general practitioners and other health-care professionals. Subsequently, we pilot-tested exemplary key questions with a group of patients. To do so, we compared the answers patients gave to a key question to their answers to more general questions that might be used in routine care (“Do you have any difficulties with using this drug/dosage form? Do you have any questions regarding the use of this drug/dosage form?”) and to how they actually administered their medication when asked to demonstrate this. In more than half of the cases patients made an error when administering their medication. We observed that patients’ answers to key questions corresponded better to their actual handling than their answers to general questions. The use of key questions might be a promising approach to identify patients who experience difficulties regarding the handling of their medication.

Introduction

In the ambulatory care setting, patients play a crucial role in medication management and particularly in medication administration. Hence, they may also be involved in medication errors.1 Up to 59% of patients were found to commit at least on medication administration error at home. Even higher rates have been reported in elderly patients using more than five medications daily and suffering from at least two illnesses.2

Medication administration errors most frequently happen unconsciously.3 A prominent factor amongst the reasons for such unintentional nonadherence is the complexity of a drug treatment,4–7 that may result from different treatment-related aspects, such as particular dosage forms and complicated or irregular dosage schedules.8,9 Often, it would be possible to reduce complexity of drug treatment by simple measures, for example by using a fixed dose combination to reduce the number of medications a patient has to take.10–12 However, optimization strategies in routine care are often missed because patients are unaware of their existence and hence do not request them. At the same time, health-care providers often underestimate patients’ difficulties with medication administration13–15 and fail to offer guidance. To efficiently assess the need for optimization strategies regarding the communication between patients and health-care providers, appropriate questioning techniques are needed. For several years now, specific interview guides containing distinct questions have been used to detect nonadherence.16,17 However, to our knowledge, no specific questions have yet been developed and evaluated that identify the difficulties patients have when handling a complex drug treatment. Up to date, physicians and pharmacists typically use general questions like “Do you have problems with the administration of your medication” which however seem not suitable to detect actual errors as a correct answer would require the patient to be actually aware of the problem he/she is having.18 Furthermore, patients tend to downplay the challenges of medication administration. In a previous work, we observed that patients were more likely to disclose difficulties when critical steps were addressed directly.3 Hence, we hypothesized that it would be more appropriate to ask (key) questions that clearly address a potential problem. Thus, patients can easily admit the problems they might experience and do not actually have to identify the problem themselves. We therefore developed key questions in a multistep process. We then pilot-tested exemplary key questions with patients in order to verify their suitability to identify the difficulties patients have with a complex drug treatment.

Materials and Methods

Development of Key Questions

Step 1) Initial Draft of Key Questions

In a previous work, different factors that increase the complexity of a drug treatment for patients have been identified.4 Based on these literature findings, preliminary key questions were drafted. Each key question referred to a potential problem, such as an error-prone step in the administration of a complex dosage form or a particularly difficult characteristic of a drug treatment (eg, the need to split tablets or a once weekly administration of a medication).

All key questions were developed in German; in a second step, the key questions and the results of the development process were translated into English by the authors. The translation was revised by a proficient English speaker.

Step 2) Assessment of Comprehensibility with Patients

In a second step, individuals not involved in the development process identified questions or terms that might be potentially difficult to understand or ambiguous for patients (1 pharmacist and 2 persons without a medical background). To ensure that a key question can be fully understood by patients, its comprehensibility was subsequently tested in the target population by cognitive pre-testing. Based on patients’ feedback, the key questions were adapted and a retesting following the same principles was conducted. The patients necessary for this step of the development process were recruited in a community pharmacy, a hospital outpatient clinic, and a clinical trials unit and a convenient sampling approach was chosen. Inclusion criteria were 18 years of age, cognitive as well as physical ability to participate and informed oral consent. Recruitment of patients was stopped when it became apparent that the key questions were comprehensible to the included patients. In total, 12 patients were included for the assessment of comprehensibility and re-testing was done with further five patients (Table 1).

Table 1

Sociodemographic Data of Participants in Step 2) and 3) of the Development Process and of Participants in the Pilot-Testing

	Step 2) Assessment of Comprehensibility	Step 2) Re-Testing	Step 3)	Pilot-Testing
Number of patients	12	5	27	36
Average Age ± SD [years]	56.3 ± 16.5	45.6 ± 9.6	62.1 ± 16.4	56.2 ± 20.5;1 Ma
Number of women (%)	7 (58.3)	4 (80.0)	14 (51.9)	21 (58.3)
Average Number of drugs ± SD	3.8 ± 3.4	2.2 ± 1.1	5.1 ± 2.7	4.3 ± 2.9; 5 Ma
Number of participants having a medical background (%)	3 (25.0)	1 (20.0)	3 (11.1)	4 (11.1)
Percentage of participants having received a training in the use of the respective dosage form [%] (n/total)a	Na	Na	Na	65.4 (17/26)
Percentage of participants using dosage form/splitting tablets regularly for more than three years [%] (n/total)b	Na	Na	Na	52.9 (18/34); 1 Ma

Notes: aOnly participants using any inhalers, ophthalmic preparations, or injection devices considered; bonly participants using any inhalers, ophthalmic preparations, or injection devices or splitting tablets considered.

Abbreviations: Ma, missing answer(s); Na, not applicable.

Step 3) Pre-Testing with Patients

In the pre-testing phase, a member of the study team asked patients a number of exemplary key questions with patients that were referring to complex dosage forms (ie, different types of inhalers, injection devices, ophthalmic preparations, liquid oral dosage forms), the splitting of tablets and a potential lack of training in dosage form use. Patients’ answers were tested against actual patient behavior. The patient first answered the key question and then demonstrated the actual corresponding administration step, either by using placebo medication and devices or through a detailed verbal description of a specific administration step. Patients were recruited in a community pharmacy. In order to be admitted to the pre-testing phase, they needed to regularly self-administer the respective dosage form or regularly split their tablets. Furthermore, patients had to be at least 18 years old, to show cognitive and physical ability to participate and to give written informed consent. For the sample, a variation regarding age and the key questions tested was sought and inclusion of patients was stopped when no additional findings were expected by further pre-testing. Eventually, 27 patients (Table 1) were recruited for this step of the development process with 24 patients participating in the pre-testing of more than one key question.

Step 4) Evaluation by General Practitioners

To find out whether the key questions can be easily implemented into clinical routine, the applicability of key questions was evaluated by six general practitioners, who were asked to give written informed consent regarding their participation in the evaluation. No further inclusion criteria were defined in advance. The participants were provided with all key questions and were asked to put these questions to patients during routine care visits whenever applicable. After having tested the key questions in practice, the general practitioners were invited to complete a paper-based questionnaire to evaluate the implementability into routine care. We used the following questions: “Do you believe that the questions will be understood by your patients?”, “Do you believe that the questions will be answered honestly by your patients?”, “Do you believe that the questions take too much of your time during a patient consultation?”, “Do you believe that the content of the questions fits into a typical consultation with a patient?”, “Do you believe that the questions can be applied directly in the patient interview without any rephrasing?”. In addition, general practitioners were asked (via paper-based questionnaire) in how many patient consultations they tested at least one of the key questions and to comment on the individual key questions No further information on patients was collected.

Step 5) Final Review

All questions were reviewed and discussed in conclusion by the study team and minor adaptions were made. Special care was taken to make certain that the results of the previous steps were accurately transferred and that the questions still reflected their initial purpose.

Piloting-Testing of Key Questions

To ensure that the final key questions were indeed suitable to identify patients who experience difficulties with their drug treatment (positive predictive value), six exemplary key questions as well as two general questions that were considered to be typical questions in patient consultations were tested against the actual patient behavior as gold standard to detect administration errors19,20 (similar to step 3 in the development process). Patients were asked to demonstrate or explain the administration of the medication in more detail, eg, by using placebo medication and devices (Table 2). Furthermore, patients were asked two general questions in order to evaluate the difficulties they were experiencing with their drug treatment (“Do you experience any difficulties concerning the use of [this medication/dosage form]?” “Do you have any question concerning the administration of [this medication/dosage form]?”). The order, in which data was collected was alternated by either asking the questions first and then demonstrating the application (or vice versa) and also by changing the order of the questions (first general questions and then key question or vice versa).

Table 2

Description of Pilot-Testing of Exemplary Key Questions

	Key Question	Second Key Questiona	Description
Inhalers (Metered dose inhaler)	Many patients find it difficult to pull the trigger of their inhaler and to inhale at the same time. Are you having any trouble using your inhaler?	For how many seconds after inhalation do you hold your breath?	● Demonstration of inhalation using placebo or own device.● Placebo were ensured to be equal to own drug.
Inhalers (Capsule-based/Other inhalers)	For how many seconds after inhalation do you hold your breath?	Do the capsules contain powder residues after inhalation?b	● Demonstration of inhalation using placebo or own device.● Placebo were ensured to be equal to own drug.
Ophthalmic preparations (drops)	Do you always succeed in inserting a drop into the conjunctival sac at the first attempt?	Do you always keep both eyes closed after the drop?	● Demonstration of eye drop installation using eye drops for moistening.● Different eye drops (and bottles were available).
Injection devices (prefilled)	How frequently do you change the injection site?	Na	● Demonstration of injection technique using a placebo insulin pen.● Injection in puppet.● Patients were asked to show the injection site of the following 7 days.
Tablet splitting	Do you find it difficult to split your tablets consistently into pieces that have the same size?	Na	● Patients were asked to choose a tablet that resembles their own tablet from a selection of tablets.● Patients were offered different tools (eg a knife, a tablet splitter, a scissor etc.) that might be used for splitting tablets.● Patients were asked to split tablets if it would be their own.● Patients were asked if splitting their own tablets was different.● Demonstration of splitting own tablet was also possible.
Meal-dependent administration	Do you find it difficult to coordinate the daily use of your medication with your meals?	Na	● Patients were asked which drugs have to be taken.● Patients were asked to mark their usual time for meals and their usual times for drug administration on two time frames (one for weekdays and one for the weekend).

Notes: aOnly applied when first key question did not lead to disclosure of difficulties; bonly applied in case of capsule-based inhalers.

Abbreviation: Na, not applicable.

In order to be included, patients had to be at least 18 years old and should potentially experience the difficulties mentioned in the key question (eg, using the respective dosage form independently). Furthermore, patients had to show cognitive and physical ability to participate and were asked to give written informed consent.

Patients were recruited in different places, such as a community pharmacy or local associations (eg, a choir). A variation sampling approach was used because a variation regarding age and the eligibility for the six different key questions was aimed at. A total of 36 patients were recruited (Table 1) with five of them being potentially affected by more than one difficulty and thus participating in the pilot-test of at least two key questions, resulting in 43 individual tests of key questions.

Specificity, sensitivity and positive predictive value of key questions and general question was calculated. Statistical analysis was performed using IBM SPSS Statistics Version 25. The chi-squared test was performed to compare the number of observations where the key question revealed difficulties a patient might experience with the number of observations where the general questions revealed difficulties. Furthermore, the number of observations in which the key question correctly predicted any difficulties the patient showed during observation was compared with the number of observations in which the general question correctly predicted any difficulties using chi-squared test.

Results

Step 1) Initial Draft of Key Question

A first set of 43 key questions was developed, subsequently discussed by the five pharmacists of the study team and revised several times to develop a first draft containing one key question for each factor that increases the complexity of drug treatment (). Short sentences and simple wording were used to ensure that the patients could easily understand the questions. Nearly all questions were either Yes-No questions (eg, “Is it a problem for you to take your medication several times a day in everyday life?“) or open-ended questions with a predefined correct answer (eg, “For how many seconds after inhalation do you hold your breath?“), thus facilitating time efficient integration in patient visits and interpretation of patients’ answers. Furthermore, normalizing statements were used to make the patient feel comfortable (eg, “Many patients report that they are uncomfortable with the rectal administration of a drug. Does the rectal application of your drugs also cause you problems?”).

Of the initial key questions, 21 were considered potentially difficult to understand and therefore considered for this step of the development process (). The testing with 12 patients revealed that only one of the key questions was difficult to understand for the patients with three of 12 patients (25.0%) being unable to explain the meaning of “rectal use”. For this reason, we reworded the question and replaced the term “rectal use” by naming the specific dosage form, eg, suppositories. The modified question was tested with five more patients and the wording was judged to be appropriate. The re-testing did not suggest any further revisions.

For inhalers and eye drops, the pre-testing revealed that administration of the placebo and the response to the key question were actually consistent. However, for some other, unfortunately unknown, reason all patients failed to correctly administer the placebo. Hence, to better represent the complex administration process of these dosage forms, we added a supplementary second key question.

General practitioners exemplarily integrated key questions into a total of 23 doctor-patient consultations.

The results of the respective questionnaire showed that the general practitioners mostly expected their patients to understand the questions and to answer them honestly. Furthermore, the content of the key questions in general was judged to fit well into a typical patient-doctor consultation without being overly time-consuming. However, the integration in routine care revealed that minor adaptions were necessary to simplify the wording ().

In particular, the wording of the key questions was revised and their clinical correctness was equally checked. For example, the word “eye” in the key question on ophthalmic preparations was changed into “conjunctival sac” to better reflect the correct administration process ().

Summary of Development Process

Over half of the questions (55.8%) were revised at least once during the development process (). The evaluation by general practitioners who tested whether the questions were easily implementable into routine care (step 4) proved particularly helpful with respect to the improvement of the key questions with approximately one-third (32.6%) being edited after this step of the development process.

Pilot-Testing of Key Questions

In 24 out of 43 tests, difficulties with medication administration, like an administration error, could be identified ( and ). In 18 tests, the key question revealed that the patient experienced difficulties in medication handling compared to nine tests where at least one of the general questions led to the disclosure of difficulties (chi-squared test, P = 0.063). Indeed, in 17 of 18 tests respectively 8 of 9 tests, when patients disclosed a problem according to a key question respectively a general question, this could be verified by the observation (positive predictive value = 94.4% vs 88.9%). Only one test (capsule-based/other inhalers) revealed a false positive answer to the key question (false positive rate = 5.3%), because no difficulties could be verified in the subsequent observation. Thus, patients` answers to the key questions predicted 70.8% of erroneous administrations in the observation compared to 33.3% that were predicted by the general questions (chi-squared test, P = 0.021; Figure 1).

Figure 1

Percentage of errors in medication administration identified by key questions and general questions per complexity factor.

Considering only those difficulties that were directly related to a key question (N = 15), the key questions predicted even 93.3% (N = 14) of difficulty-revealing, respectively erroneous, administrations. Moreover, in seven tests, difficulties could be identified in the observation that were not predicted by a key question (negative predictive value = 72.0%). In most cases, this concerned the use of an injectable (N = 6), but it also occurred in tablet splitting (N = 1).

Overall specificity and sensitivity of the key questions were 94.7% and 70.8%, compared to a sensitivity of 33.3% and a specificity of 94.7% of the general questions.

Discussion

We developed key questions to predict difficulties patients experience and that arise from the complexity of their drug treatment. To the best of our knowledge, there currently exists no approach describing the development of such specific questions for implementation in patient consultations during primary care, although the use and evaluation of purpose-fitted communication techniques such as patient-centered interviewing or motivational interviewing are subject of current research.21–25 Furthermore, the development and evaluation of written patient information has already been described in the literature.26–28 Therefore, we based the development process on the experiences made with a previously described approach to develop comprehensible patient leaflets, that combined desk-based and patient-centered measures26 and, consequently, focused on the inclusion of the real-world users of such key questions, namely patients and health-care professionals.

The development process revealed that the multistep approach was helpful to optimize the questions, as more than half of the questions were revised at least once in the development process. The multistep approach thus seems particularly useful to integrate different views and perspectives both from patients and health-care professionals and increase the likelihood that each question is at least reviewed once.

The pilot-testing of exemplary key questions showed that they predicted difficulties with a drug treatment correctly for the respective step of administration in all but one case. It might be hypothesized that the close-ended questions invited patients, to admit the difficulties they have, because the questions already imply that problems frequently occur. In the context of measuring adherence, it has already been shown that negatively framed closed-ended or normalizing (eg, asking for the last time a dose has been missed) questions that explicitly ask for nonadherence led more frequently to the patient’s disclosure of nonadherence than other types of questions.29 Furthermore, the key questions identified significantly more difficulties than the general questions. This might emphasize the finding that patients are generally not aware of the mistakes they make regarding the use of their medication and that general questions presuming error awareness, are not suitable for the detection of patients, who experience difficulties when handling their drug treatment. Hence, only a key question that specifically addresses a potential problem allows the identification of patients with difficulties. This is in accordance with a previous work, where we already found out that patients are more likely to disclose any difficulties when critical steps of medication administration were directly addressed by a question.3

The key question even predicted difficulties in more than two thirds of the tests where an administration error was observed, regardless of whether the error made was directly addressed by the key question or not. Therefore, a differentiation between the key questions (respectively the complexity factors) considered is necessary, because the administration of complex dosage forms consists of multiple steps1 and key questions can only address specific error-prone steps. Injectables were particularly often the reason for difficulties, that were not addressed and, hence, not identified by the key question. Therefore, it seems necessary to add another key question about injectables, like we did with respect to eye drops and metered dose inhalers to better reflect the complex administration process. However, most key questions – except for the ones referring to dosage forms – refer to a single factor that increases complexity of drug treatment (eg, intake with meals).

The pilot-testing suggested that key questions may detect difficulties with medication administration with a high specificity and a higher sensitivity than general questions. However, in some cases, other methods such as the demonstration of medication administration should be considered to identify patients who experience difficulties in handling their drug treatment. Such resource-intensive methods can be limited to complex multistep administration processes where not all error-prone steps can be addressed by a key question. The use of specific key questions might be a promising approach that can be implemented in routine care to identify patients that experience difficulties when handling their drug treatment. Although their implementability in patient consultations was initially tested with general practitioners as part of the development process, the use of such questions by other health-care professionals, such as pharmacists and nurses, or resident specialized physicians is also conceivable. Further research is needed to evaluate the use in routine care with more patients and health-care professionals to confirm the results. A comparison of such specific key questions with routine care or other types of questions would be helpful (eg, open questions).

This work has several limitations. First, we did not compare the development process to conceivable alternative processes for the development of key questions. However, to our knowledge, there are currently no such standard techniques available. Second, all key questions were formulated as closed-ended questions (respectively questions with a predefined correct answer) and thus, other types of questions, eg, open-ended questions, were not considered. In general, in order to reach a patient-centered communication, open-ended questions should be used.30,31 Hence, it might be promising to combine such open-ended questions with key question that specifically address potential problems in order to comprehensively capture the difficulties of an individual patient when it comes to the handling of his or her drug treatment. Furthermore, not only the key question but also the general questions that were used in comparison to the key questions were closed-ended. The questions could have been compared to other types of questions, such as open-ended questions. Third, the pilot-testing was done using only exemplary key questions and especially complex dosage forms were considered because the actual patient behavior can easily be assessed by using placebos. Fourth, only a limited number of patients and health-care professionals could be included in the development process and the pilot-testing and accordingly the generalizability of the results is somewhat limited. Moreover, the patients’ health literacy was not assessed and some patients involved had a medical background, yet both might have influenced the results. Fifth, the identification of difficulties with a drug treatment during the pilot-testing might be influenced by the observational design and the questions asked. However, even with this limitation, a considerable number of patients who experience difficulties with medication administration could be identified.

Conclusion

In conclusion, we developed key questions in a five-step consecutive process to standardize the identification of patients’ difficulties in handling a complex drug treatment. The key questions predicted difficulties better than general questions suggesting that such key question might be helpful to identify patients with a need for help. To the best of our knowledge, this is the first set of key questions that was specifically developed to detect patients’ difficulties in handling a complex drug treatment in a standardized, error-oriented manner. The development process took into account established approaches and drew on preliminary work from related pitfalls in drug treatment such as nonadherence.