Drug-Associated Risk Tool: development and validation of a self-assessment questionnaire to screen for hospitalised patients at risk for drug-related problems.

INTRODUCTION: Identifying patients with a high risk for drug-related problems (DRPs) might optimise the allocation of targeted pharmaceutical care during the hospital stay and on discharge.
OBJECTIVE: To develop a self-assessment screening tool to identify patients at risk for DRPs and validate the tool regarding feasibility, acceptability and the reliability of the patients' answers.
DESIGN: Prospective validation study.
SETTING: Two mid-sized hospitals (300-400 beds). PARTICIPANTS: 195 patients, exclusion criteria: under 18 years old, patients with a health status not allowing a meaningful communication (eg, delirium, acute psychosis, advanced dementia, aphasia, clouded consciousness state), palliative or terminally ill patients.
METHODS: Twenty-seven risk factors for the development of DRPs, identified in a previous study, provided the basis of the self-assessment questionnaire, the Drug-Associated Risk Tool (DART). Consenting patients filled in DART, and we compared their answers with objective patient data from medical records and laboratory data.
RESULTS: One hundred and sixty-four patients filled in DART V.1.0 in an average time of 7 min. After a first validation, we identified statements with a low sensitivity and revised the wording of the questions related to heart insufficiency, renal impairment or liver impairment. The revised DART (V.2.0) was validated in 31 patients presenting heart insufficiency, renal impairment or liver impairment as comorbidity and reached an average specificity of 88% (range 27-100) and an average sensitivity of 67% (range 21-100).
CONCLUSIONS: DART showed a satisfying feasibility and reliability. The specificity of the statements was mostly high. The sensitivity varied and was higher in statements concerning diseases that require regular disease control and attention to self-care and drug management. Asking patients about their conditions, medications and related problems can facilitate getting a first, broad picture of the risk for DRPs and possible pharmaceutical needs. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

The Drug-Associated Risk Tool (DART) is a patient self-assessment risk screening tool, based on a selection of risk factors for the development of drug-related problems (DRPs), previously identified in a combination of literature search and the opinion of a multidisciplinary expert panel.

DART should enable clinical pharmacists to identify patients at risk for DRPs and target their clinical pharmacy activities to patients who benefit the most thereof.

A first validation of DART showed good acceptability and feasibility and a satisfactory reliability of patient’s answers.

The low prevalence of some risk factors hinders clear conclusions about the validity of the respective statements in DART.

Introduction

Drug-related problems (DRPs) are defined as an event or circumstance involving drug therapy that actually or potentially interferes with desired health outcomes.1 The term ‘DRPs’ has mostly taken hold in European countries where English is not the native language, while pharmacists in the USA tend to use the term ‘medicine-related problems’ or ‘drug-therapy problems’ instead of DRPs.2 DRPs are a frequent issue among hospitalised patients, leading to patient harm and increased healthcare costs.3 Many unplanned admissions are medication related4 and a considerable number could be prevented.5 Complexity and often poorly designed processes foster the development of DRPs inside and outside of the hospital. Unsurprisingly, a remarkable number of patients experience adverse drug events after discharge.6 A study from Switzerland showed that 36% of all discharge prescriptions contained technical DRPs like unreadable prescriptions, missing drug form and package size, and 19.6% showed clinical DRPs like drug–drug interactions, inappropriate drug choice and wrong dosing.7

Clinical pharmacy services in hospitals have been shown to increase patient safety by reducing medication errors and adverse drug events, as well as adverse drug reactions. They increase medication appropriateness, improve patients’ knowledge about drug therapy and adherence, and finally reduce the length of hospital stays.8 Limited resources and capacities force clinical pharmacists to target their clinical activities to those patients who are most likely to benefit therefrom, or in other words, to those patients who are at the highest risk of experiencing DRPs, and in consequence, adverse drug events. An effective screening tool to identify high-risk patients might prove a successful approach. The literature provides risk factors for the development of DRPs such as polypharmacy, renal impairment or the use of non-steroidal anti-inflammatory drugs.4 9 10 The literature is replete with assessment tools, which focus on various combinations of risk factors for DRPs. They may be created either for a specific group of patients (eg, those with renal impairment,11 geriatric patients,12–16 patients with prescribed medication for cardiovascular disease17) or for a special environment (eg, in an emergency department,18 primary care19 20). The tools may also need special resources to be applied in the hospital (eg, computerised patient files21). Screening tools often have the disadvantage of being time and personnel intensive; some are hardly applicable without electronic data. Many have not been validated.22

Therefore, we decided to develop a new risk assessment tool. The ‘Drug-Associated Risk Tool (DART)’ should serve as a reliable, easy-to-use screening instrument to detect patients at risk for DRPs. Developed as a self-assessment questionnaire for the patients, DART should save personnel resources and time.

In a previous study,23 we identified 27 risk factors for the development of DRPs, which provided the basis of the self-assessment questionnaire. Risk factors identified in relevant literature were supplemented with results from qualitative research methods: We conducted a Nominal Group Technique with practitioners to ensure relevance in everyday practice and to identify risk factors possibly neglected in quantitative research methods.

The aim of this study was to create a self-assessment questionnaire out of the identified risk factors and to validate the questionnaire regarding feasibility, acceptability and the reliability of the patients’ answers by comparing them to reference information retrieved from medical charts.

Methods

Development of the questionnaire

Figure 1 shows the development process of the questionnaire.

Figure 1

Development process of the questionnaire. BMQ, Beliefs about Medicines Questionnaire; DART, Drug-Associated Risk Tool; DRP, drug-related problem; MMT, Micro-Mental Test; NGT, nominal group technique.

Twenty-seven risk factors for the development of DRPs, identified in a previous study,23 provided the basis of the self-assessment questionnaire, DART. With the intention of creating a questionnaire for patients, we formulated a statement for each risk factor that could be answered by medical laypersons (cf. table 1).

Table 1

Risk factors, their corresponding statement in the Drug-Associated Risk Tool (DART) and criteria to evaluate correlation between the answers in DART and objective data

Risk factor	Corresponding statement in DART	Acceptance criteria for correlation
Language issues (eg, migration background)	1	No comparison with objective data
Polymorbidity: divided in subcategories
Renal impairment	2	Diagnosis of renal impairment and/or GFR <60 mL/min for at least 3 months33
Hepatic impairment	3	Diagnosis of hepatic impairment and/or chronic hepatitis and/or hepatic cirrhosis
Chronic cardiac disease	4	Diagnosis of chronic cardiac disease (heart failure, coronary heart disease, arrhythmias)
Chronic respiratory disease	5	Diagnosis of asthma or chronic obstructive pulmonary disease
Diabetes	6	Diagnosis of diabetes mellitus type 1 or 2 or diabetes caused by steroids
Cognitive impairment/dementia	7	Diagnosis of cognitive impairment or dementia or 25/30 points in the Mini-Mental State Examination34 or <14/20 points in the Micro-Mental Test26
The patient takes medication(s) besides the prescribed ones (eg, over-the-counter, vitamin supplementation)	8	No comparison with objective data possible
Polypharmacy	9	The patient takes more than five medicines when admitted to the hospital
Antiepileptic, anticoagulants, non-steroidal anti-inflammatory drugs (NSAIDs), combination of NSAIDs and anticoagulants, digoxin, corticosteroids, diuretics, tricyclic antidepressants, anticholinergic drugs, benzodiazepines, opiates/opioids, oral antidiabetics/insulin, medication with a narrow therapeutic range	10	The drug is present on patients’ medication list at hospital admission
Non-adherence	11	No comparison with objective data24
Earlier experience of adverse drug reactions	12–16	Negative total score in both—the statements 12–16 and the Beliefs about Medicines Questionnaire (BMQ)25 or a positive total score in both—the statements 12–16 and the BMQ25
Missing information, partial knowledge of the patient, the patient does not understand the goal of the therapy	17	No comparison with objective data
Impaired manual skills—causing handling difficulties	18	No comparison with objective data
Visual impairment/impaired eyesight	18	No comparison with objective data
Difficult to handle medication	19	Medicines for parenteral, transdermal or inhalative application at time of hospital admission

GFR, glomerular filtration rate.

We covered the risk factor ‘non-adherence’ with an adapted question retrieved from the adherence risk prediction tool of Krousel-Wood,24 a validated self-report 4-item questionnaire used to measure adherence. A validated self-report four-item questionnaire used to measure adherence. Risk factors with regard to patients’ concerns about medicines were covered by using five questions from the Beliefs about Medicines Questionnaire (BMQ),25 a questionnaire that comprises two five-item scales assessing patients’ opinions about the necessity of prescribed medication for controlling their illness and their concerns about the potential adverse consequences of taking it.

Amateur test

Prior to the study, we conducted an amateur test and asked 10 medical laypersons from the personal environment of the authors (no patients) to fill out DART. We did not provide any support during its completion. We asked the participants for their judgement concerning the comprehensibility of the statements and edited issues that arose within the statements. In cases of ambiguity, the study investigators (CPK, MLL, NM, DS) discussed and clarified the unclear statements.

Validation of the questionnaire

Study design and setting

For the prospective validation study, we recruited patients in two mid-sized hospitals with 300–400 beds each. We recruited on orthopaedic, geriatric and internal medicine wards in order to validate the questionnaire in very diverse patients.

Patient selection

Eligibility criteria were stationary hospitalisation, age over 18 years and ability to speak German in order to communicate with the investigator. We excluded patients with a health status not allowing a meaningful communication (eg, delirium, acute psychosis, advanced dementia, aphasia, clouded consciousness state) as well as palliative or terminally ill patients. We included patients suffering from mild dementia in case a meaningful communication was possible.

Study flow

During a predefined period, the investigators (CPK, DS, NM) and two additional trained clinical pharmacists met with every hospitalised patient on the included wards who met the inclusion criteria. They informed each patient orally and with an informational letter about the study. After giving informed consent, the patient received DART and filled in the questionnaire independently, that is, the investigator gave no assistance in filling in the questionnaire. If a patient had impaired manual skills, the investigator was only allowed to assist with writing. When finished, the investigator asked the patient five questions about the structure and content of DART in order to see if the questionnaire was easy to understand and not too intrusive. Furthermore, the investigator interviewed the patient in detail with regard to the patient’s attitude towards health and medicine. Validated questionnaires were used to investigate concerns and beliefs towards medicines (BMQ25) and mental health (Micro-Mental Test (MMT)26). Participation in the study was voluntary, the investigators offered no inducement or payment for subjects to participate. The patient was allowed to terminate the interview at any time without stating a reason.

Pretest

With a first draft of DART, we conducted a pretest with five inpatients. The procedure followed the same study flow we determined for the validation study (see the Study flow section). This pretest with inpatients served as an opportunity to correct any remaining issues of comprehensibility or ambiguity.

Data collection and analysis

All data were processed anonymously. In order to ensure traceability, we assigned each patient a unique identifying number coding for the particular hospital/ward/investigator/patient.

We used IBM SPSS Statistics Software, V.22 for data analysis. We evaluated sensitivity, specificity and prevalence of each question of DART by comparing the subjective answers in DART with objective data from medical records (diagnosis, laboratory values and medicines at entry) and answers from the BMQ25 and the MMT.26 Acceptance criteria for correlation of subjective and objective data were defined a priori (cf. table 1). In addition we calculated the negative and positive predictive values for each question in DART. Missing data were excluded from analysis.

Revision of statements

Statements with an unsatisfactory performance within reliability testing of the questionnaire (ie, sensitivity <0.5 and possible poor patient understanding) were revised in their wording. In order to find a terminology patients may be familiar with, we used official patient information leaflets (PILs) of selected drugs, which are either contraindicated or in need of a dose adaptation in presence of the risk factor assessed by the statement under revision. These PILs are contained in the official packages of the medicines, are created by the manufacturer and are bound to the Swiss legal requirements concerning readability and understandability. We extracted and analysed the wording from these PILs which is used to describe the risk factor to patients and phrased new statements. We retested the new statements with the same study flow. In this cycle, we only recruited patients presenting one or more of the risk factors assessed by the statements under revision.

Results

The first page of DART consists of items concerning the presence of diseases and high-risk medicines. The second page includes items reflecting the patient’s attitude towards his/her medicines and statements about medication management and handling difficulties. The 10 non-patient participants from the amateur test had no difficulties completing the questionnaire, and only minor adjustments in wording were necessary.

The pretest with five inpatients did not reveal any additional issues.

During ward visits, we approached 208 eligible patients. One hundred and sixty-five (79.3%) consented to participate, and we were able to complete 164 patient interviews (cf. figure 2). The median age was 74 years (range 20–95) and 49% of participants were women. The mean number of drugs per patient at time of admission was 4 and ranged from 0 to 19. Fifty-six patients (34%) came from the geriatric ward with a mean age of 81 (40–95) years and a mean number of drugs of 5 (0–19). Sixty-eight patients (42%) were from the medical ward with a mean age of 65 (20–91) years and a mean number of drugs of 3 (0–15) and 40 patients (24%) were orthopaedic patients with a median age of 67.5 (20–91) years and a mean number of drugs of 4 (0–10).

Figure 2

Flow chart of the validation study. DART, Drug-Associated Risk Tool.

After 51 interviews, we reduced the number of questions. We eliminated the questions about feasibility and understandability of DART, because we had enough meaningful data with a clear conclusion. For the same reason, we stopped answering the BMQ questionnaire that we used for comparison with the answers from DART. This allowed us to shorten the duration of the patient interview.

On average, it took patients 7 min to complete DART by themselves. None of the patients experienced any of the statements as bothersome or too intrusive on his privacy. Ten out of 51 patients (19.6%) showed some difficulties in completing the questionnaire, 7 (13.7%) did not understand the wording of a statement and in three cases we had no clear statements what the difficulties were.

DART questions of the version V.1.0 reached specificity values from 27% to 100% and sensitivity values from 21% to 100%. Positive predictive values varied between 26% and 100% and negative predictive value varied between 20% and 100%. Regarding the intake of over-the-counter (OTC) drugs, 85 patients (35%) affirmed, 103 patients (63%) denied and 3 patients (2%) gave no answer. On the question ‘I feel well informed about my medication’, 85 patients (52%) answered with ‘strongly agree’, 45 (27%) agreed, 18 (11%) disagreed, 3 (2%) strongly disagreed and 13 patients (8%) gave no answer. Ten patients (6%) named difficulties with tablet splitting, 17 (10%) mentioned swallowing difficulties, 5 patients (3%) affirmed difficulties with visual recognition and 122 (74%) stated no such difficulties. Fifteen answers (9%) were missing. One hundred and twenty-five patients (74%) managed their medication by themselves, 12 (7%) had a relative or a friend who did the management, 15 patients (9%) named a home care person as their medication manager and 16 patients (10%) gave no answer. Sixteen patients (10%) indicated the use of an inhaler, 15 (9%) the use of a transdermal therapeutic system and 18 (12%) the use of a syringe for self-injection. One hundred and one patients (62%) did not use any of these application forms and 20 (12%) gave no answer.

Revision of statements

Initially, statements about heart insufficiency, renal impairment and liver impairment showed low sensitivity (0.43, 0.28 and 0.33, respectively) due to possibly poor patient understanding. The PILs of in total 134 medicines, either contraindicated or in need of dose adaptation in presence of heart insufficiency, renal impairment or liver impairment, were used to identify expressions most frequently used to describe these conditions to patients. For DART V.2.0, the statements were changed accordingly: ‘I am suffering from a chronic renal disease’ was changed to ‘I have a restricted kidney function/kidney dysfunction/kidney disease’, ‘I am suffering from a chronic cardiac disease’ was changed to ‘I have a heart weakness/heart performance weakness’ and ‘I am suffering from a chronic hepatic disease’ was changed to ‘I have a liver disease/liver dysfunction’ (cf. figure 3). These expressions were directly translated from German to English and may be written differently in English-speaking countries.

Figure 3

Drug-Associated Risk Tool (DART). Drug names mentioned in the section ‘My medicine’ correspond to the most commonly used medicines in the respective therapeutic class from the Swiss market.

A total of 31 patients (median age: 82 years (range 59–96 years), 61% women), each presenting heart insufficiency, renal impairment or liver impairment as comorbidity, filled out the revised questionnaire (cf. figure 2).

After the second comparison to medical records, the sensitivity of the reworded item ‘heart failure’ improved from 0.43 to 0.80, while the specificity dropped from 0.96 to 0.60. Similarly, the sensitivity for ‘renal insufficiency’ ameliorated from 0.28 to 0.38, while the specificity was lowered from 0.98 to 0.80. The small sample size combined with the low prevalence of liver insufficiency prohibited the evaluation of the refined statement covering liver insufficiency. With these modifications DART V.2.0 reached an overall sensitivity of 67% with an overall specificity of 88% (cf. table 2).

Table 2

Sensitivity and specificity of the single statements of DART V.2.0

Statements or questions of DART	Number of answers (n)	Missing data	True positive	False positive	True negative	False negative	Prevalence of the Rf (%)	Sensitivity (%)	Specificity (%)	Positive predictive value (%)	Negative predictive value (%)
I have a restricted kidney function/kidney dysfunction/kidney disease	31*	0	10	1	4	16	84	38	80	90	20
I have a liver disease/liver dysfunction	NA*	NA	NA	NA	NA	NA	Na	Na	NA	NA	Na
I have a heart weakness/heart performance weakness	30*	1	8	8	12	2	33	80	60	50	86
I am suffering from a chronic respiratory disease	157	7	14	1	129	13	17	52	99	93	91
I am suffering from diabetes	158	6	23	0	129	6	18	79	100	100	96
I have troubles remembering things or tend to forget things	157	7	9	26	116	6	10	60	82	26	95
I take more than five drugs every day, prescribed by my physician	144	20	10	12	84	38	33	21	88	46	69
Sleeping pills	147	17	15	10	121	1	11	93	92	60	99
Cortisone or other steroids	149	15	11	2	129	7	12	61	98	85	95
Antiepileptic drugs	149	15	0	0	149	0	00	NA	100	NA	NA
Oral anticoagulants	149	15	21	5	123	0	14	100	96	81	100
Tricyclic antidepressants	149	15	2	2	145	0	01	100	99	50	100
Drugs for rheumatism/inflammation	149	15	7	18	120	4	07	64	87	28	97
Drugs for drainage (diuretics)	149	15	26	9	89	25	34	51	91	74	78
Digoxin	149	15	1	0	147	1	01	50	100	100	99
Anticholinergic drugs	149	15	1	0	146	2	02	33	100	100	99
Insulin/drugs used in diabetes	148	16	16	2	127	3	13	84	98	89	98
Do you sometimes forget to take your medicine?
BMQ	54	110	39	8	3	4	20	91	27	83	43
I use some of these application forms: spray for inhalation, skin patch, syringe for self-injection	129	35	27	12	84	6	26	82	88	96	93
Mean value								67	88	74	86
Range								21–100	27–100	26–100	20–100

Rephrased statements for DART V.2.0, revalidated with 31 patients.

BMQ, Beliefs about Medicines Questionnaire; DART, Drug-Associated Risk Tool; NA, not applicable; Rf, risk factor.

Discussion

We intended to create an easy-to-use and reliable screening tool to identify patients who are at increased risk for DRPs. The application of such a tool has the potential to support the healthcare professionals in choosing patients who benefit the most of intensified pharmaceutical care. A patient self-assessment tool may save time and resources of caregivers, but also allows the better involvement of the patient. Assessing DRPs with such involvement of the patient may reveal more issues.27

We used risk factors for the development of DART, previously identified in a combination of a literature search and an expert panel.23 To our knowledge, this approach has not been adopted previously in this area of research.

DART V.1.0 showed good acceptability and feasibility. The patients were able to complete the self-assessment within on average 7 min and indicated no major difficulties with understanding the content of the questionnaire. The 48 patients (23%) who refused to participate were either not interested in participating or felt too tired to follow an interview.

After the validation of the first version of DART (V.1.0), we engaged three statements with an identified low sensitivity and possible poor patient understanding and aimed to improve their wording by implementing expressions into our questionnaire which are frequently used in PILs. We were able to include a statement covering heart failure with an acceptable sensitivity, while observing some more false positive answers. The reliability of patients to answer questions about renal insufficiency remains a challenge: Disease awareness among patients with chronic kidney disease is generally low,24 28 29 hence making it difficult to retrieve information on from a self-assessment questionnaire. The low knowledge of chronic comorbidities like chronic kidney disease may show a lack of patient education within counselling and may therefore pose an additional task for pharmaceutical care.

Finally, after the validation of the revised questionnaire, most statements of DART V.2.0 showed high specificity (mean value 88%, range 27%–100%) preventing false positive answers with a high probability. The sensitivity of the statements was lower and showed higher variability (mean value 67%, range 21%–100%). The sensitivity turned out to be higher in statements addressing conditions that require regular disease control and daily attention to self-care and drug management. Drugs requiring a high level of self-management showed the highest sensitivity (eg, oral anticoagulants, insulin and oral antidiabetics).

Several factors may have influenced the sensitivity values. First, the defined criteria for correlation (cf. table 1) served as a basis for the validation of the questionnaire. Depending on how we defined the criteria, we reached a certain degree of correlation between patients’ answers and the objective data. Second, we evaluated the sensitivity and specificity of each question by comparing the subjective answers in DART with objective data from medical records. Literature shows that medication histories at the time of admission are often erroneous and incomplete,30 which might have influenced our results. Especially the statement ‘I take more than 5 drugs every day, prescribed by my physician’, showed surprisingly weak correlation between subjective patient answers and objective medical data. Lau et al31 stated that regarding at the medication history in the hospital medical record, 25% of the prescription drugs in use are not recorded and 61% of all patients have one or more drugs not registered. Bedell et al32 evaluated the discrepancies between what physicians prescribe and what patients report they actually take. They showed that discrepancies between recorded and reported medication are common. Half of the discrepancies (51%) result from patients taking medications that were not recorded. One-third of the discrepancies involved OTC drugs or herbal therapies. We used medical records as reference for testing our statements’ and the patients’ reliability to provide correct answers in our self-assessment questionnaire. Errors within the medical histories as described above would carry over to our findings about the statements. Third, patients stated that they had no problems with filling in DART; however, we noticed some problems with their understanding of the word ‘chronic’. And we were aware of the possible existence of a social desirability bias when we directly asked patients for their opinion about the questionnaire.

Finally, the low prevalence of some risk factors (eg, antiepileptic drugs, tricyclic antidepressants, digoxin and anticholinergic drugs) hinders clear conclusions about the validity of the respective statements in DART.

Conclusions

The self-assessment questionnaire ‘DART’ showed a satisfying feasibility and reliability. Despite some low sensitivity values, this questionnaire seems to be applicable to patients in a hospital setting. Patients may be a valuable, but often neglected source of information. Asking them about their conditions, their medicines and related concerns and problems may facilitate getting a first, but broad picture of the risk for DRPs and possible pharmaceutical needs. Compared with gathering all the relevant data from case notes, electronic patient files and other sources, a self-assessment questionnaire seems to be a quick and easy method to identify patients in need for intensified pharmaceutical care.