How should we measure medication adherence in clinical trials and practice?

OBJECTIVE: To determine if simple adherence measures, such as twenty-four hour recall and refill history, are accurate for routine use, compared to more time-consuming measures such as pill counts.
DESIGN: Randomized, double-blind, placebo-controlled trial.
SETTING: Walter Reed Army Medical Center, a tertiary medical center in Washington. PATIENTS: Men and women >30 years old with known coronary heart disease and taking a statin medication. INTERVENTION: Clinical pharmacists met with patients for adherence assessments. MAIN OUTCOME MEASURES: Adherence was measured by pill counts, twenty-four hour recall by patient, and refill history per computer record. Temporal changes in these adherence measures were assessed using general linear models for repeated measures.
RESULTS: Adherence was consistently greater for the experimental agent than for the statin therapy (n = 148). Mean pill count adherence for statin drug was 78.7 +/- 25.2% compared to 93.5 +/- 11.6% (P < 0.001) for the study agent. Refill history and twenty-four hour recall inaccurately measured adherence when compared to pill counts. Adherence, as determined by pill count, for both experimental (P = 0.029) and statin therapy (P = 0.015) showed significant variability across time in general linear models. Neither refill history nor twenty-four hour recall was sensitive to temporal changes.
CONCLUSIONS: Twenty-four hour recall and refill history inaccurately measure medication adherence for both clinical trial and clinical practice pharmacotherapies. Further, these measures are insensitive to changes in adherence. For a single or multiple assessments across time, pill count more accurately measures medication adherence. Pill count should be the standard for monitoring medication adherence for both clinical trials and clinical practice.

RCT Entities:   Population Interventions Outcomes

Introduction

Adherence to prescribed medications, both in research trials and in clinical practice, is crucial to the success of the pharmacologic interventions. However, for both settings there is no accepted standard method of assessing adherence, both as a cross sectional measurement and for measurements across time (Gordis 1984; Mattson et al 1984; Norell 1984; Cramer et al 1989; Rudd et al 1989; Avorn et al 1998; Farmer 1999; Liu et al 2001; Benner et al 2002; Garber et al 2004). The latter is crucial to understanding the success of pharmacy adherence programs. Frequently reported methods of measuring medication adherence include patient self-report, the use of electronic databases for prescription claims or refill history, and the use of pill counts. In general, pill counts are regarded as being more accurate than self-report or refill history (Inui et al 1981; Stewart 1987; Botelho et al 1992; Choo et al 1999; Hamilton 2003). However, pill counts are tedious and difficult to administer, thus an understanding is needed of the quantitative disagreement with easier methods such as twenty-four hour recall and refill history.

Objective

We compared pill count, twenty-four hour recall and refill history, and their temporal relationships for both an experimental and clinical drug in a prospective, blinded clinical trial. Our objective was to determine if simple adherence measures, such as twenty-four hour recall and refill history, are sufficiently accurate for routine use, compared to more time-consuming measures such as pill counts.

Methods

ARBITER 2 was a prospective double-blind, randomized clinical trial comparing the efficacy of extended-release prescription niacin (Niaspan®, Kos Pharmaceuticals Inc., Cranbury, NJ) and matching placebo on the rate of atherosclerosis progression. The study was conducted from December 2001 to May 2004. By study design, all patients had known coronary heart disease and were required to already be taking a statin medication prior to study entry and continue taking the statin medication throughout the duration of the study. The trial was conducted at Walter Reed Army Medical Center, a tertiary medical center, and the protocol was approved by the institution’s Department of Clinical Investigation. All volunteers provided written, informed consent.

A dedicated research pharmacy, which manages all medications provided for approved protocols, dispensed the study medication. Statin therapy was dispensed by the main outpatient pharmacy of the facility along with the remainder of the participant’s medications. The majority (93%) of patients in the study used simvastatin, the primary statin provided by the Department of Defense, with the remainder prescribed atorvastatin. All participants met with one of the six clinical pharmacists, per study protocol, for planned adherence assessments of their study drug and statin medication. After an initial meeting at 30 days, adherence assessments were performed at 90, 180, 270, and 365 days for the 148 study patients. Of these, 14 participants (9.5%) forgot to bring their statin bottles to their study visit, (but did partake in study drug pill counts). These subjects are excluded from some analyses where noted. Participants were instructed to bring their medication bottles to each visit. Pill counts were calculated as the number of pills taken (the number of pills dispensed – the number of pills counted). The number of pills expected to have been taken was calculated by multiplying the daily dose (1/2, 1 or 2 tablets) by the number of days since the date dispensed. We a priori and rigorously defined successful adherence on pill counts as 85–100% of the pills taken during each follow-up period (Krueger et al 2003). Adherence per drug use by twenty-four hour recall was assessed based on the patient’s verbal response (from memory or with an aid of a personal medication list) of all of the chronic medications taken (including the study and statin medications) within 24 hours prior to each pharmacy visit. We considered the patients to be adherent according to the twenty-four hour recall if they were able to report taking all of their chronic medications. Adherence per refill history was determined by assessing the electronic pharmacy record system (Composite Health Care System, CHCS) to quantify the number of pills dispensed relative to time. Adherence was noted if all of the patient’s chronic medication refills were consistent with the number of days dispensed by the pharmacy, always within 90-days. All patients received statin and study medications from the military health care system.

The 148 study participants were seen and interviewed in a Pharmacy Outcomes Clinic at Walter Reed Army Medical Center throughout the study duration (up to 1 year). Pharmacists working in the clinic received training in each of the adherence assessment and documentation. Data were collected on a custom data collection form including written reminders on adherence assessment methods and adherence definitions. A data dictionary was collectively reviewed and discussed among the participating pharmacists in order to minimize the potential for inter-rater variability.

At the conclusion of each visit, the remaining study medication was taken and returned to the research pharmacy and a new drug supply was provided. This ensured that the exact start and end date of the pill count assessment was clearly documented. Participants were responsible for refilling their statin medication through the main outpatient pharmacy on schedule.

Mean pill counts for statin and study drug were computed and expressed as the mean ± standard deviation. Means were compared using paired t-tests or t-tests for independent groups, as appropriate. Temporal changes in pill count adherence were assessed using general linear models during a monitoring phase of up to 1 year. All analyses were conducted using SPSS statistical software (v 13.0; Chicago, IL). A P value ≤0.05 was considered statistically significant.

Results

A total of 148 subjects participated in pill counts of the study medication and assessment provided by the pharmacy intervention team. One-hundred thirty four subjects participated in pill counts of the statin medication. The mean age was 67 ± 10 years, 92% were men. All had a history of prior coronary heart disease (by study design) and had a high prevalence of cardiovascular risk factors including hypertension in 74.3% and diabetes mellitus in 27.7%. Most subjects, 86.3%, had a complex pharmacy regimen, defined as taking greater than 5 daily medications and/or 12 or more medication doses per day. 32.9% of the participants were taking multiple daily doses, defined as medications taken 3 or more times per day.

Adherence was consistently greater for the experimental agent (n = 148) than for the statin therapy (n = 134). The mean pill count for statin drug at 90, 180, 270 and 365 days was 78.7 ± 25.2% (n = 133; note the difference of 1 patient’s missing pill count for this measure) compared to 93.5 ± 11.6% (n = 148) (P < 0.001) for the study agent. This differential was consistently observed across the 12 month observation period (Figure 1). Despite this within group stability or persistence, there was a substantial degree of intra-subject variability in pill counts. Adherence for both experimental (P = 0.029) and statin therapy (P = 0.015) showed significant variability across time in general linear models.

Figure 1

Temporal pattern of pill-count adherence across 12 months for both statin (Panel A) and study drug (Panel B).

Twenty-four hour recall and refill history were insensitive to changes in pill counts over time. To assess this, we compared the maximum differential between separate pill counts for subjects who either were or were not always adherent by recall or refill history as defined under Methods. This analysis was limited to subjects in whom at least 3 of the 4 possible pill counts were completed (statin, n = 85; study drug, n = 126) in order to provide a more stable estimate of pill count differential. The proportion of subjects who were always adherent according to recall and refill history was 58.8% and 60.1%, respectively. The maximum differential in pill counts across the various time points (eg, 90 days vs. 180 days) in adherent subjects was similar to that observed in subjects considered nonadherent when categorized either by twenty-four hour recall (Figure 2A) or refill history (Figure 2B) [P = NS for all comparisons]. Among participants who were always adherent by refill history, the maximum differential of pill counts to statin agent was 43.6 ± 28.0% vs 36.8 ± 20.4% in those who were not always refill adherent (P = NS).

Figure 2

Maximum differential in pill-counts expressed as the greatest difference in pill counts between 90, 180, 270 and 365 day assessments in groups of patients categorized as adherent or nonadherent for 24-hour recall (Panel A) and refill history (Panel B).

The mean number of pill counts completed per patient (maximum of 4 possible pill counts occurring at 90, 180, 270, and 365 days) was 3.4 ± 0.9. The relationship for adherence assessments using pill counts compared to either twenty-four hour recall or refill history were different for study drug and statin therapy. Both twenty-four hour recall and refill history underestimated adherence for the study drug, and overestimated adherence for statin agent, compared to the pill count assessment (Table 1).

Table 1

Mean pill counts vs 24-hour recall and refill history adherence assessments across 12 months

	N*	Study agent		P values	N(#)	Statin		P values
		Pill count	24-hour recall			Pill count	24-hour recall
Pill count vs 24-hour recall	147	93.5 ± 11.6	85.7 ± 20.2	<0.001	134	78.3 ± 25.6	85.8 ± 21.3	0.009
		Pill count	Refill history			Pill count	Refill history
Pill count vs Refill history (Mean ± SD)	148	93.5 ± 11.6	81.9 ± 24.9	<0.001	133	78.7 ± 25.2	82.3 ± 25.2	0.225

Missing one twenty-four hour recall data.

Missing one statin pill count data.

Discussion

Understanding adherence to pharmacotherapies is crucial in clinical practice and research studies to ensure optimal clinical outcomes and valid study results. Research in this field has been limited by the lack of a true gold standard, recognizing that each method has strengths and weaknesses (Evans et al 1983; Mattson et al 1984; Craig 1985; Wright 1993; Grymonpre et al 1998; Choo et al 1999; Farmer 1999; Schwed et al 1999). Methods used in clinical practice are typically simple assessments of medication refill history or a patient-recall assessment (Fletcher et al 1979; Stewart 1987; Christensen et al 1997; Grymonpre et al 1998). These methods are easy to perform; however, they are more crude and their accuracy is limited. Although some suggest that the patient-recall and refill history assessments are accurate enough, especially when performed in combination (Mattson et al 1984; DiMatteo et al 2002; Farley et al 2003), these methods are generally regarded to substantially overestimate medication adherence (Haynes et al 1980; Inui et al 1981; Stewart 1987; Grymonpre et al 1998; Farmer 1999; Shalansky et al 2004). Additionally, the ability of the patient-recall and refill history to detect changes in adherence is unknown. In comparison, pill counts are laborious and rely upon the assumption that medications missing from the pill bottle were taken (Norell 1984; Rudd et al 1988; Cramer et al 1989; Farmer 1999). They also rely upon accurate reporting dates for starting prescriptions but can be more precise when carefully performed as in this study.

This study compared the longitudinal relationships among different methods of measuring medication adherence to both a study drug (placebo or extended-release prescription niacin) and the “control” clinical medication, their statin prescription. Our analysis has shown important differences in adherence between study and clinical medications, differences in the ability of twenty-four hour recall and refill history to detect nonadherence, and insensitivity of twenty-four hour recall and refill history to detect temporal changes in adherence. Pill count assessment not only showed the differences in patient adherence to study agent and clinical therapy, but reported the rate of adherence (from which we can detect nonadherence) and the changes in adherence across time (intra-subject variability). The observed differences in study and clinical medications are of interest. Although this study cannot provide the cause of this observation, it is likely that study participants were influenced by the intervention of the clinical pharmacists providing their study medication and observing their medication taking behavior.

Strengths of our study, performed within a cardiovascular disease population prescribed multiple additional pharmacologic agents, include its overall low attrition rate (Taylor et al 2004), quantitative data, inclusion of multiple drugs and manners of adherence assessments, and its longitudinal design. Even though nonadherence has been shown to be common in clinical practice, patients in this study were generally adherent despite complex medical regimens.

Limitations

The generalizability of our findings is limited by the application in a population of patients consenting to a clinical trial, who may more closely follow health advice and be interested in their health. Because there were six clinical pharmacists assessing the medication adherence for this study, there is a potential for inter-rater variability even though we instituted a standardized training for them. Since the clinical agent, statin, was not provided by the research pharmacy, some forgot to bring the bottles for pill count, hence the 14 subjects excluded from some analysis. Also, the scope of this study was limited by our inability to include other types of adherence measures such as electronic prescription bottles and drug blood levels (direct measures of adherence).

Conclusion

We found that twenty-four hour recall and refill history inaccurately assess medication adherence compared to pill counts for both clinical trial and clinical practice pharmacotherapies. Further, twenty-four hour recall and refill history are insensitive to temporal changes in adherence when assessed longitudinally. From these data, we conclude that pill count is a superior method of medication adherence assessment compared to twenty-four hour recall and refill history in both clinical practice and long term medication studies. Pill counts should serve as the standard for monitoring patient adherence for both experimental and clinical drugs.