The agreement of patient-reported versus observed medication adherence in type 2 diabetes mellitus (T2DM).

OBJECTIVE: Medication adherence in type 2 diabetes mellitus (T2DM) improves glycemic control and is associated with reduced adverse clinical events, and accurately assessing adherence assessment is important. We aimed to determine agreement between two commonly used adherence measures-the self-reported Morisky Medication Adherence Scale (MMAS) and direct observation of medication use by nurse practitioners (NPs) during home visits-and determine the relationship between each measure and glycated hemoglobin (HbA1c). RESEARCH DESIGN AND METHODS: We evaluated agreement between adherence measures in the Southeastern Diabetes Initiative (SEDI) prospective clinical intervention home visit cohort, which included high-risk patients (n=430) in 4 SEDI-participating counties. The mean age was 58.7 (SD 11.6) years. The majority were white (n=210, 48.8%), female (n=236, 54.9%), living with a partner (n=316, 74.5%), and insured by Medicare/Medicaid (n=361, 84.0%). Medication adherence was dichotomized to 'adherent' or 'not adherent' using established cut-points. Inter-rater agreement was evaluated using Cohen's κ coefficient. Relationships among adherence measures and HbA1c were evaluated using the Wilcoxon rank-sum test and c-statistics.
RESULTS: Fewer patients (n=261, 61%) were considered adherent by self-reported MMAS score versus the NP-observed score (n=338; 79%). Inter-rater agreement between the two adherence measures was fair (κ=0.24; 95% CI 0.15 to 0.33; p<0.0001). Higher adherence was significantly associated with lower HbA1c levels for both measures, yet discrimination was weak (c-statistic=0.6).
CONCLUSIONS: Agreement between self-reported versus directly observed medication adherence was lower than expected. Though scores for both adherence measures were significantly associated with HbA1c, neither discriminated well for discrete levels of HbA1c.

In the clinical setting, medication adherence is difficult to assess due to the complexity of the medication regimen.

Agreement between self-reported medication adherence and nurse practitioner observed adherence ranked ‘fair’.

Higher adherence was significantly associated with lower glycated hemoglobin levels for both measures.

Introduction

Poor medication adherence is a public health threat that increases the risk of disability and death1 and disproportionately affects those with chronic illness,2 poor access to healthcare,3 and low health literacy.4 The cost of non-adherence in the USA falls between $100 billion and $289 billion annually.5 6 Efforts to improve adherence require valid and reliable measures that can be easily integrated into the existing workflow of real-world clinical settings. Using a self-reported medication assessment tool such as the Morisky Medication Adherence Scale (MMAS)7 has shown conflicting results in many chronic illness populations, including heart failure,8–10 hypertension,11 12 mental illness,13 and diabetes.14–16 In the USA, as the prevalence of type 2 diabetes mellitus (T2DM) rises to nearly 10% of the population,17 improving adherence to medications that affect glycemic control will be increasingly important to preventing debilitating complications and death. More evidence is needed to establish the validity and clinical utility of self-reported measures of medication adherence for patients with diabetes.

Research design and methods

In this study, we examined the level of agreement between patient-reported medication adherence using the MMAS and directly observed assessment of pills and injectables by nurse practitioners (NPs) during home visits. In addition, we sought to determine the ability of each of these two measures to discriminate for higher versus lower levels of blood glucose (glycated hemoglobin, HbA1c >7.5).

The cohort comprised high-risk patients (n=430) from four geographically diverse counties participating in the Southeastern Diabetes Initiative (SEDI): Durham County, North Carolina, USA; Cabarrus County, North Carolina, USA; Quitman County, Mississippi, USA; and Mingo County, West Virginia, USA. The study design and intervention of SEDI have been previously described.18 Participants were adults with a diagnosis of T2DM and defined as high risk for serious adverse events or death. The criteria used to define ‘high-risk’ for this study included baseline indicators of diabetes complications such as microvascular and macrovascular damage, renal and cardiovascular comorbidities, and social factors available in the electronic health record, including insurance and marital status. These factors were integrated with geographic information and used to create a geographic health information system (GHIS), which combined patient-level and community-level risks and was used to generate a ‘risk algorithm’.18 Using this algorithm-generated score, we identified the top 10% of the population as ‘highest-risk’ and invited these patients to enrol in the intensive intervention as represented here (n=430). Exclusion criteria included an inability to make health-related decisions, a terminal illness with a life expectancy of 6 months or less, a diagnosis of T1DM, or gestational diabetes or pregnancy.

In each county, multidisciplinary healthcare teams—including a physician, NP, dietitian, pharmacist, licensed clinical social worker, and community health assistant—provided care. Participants provided written informed consent to participate in the clinical intervention, consisting of a home visit that included a physical assessment, review of medications, and completion of patient-reported outcome surveys for medication adherence, nutrition, physical activity, diabetes care self-efficacy, and health behaviors. At each home visit, NPs administered the MMAS—a validated, 8-question survey that assesses self-reported medication adherence.7 NPs determined medication adherence using pill counts, medication bottle dates, and direct observation of insulin administration. Through these measures, NPs gave a categorical score to reflect adherence. Patients were assigned medication adherence scores of 0–20%; 21–80%, or >80%, reflecting the proportion of actual medications taken. Adherence >80% of the time was considered adherent, a method previously described.10 19

We used Cohen's κ coefficient to determine the level of inter-rater agreement between the two measures. We evaluated the association between adherence and blood glucose using a Wilcoxon rank-sum test. Finally, we calculated the c-statistic to assess the ability of each measure to discriminate between high and low blood glucose.

Results

Of 536 patients enrolled in the clinical intervention, 430 had complete data for baseline MMAS scores, observed adherence, and HbA1c values and were therefore eligible for analysis. There were no systematic differences between participants and those excluded for missing data. Descriptive results are shown in table 1. About half of the patients were categorized as adherent, though self-reported adherence (n=261, 61%) was lower than directly observed adherence (n=338, 79%) (table 2). The level of agreement between the two adherence measures was fair (κ=0.24; 95% CI 0.15 to 0.33; p<0.0001). The median number of medications was 6.0 (IQR 7.0). In addition, oral medications were associated with higher rates of adherence as compared with subcutaneous injections (χ2=6.88; p<0.009).

Table 1

Demographics of the baseline cohort (n=430)

	CabarrusN=76	DurhamN=179	MingoN=151	QuitmanN=24	OverallN=430
Age, years
<35	2 (2.6%)	3 (1.7%)	2 (1.3%)	0 (0.0%)	7 (1.6%)
35–65	62 (81.6%)	141 (78.8%)	96 (63.6%)	10 (41.7%)	309 (71.9%)
>65	12 (15.8%)	35 (19.6%)	53 (35.1%)	14 (58.3%)	114 (26.5%)
Overall
Mean (SD)	56.2 (9.9)	56.6 (11.5)	61.0 (11.1)	68.5 (12.2)	58.7 (11.6)
Median (Q1, Q3)	56.0 (51.5, 63.0)	56.0 (49.0, 64.0)	61.0 (53.0, 69.0)	66.5 (59.5, 75.5)	59.0 (51.0, 66.0)
Min–max	29–77	23–89	33–84	47–96	23–96
Male	39 (51.3%)	79 (44.1%)	67 (44.4%)	9 (37.5%)	194 (45.1%)
Female	37 (48.7%)	100 (55.9%)	84 (55.6%)	15 (62.5%)	236 (54.9%)
Ethnicity
Hispanic or Latino	4 (5.3%)	14 (7.9%)	1 (0.7%)	0 (0.0%)	19 (4.4%)
Not Hispanic or Latino	72 (94.7%)	163 (92.1%)	150 (99.3%)	24 (100.0%)	409 (95.6%)
Not reported/missing	0	2	0	0	2
Race
White	41 (53.9%)	18 (10.1%)	144 (95.4%)	7 (29.2%)	210 (48.8%)
African-American	35 (46.1%)	145 (81.0%)	7 (4.6%)	17 (70.8%)	204 (47.4%)
Other	0 (0.0%)	16 (8.9%)	0 (0.0%)	0 (0.0%)	16 (3.7%)
Not reported/missing	0	0	0	0	0
Living arrangement
Alone	11 (14.5%)	51 (29.3%)	37 (24.7%)	9 (37.5%)	108 (25.5%)
Not alone	65 (85.5%)	123 (70.7%)	113 (75.3%)	15 (62.5%)	316 (74.5%)
Not reported/missing	0	5	1	0	6
Health literacy
3rd grade or below	8 (12.1%)	12 (10.2%)	13 (9.1%)	4 22.2%)	37 (10.7%)
Above 3rd grade	58 (87.9%)	106 (89.8%)	130 (90.9%)	14 (77.8%)	308 (89.3%)
Not reported/missing	10	61	8	6	85
Insurance
With	60 (78.9%)	134 (74.9%)	143 (94.7%)	24 (100.0%)	361 (84.0%)
Without	16 (21.1%)	45 (25.1%)	8 (5.3%)	0 (0.0%)	69 (16.0%)
Not reported/missing	0	0	0	0	0

Data reported as n (%), unless otherwise marked.

Table 2

Correlation of baseline self-reported adherence (MMAS) versus directly observed assessment

	NP-reported
	Adherent	Not adherent	κ coefficient	CI	p Value*
Oral†	205 (88.0%)	28 (12.0%)	NA	NA	NA
Subcutaneous	300 (79.4%)	78 (20.6%)	NA	NA	NA
Total	338 (78.6%)	92 (21.4%)	NA	NA	NA
Morisky adherent	228	33	0.24	0.15–0.33	<0.0001
Morisky not adherent	110	59

*Two-sided Pr |Z| Test of H0: κ.

†Oral medications are associated with higher rates of adherence as compared with subcutaneous medications (χ2<6.88; p=0.009).

MMAS, Morisky Medication Adherence Scale; NA, not applicable; NP, nurse practitioner.

HbA1c indicated uncontrolled blood glucose (HbA1c >7.5) in a majority of the cohort (n=350, 86%) (table 3). For both methods of assessment, higher adherence was significantly associated with lower HbA1c levels (p<0.001); yet, the ability of each measure to discriminate between lower and higher blood glucose was weak (c-statistics=0.63 and=0.61, respectively).

Table 3

Blood glucose control and self-report versus directly observed adherence

	Adherent	Not adherent	p Value	C-statistic
Morisky adherence
HbA1c (%)			<0.001*	0.626
N	242	164
Mean (SD)	9.5 (2.2)	10.5 (2.2)
Median (Q1, Q3)	9.3 (8.0, 10.8)	10.4 (9.0, 12.2)
Min–max	4.5–15.9	5.5–17.2
NP-reported adherence
HbA1c (%)			0.002*	0.611
N	320	86
Mean (SD)	9.8 (2.2)	10.6 (2.2)
Median (Q1, Q3)	9.4 (8.2, 11.1)	10.5 (9.2, 12.1)
Min–max	4.5–15.9	5.7–17.2

*Based on Wilcoxon test.

HbA1c, glycated hemoglobin; NP, nurse practitioner.

Discussion

In this large cohort of patients with high-risk diabetes across four geographically diverse counties of the Southeastern USA, the agreement between self-reported medication adherence and directly observed counts of pills, insulin pens, and medication bottles was only fair. Several possible conclusions may be drawn.

First, patients' perceptions of their medication-taking behavior may be inaccurate, confounded by the myriad of medications for which dosing instructions vary with life events, such as food intake, activity, or time of day. In this study, for example, 61% (n=261) of patients reported consistently taking medications as prescribed. Yet, of those, 31 people were actually considered non-adherent based on direct observation. Conversely, 40% (n=169) of patients reported poor adherence. Yet, of those, 110 people were doing better than they thought and were assessed as adherent by an NP. For many, this misperception of good versus poor adherence may simply be due to the high proportion of medications that are prescribed to be taken ‘as needed’ or in sliding scale doses. In previously reported studies, discrepancies in self-reported adherence have been attributed to the complexity of the medication regimen20 21 and the inherent difficulty in recognizing what adherence is.

Another interpretation of the results may be that in high-risk patients with T2DM, the MMAS may lack sensitivity and may be a poor indicator of actual medication use. Previous work with the MMAS in other populations has shown a strong correlation between self-reported adherence and actual adherence. In hypertension, for example, the MMAS has demonstrated a strong correlation between self-reported adherence and subsequent blood pressure control.7 12 Though early studies support the MMAS as being sensitive to actual changes in medication adherence, more recent studies have disputed these findings.11 22 23

In diabetes, numerous studies have evaluated self-reported medication adherence using the MMAS, some showing a positive relationship between adherence and HbA1c.24 25 In many of these studies, however, participant demographics differed widely from those in SEDI, with fewer comorbid illnesses, higher rates of insurance coverage, more frequent single-dose regimens, and higher levels of education and health literacy.16 In SEDI, factors that classified patients as high risk included recent hospitalizations, substance use, tobacco use, and multiple comorbidities—including coronary artery disease, hypertension, heart failure, or chronic kidney disease—all of which require complex medication regimens. As a result, patients may report that they are ‘getting enough medications’ daily, skewing self-reported results26 and suggesting that improvement in diagnostic measures is needed, particularly in illnesses with multiple comorbidities.

A second finding of this study was low discrimination. The c-statistics for both measures were similar, 0.63 for self-reported MMAS and 0.61 for direct observation. Though higher adherence scores were significantly associated with lower HbA1c values for both measures, neither was able to discriminate well between lower and higher levels of serum glucose control as indicated by HbA1c. Similar findings have been previously reported.11 16 One possibility is that other factors, beyond medication usage, are driving A1c in this high-risk population.

Regardless of the underlying reasons for lack of agreement between self-report and observed counts, every effort must be made to discover where, in this high-risk population, the breakdown occurs. Despite only fair agreement between self-report and observed medication counts, using the MMAS as a screening tool could improve identification of those highest risk patients in need of follow-up. The MMAS offers a quick survey that can be used in inpatient and outpatient as well as individual and group settings, and is useful for identifying those who need further evaluation and support for medication management. The individual reasons for non-adherence were not included as part of the analysis, as our focus was to evaluate and objectively report the agreement between patient-reported adherence and the observed assessment of a healthcare provider who was visualizing and counting bottles, vials, and pills. Though reasons for medication-taking behavior are critical in an intervention, this analysis was designed to provide evidence for the use of preintervention baseline data for determining the value of patients' self-reported medication adherence. Using these results substantiates the value of a baseline adherence assessment, and opens a forum for patients and providers to then earnestly consider reasons for various medication-taking behaviors and take these into account as part of the subsequent plan of care.

Limitations to this study include the following: a relatively long intake visit and placement of patient-reported outcome surveys, including the MMAS, at the end of the visit when patients may have been fatigued could affect results; patients took the self-reported survey while the healthcare member was present, which may have imposed an inadvertent Hawthorne effect; and a relatively high proportion of the cohort was unable to self-administer the survey due to blindness or very low health literacy. Though NPs read the survey for patients in the latter category, the results may have been skewed.

Conclusion

With 29 million diagnosed and undiagnosed people with diabetes,17 providing a brief but accurate assessment of medication adherence is a requisite tool for clinical practice. This study suggests that in chronic illnesses with multiple comorbidities in which a complex medication regimen is required, identifying more sensitive and discriminatory measures of medication adherence is necessary.