Outcome measures for adherence data from a medication event monitoring system: A literature review.

WHAT IS KNOWN: Currently, medication bottles with an electronic cap are frequently used to measure medication adherence. This system is termed medication event monitoring system (MEMS). To our knowledge, the optimal method to summarize data from MEMS has not yet been determined.
OBJECTIVE: Look for best practices on how to quantify adherence data from MEMS.
METHODS: Review of PubMed, Embase and Cochrane databases for the articles on medication adherence with MEMS.
RESULTS: Of 1493 identified articles, 207 were included in this review. The MEMS cap was used for a median of 3 months (IQR: 4; range: 1 week to 24 months) in various health conditions. Many different outcome measures were used. Most studies computed an adherence score, expressed as the percentage of days on which the correct dose of medication was taken. The threshold to mark people as adherent was most frequently, arbitrarily, set at 80% (range: 67%-95%). We found no data to support a specific threshold. DISCUSSION: Although the commonly used definition of adherence has face validity, we found no validation studies, and not all studies used the same cut-off for adherence. Ideally, a cut-off should be defined and validated in the context of the specific drug and its pharmacokinetic and dynamic characteristics, and perhaps other contextual factors, rather than generically. In addition, there was large heterogeneity in the definition of what "correct intake" of medication is. WHAT IS NEW AND
CONCLUSION: Outcome measures for MEMS data lacked standardization, and no demonstrable effort to validate any definition against a relevant clinical outcome is available. Consensus on the definition of adherence is urgently needed.

WHAT IS KNOWN

Medication adherence can be measured noninvasively in different ways. These include questionnaires, pill counts and electronic monitoring1; medication bottles with an electronic cap are often seen as the preferred method to measure adherence.1 This method, termed medication event monitoring system (MEMS), comprises a cap that contains an electronic device which records the date and time of each opening and closing of the bottle.1 However, to our knowledge, there is no standard method to summarize the adherence data from MEMS.

OBJECTIVE

The rationale for this narrative literature review is to enable an informed choice on the preferred methods to summarize the adherence data from the currently running GLORIA trial.2 This trial, part of a project funded under the EU‐horizon 2020 programme, examines harm, benefit and costs of low‐dose glucocorticoids added to the standard treatment of rheumatoid arthritis patients of 65 years or older. Adherence is measured with MEMS throughout the trial.

In this literature review, the methods to summarize MEMS data will be described.

METHODS

Search strategy

A literature search was conducted in September and October 2016 in the databases, PubMed, Embase and Cochrane, and updated in July 2017. Search terms were related to the following main MESH search terms: medication (non)adherence/compliance, medication persistence, chronic disease/illness, chronically ill, medical electronics, treatment, (drug) therapy, data analysis and statistical study. For an additional search, the following terms were used: reminder system, smartphone and mobile/electronic app(lication). Synonyms of these search terms were also used. The main search terms and their synonyms were used in different combinations. Our search strategy is described in Appendix S1.

Study of title and abstract resulted in a first list of titles eligible for full‐text review. Articles not written in English or Dutch and those that did not describe electronic monitoring caps were excluded. All other articles were reviewed in full text. In this phase, articles lacking useful information for this review were excluded. A search of the reference lists of included articles did not provide extra articles.

An additional search was performed to find validation studies that compared the definitions that are used to summarize MEMS data.

Data extraction

One investigator (LH) extracted the following information: design of the study, sample size, mean age and health condition of participants, duration of monitoring and methods to calculate adherence (Appendix S2). Study results (eg, the effects of interventions on adherence) and the quality assessment of the study were not the object of study and thus not extracted.

RESULTS

Study selection

The search identified 1493 articles, of which 1127 off‐topic articles and 48 double entries were excluded after screening of title and abstract. Of the remainder, 71 articles were excluded because the subject was not about electronic monitoring (n = 34), the full text was not available (n = 29) or the article was not written in English or Dutch (n = 8). Of the 247 articles read in full, 40 contained no useful information for this review. In the end, 207 articles were included (see Figure 1 and Appendix S1). No validation studies were identified.

Figure 1

PRISMA flow diagram of article selection

Study characteristics

Most of the included studies had a prospective design; about one third was a randomized controlled trial. The sample size was a median of 83 patients (IQR: 106, range: 4‐3004). The mean age of the patients was 52 (SD 46) years, and 57% of them were male. A total of 62 different health conditions were studied. Most patients had HIV (29%) or heart failure (10%) (Table 1). The MEMS cap was used for a median of 3 months (IQR: 4; range: 1 week to 24 months).

Table 1

Number of studies by health condition and adherence outcome measures

Health condition
HIV	60
Heart failure	20
Hypertension	17
Schizophrenia	12
Diabetes	6
Glaucoma	6
Depression	5
Type 2 diabetes and depression	5
Cancer	4
Kidney transplantation	4
Other conditions
In 3 studies	3
In 2 studies	10
In 1 study	39
Adherence definitions
Adherence score: percentage of days on which the correct dose was taken	156
In the week or month before the return date of the medication bottle	3
After intervention	1
Average change per month	1
Dose compliance: in drugs with multiple dosing on a day, the mean percentage of doses taken correctly per day	48
Timing compliance: percentage of doses taken at the appropriate time	28
Drug holiday: period of a certain number of days on which the medication bottle was not opened	9
Under (hypo‐)adherence: missing ≥10% of doses	6
Over (hyper‐)adherence: ≥10% more openings than expected	5
Dosing interval: the exact time between two openings (ie, doses)	3
Omissions: multiple missed doses	2
Noncompliance: percentage of skipped and extra doses	1
Patterns of missed doses: number of days without a dose, number of treatment interruptions lasting ≥48 hours, duration of the longest treatment interruption	1
Timing distribution index: indicates the regularity of the timing of drug intake	1

Outcome measures

Medication event monitoring system systems can supply a wealth of information, including dates and times of openings, the intervals between two consecutive doses and a graph which presents the number of cap openings per day.3 The included studies reported several outcome measures (Table 1). Most studies computed an adherence score (n = 156), expressed as the percentage of days on which the correct dose of medication was taken.4 The choice of outcome measure was independent of the health condition (results not shown).

In 76 studies, a threshold was defined on the adherence score to mark people as adherent or nonadherent. The thresholds ranged from 67% to 95%, and in half of the studies, it was 80%; most frequently chosen alternatives included thresholds of 90% (n = 13), 88% (n = 8) and 95% (n = 8).

The dose compliance (ie, in drugs with multiple dosing on a day the mean percentage of doses taken correctly per day) and the timing compliance (ie, the percentage of doses taken at the appropriate time) were also calculated in several studies (in 14% and 23%, respectively). The time frame ranged from 2 to 4 hours in studies on glaucoma,5, 6, 7, 8 diabetes mellitus,9 HIV9, 10 and schizophrenia.11

A few studies (4%) calculated “drug holidays,” that is periods of a certain number of days on which the medication bottle was not opened, followed by a bottle opening.12, 13 In contrast, Israni et al excluded patients who had fewer than 14 days of usable adherence data.14 Olds et al considered MEMS data as missing if the bottle was not opened for a certain number of consecutive days.15

In some studies, multiple openings were counted as 1 opening if the bottle was opened several times within 1516 or 30 minutes17, 18 of the previous opening. Unexpected openings outside this time window were assumed to represent a taken dose.16

DISCUSSION

In this narrative literature review (the first to our knowledge), we looked for best practices on how to quantify adherence. We chose a broad scope, but were somewhat limited due to language restrictions and the unavailability of some studies.

The adherence score, that is the percentage of days correctly dosed, and its cut‐off of 80% were the most frequently used quantification and definition of sufficient adherence. Although this definition has face validity, we found no validation studies, and not all studies used this cut‐off. Ideally, a cut‐off should be defined and validated in the context of the specific drug and its pharmacokinetic and dynamic characteristics, and perhaps other contextual factors, rather than generically.

In addition, there was large heterogeneity in the definition of what “correct intake” is. This included definitions of the allowable time window between doses, overdosing and dealing with consecutive days with no bottle openings. Some studies mark these periods as drug holidays, whereas other studies consider these periods as missing data. Any definition should ideally be tested/validated against a clinically relevant outcome to be of use in the clinic. In addition, an array of definitions for adherence was used, indicating an urgent need for a consensus effort. Such efforts have been successful in rheumatology19 and are gaining traction in other fields.20

It is also remarkable that for so many health conditions, adherence studies with MEMS are rare or even nonexistent. Most studies were about HIV, heart failure, hypertension or schizophrenia. We did not find any relation between these health conditions and the methods that were used to summarize the MEMS data. MEMS is often seen as the reference standard to measure medication adherence, but it still assumes that one bottle opening equals the intake of one medication dose,3 a simplification that cannot be easily checked.16, 17, 18 For example, a patient could open the bottle and either not take any or more than the appropriate dose. Validation of MEMS data may become possible with compliance capsules with an ingestion sensor.21, 22 This is a new method, where the sensor signals when the drug is taken. Compliance capsules have the potential to become the new reference standard in the future.

WHAT IS NEW AND CONCLUSION

While adherence is clearly critical to treatment success, this review demonstrates a lack of consensus on a concrete working definition to be used in studies and no demonstrable effort to validate any one definition against a relevant clinical outcome. Progress in this field is unlikely unless these issues are addressed.

Click here for additional data file.

Click here for additional data file.