Validation of a patient-level medication regimen complexity index as a possible tool to identify patients for medication therapy management intervention.

BACKGROUND: The Medication Regimen Complexity Index (MRCI) is a 65-item instrument that can be used to quantify medication regimen complexity at the patient level, capturing all prescribed and over-the-counter medications. Although the MRCI has been used in several studies, the narrow scope of the initial validation limits application at a population or clinical practice level.
PURPOSE: To conduct a MRCI validation pertinent to the desired clinical use to identify patients for medication therapy management interventions.
METHODS: An expert panel of clinical pharmacists ranked medication regimen complexity for two samples of cases: a single-disease cohort (diabetes mellitus) and a multiple-disease cohort (diabetes mellitus, hypertension, human immunodeficiency virus infection, geriatric depression). Cases for expert panel review were selected from 400 ambulatory clinic patients, and each case description included data that were available via claims or electronic medical records (EMRs). Construct validity was assessed using patient-level MRCI scores, medication count, and additional patient data. Concordance was evaluated using weighted κ agreement statistic, and correlations were determined using Spearman rank-order correlation coefficient (ρ) or Kendall τ.
RESULTS: Moderate to good concordance between patient-level MRCI scores and expert medication regimen complexity ranking was observed (claims data, consensus ranking: single-disease cohort 0.55, multiple disease cohort 0.63). In contrast, only fair to moderate concordance was observed for medication count (single-disease cohort 0.33, multiple-disease cohort 0.48). Adding more-detailed administration directions from EMR data did not improve concordance. MRCI convergent validity was supported by strong correlations with medication count (all cohorts 0.90) and moderate correlations with morbidity measures (e.g., all cohorts; number of comorbidities 0.46, Chronic Disease Score 0.46). Nonsignificant correlation of MRCI scores with age and gender (all cohorts 0.08 and 0.06, respectively) supported MRCI divergent validity. LIMITATIONS: This study used cross-sectional, retrospective patient data for a small number of patients and clinical pharmacists from only two universities; therefore, results may have limited generalizability.
CONCLUSIONS: The patient-level MRCI is a valid tool for assessing medication regimen complexity that can be applied by using data commonly found in claims and EMR databases and could be useful to identify patients who may benefit from medication therapy management.

Currently in the United States, health care systems are being asked to increase the volume of patients who receive medication therapy management (MTM) services and to improve quality indicators that have a heavy emphasis on appropriate medication use. MTM services include comprehensive assessment and evaluation of a patient's complete medication therapy regimen.1 The Centers for Medicare and Medicaid Services (CMS) 2014 guidelines for MTM programs added requirements for health plan sponsors to have a dedicated MTM information page linked to their website and to actively promote available MTM services to beneficiaries in an effort to increase awareness of MTM service availability and ultimately utilization.2 A similar medication focus can be found in the recent CMS star rating system for health plans that implemented incentive quality-based payments based on number of stars earned, where some star rating criteria are related directly to medication use and could be improved through MTM programs.3 For example, star ratings are earned through direct medication targets, such as medication adherence rates (e.g., diabetes, hypertension), percentage of members receiving medications with a high risk of side effects, or other criteria that depend on appropriate medication use (e.g., percentage of patients with diabetes with controlled levels of blood sugar). A screening tool that reliably identifies patients with greater medication regimen complexity could be useful for health plans (e.g., CMS ratings) and for practicing clinicians.

A literature review of measures of medication regimen complexity and associated outcomes found that medication regimen complexity is related to patient nonadherence, caregiver burden, quality of life, and medical resource utilization.4 Many different methods were used to quantify the complexity of medication regimens. The medication regimen complexity index (MRCI) was the most frequently used method. The MRCI is a tool that quantifies medication regimen complexity beyond the number of medications to include weighted scores for types of prescribed dosage forms, dosing frequency, and additional administration directions.5

The MRCI has been used to quantify medication regimen complexity in several studies.6–21 Results of MRCI studies describe medication regimen complexity for general groups of patients8,10–13,15–18 and for defined cohorts based on specific disease management (e.g., diabetes, hypertension).6,7,9,14,19–21 Two studies assessed the relationship of MRCI scores with medication adherence and found that increased MRCI scores were related to lower medication adherence levels.6,12 Earlier MRCI studies included only prescription medications for specific target diseases of interest (e.g., prescribed diabetes medications) and ignored other prescription and over-the-counter (OTC) medications for comorbidities (e.g., hypertension, pain management). In contrast, recent studies broadened the MRCI score to include all prescription and OTC medications used for all comorbidities, thus reporting a patient-level MRCI score.11,18,20,21 Patient-level MRCI scores have been shown to differentiate between high, medium, and low medical complexity as determined by other accepted measures of patient complexity, hospital readmission rates, medication count, comorbidity count, and Charlson comorbidity scores (a method of predicting mortality risk based on presence of comorbidities).11,20,21

The potential for the MRCI to be used as a tool in clinical practice and by health plans to identify patients who may benefit from subsequent interventions (e.g., comprehensive medication review) is limited by the original narrowly defined scope for the tool and the associated validation evidence. Although the MRCI was judged to be a reliable and valid tool from the original expert panel validation, the process was conducted using a single cohort of patients (chronic obstructive pulmonary disease) and only included prescription medications in the assessment of medication regimen complexity.5 Also, concordance of MRCI score ranking versus five expert clinician rankings was conducted by asking five experts from different fields (academic, research nurse, adherence expert, clinical pharmacist, and home medication review expert) to judge “the difficulty in coping with the provided regimens without taking into account any drug-, clinical-, or patient-related factors.”5 Considering the number of studies conducted since this initial validation, there appears to be a desire to use the MRCI as a risk assessment tool, along with other patient data, across disease states to identify patients for subsequent interventions.6–21 Thus, there is a need to validate the MRCI to identify patients for subsequent MTM interventions using commonly available drug- and patient-related data across multiple disease states.

Study Objectives

The purpose of this study was to examine the validity of the MRCI, using patient-level scores, as a tool to identify patients in two cohorts who might benefit from MTM intervention that could improve adherence or reduce risk of drug-related events. The two cohorts were a single-disease–defined cohort and multiple-disease–defined cohort. Study objectives were to assess concordance between expert pharmacist patient-level medication regimen complexity rankings and patient-level MRCI score rankings for patients within the same target disease state and patients in one of four target disease states. This study compared concordance of expert patient-level medication regimen complexity rankings and patient-level MRCI score ranking using alternative views of information (i.e., claims data vs electronic medical record [EMR] data). The final objective was to assess convergent and divergent validity of the MRCI using measures of patient complexity and other patient factors.

Methods

The study was designed to assess the validity of a patient-level MRCI based on expert opinion and systematically varying patient information. The process is depicted in Figure1. The initial step involved an in-person meeting at University of Colorado Anschutz Medical Campus (CU), with collaborators joining by conference call. During that meeting, the MRCI content (or face) validity was assessed by six clinical pharmacists practicing in ambulatory care clinics related to each of the four disease-defined cohorts studied: hypertension (HTN), diabetes mellitus (DM), human immunodeficiency virus infection (HIV), and geriatric depression (GD). Each pharmacist determined if the 65 MRCI items were related to medication regimen complexity and if other items were needed to more fully assess medication regimen complexity.

Figure 1

Analysis steps.

In a prior unpublished study, clinical pharmacists in the practice areas of HTN, DM, infectious disease, and geriatrics calculated patient-level MRCI scores (including prescription and OTC medications) in their area of expertise for 800 patient-level medication regimens collected from ambulatory clinics at the CU (400 regimens) and at the University of California, San Diego (SD, 400 regimens). An electronic data capture and coding tool was used to calculate MRCI scores (http://www.ucdenver.edu/academics/colleges/pharmacy/Research/researchareas/Pages/MRCTool.aspx).20

Briefly, the MRCI score calculated for each patient was based on three separate components of their medication regimen: (i) dosage formulations, (ii) dosing frequency, and (iii) additional administration directions. A weight of 1 was given to each tablet or capsule dosage form given once/day. Other dosage formulations and dosing frequencies were assigned increasing weights related to the increasing difficulty in administration (e.g., prefilled injectable agents receive a weight of 3). Additional administration directions (e.g., “break” or “crush,” “take with food”) associated with a medication added to the MRCI score with increasing weight according to difficultly in administration. The patient-level MRCI score calculated for each patient included all of a patient's prescription and OTC medications. These patient-level MRCI scores formed the pool from which cases used in the panel ranking exercises were randomly selected and provided data for convergent and divergent correlation testing.

Two samples of medication regimens from the CU patient pool were used as cases in the panel ranking exercises. The first was the single-disease cohort composed of nine patients with diagnosed and treated DM. The second was the multiple-disease cohort composed of nine patients diagnosed and treated for HTN (two patients), DM (three patients), HIV (two patients), and GD (two patients). The number of patients was limited to nine in each cohort based on anticipated time needed for the panel ranking exercise. To assess the ability of the patient-level MRCI to identify patient complexity (defined a priori as “expected patient difficulty managing medication regimens, thus warranting intervention”), the cases in the highest and lowest deciles of the patient pool (400 cases) of patient-level MRCI scores were excluded from possible consideration in the panel ranking exercise. Cases were chosen with a goal difference of at least 1.5 points between patients' scores (range 1.5–4 points on patient-level MRCI) and from the middle 50% of scores, although some cases were slightly outside of the middle range to maintain the goal difference. The intent was to find a sample of patients with monotonically increasing medication regimen complexity in the middle ranges of complexity. Excluding patients with extreme values was a more rigorous test of the MRCI tool because it removed obviously high- and low-complexity cases, in contrast to ranking across the full range of medication regimen complexity used in the original validation study that found “full agreement on rankings of regimens with extreme complexities.”5

The medication regimens for the two samples (nine for each) presented to the expert panel in two different views are summarized in Table1. The first view represented an administrative claims data view and presented medication regimen (medication names, strengths, dose, and calculated daily medication count), patient age, gender, and current diagnosed comorbidities (using International Classification of Disease, 9th Revision classifications). The second view represented an EMR view and included the same information as just given but instead of a calculated daily medication count, more-specific directions for medication administration and dose frequency were provided. An example case with the two views is presented in Appendix S1.

Table 1

Case Description and Ranks by MRCI Score and Medication Count

Casesa	Gender	Age	Number of Comorbidities	Patient-Level MRCI Score	Total Medication Count	MRCI Rankb	Medication Count Rankb
Single-disease cohort (DM)
A	F	53	17	25.5	13	6	9
B	M	65	34	9.5	5	1	1
C	M	48	21	27.5	5	7	2
D	M	57	29	33	12	9	8
E	M	40	43	29	11	8	7
F	F	45	17	20.5	6	4	3
G	M	59	12	22.5	8	5	5
H	M	71	18	17	10	3	6
I	M	56	12	12	7	2	4
Multiple-disease cohort (DM, HTN, HIV infection, GD)
A (HTN)	F	48	20	19.5	5	5	2
B (GD)	F	79	24	10	5	1	1
C (HIV infection)	M	51	7	25	10	7	8
D (DM)	M	40	43	29	11	9	9
E (DM)	M	56	12	12	7	2	4
F (HTN)	M	51	10	23	10	6	6
G (GD)	F	82	13	18	9	4	5
H (DM)	M	48	21	27.5	5	8	3
I (HIV infection)	M	58	15	15	10	3	7

MRCI = Medication Regimen Complexity Index; DM = diabetes, HTN = hypertension; HIV = human immunodeficiency virus; GD = geriatric depression.

Cases randomly selected from 400 ambulatory clinic patients from University of Colorado Anschutz Medical Campus.

Rank: 1 = lowest complexity.

A seven-member expert panel consisting of clinical pharmacists practicing in ambulatory care settings was used to rank regimens according to medication regimen complexity using only the information provided within the view (i.e., without using the MRCI). Two of the panel members had previous experience using the MRCI tool to score patient-level medication regimen complexity, and the remaining panelists were naïve to the MRCI. Medication regimen complexity was defined for the experts a priori as “expected patient difficulty managing medication regimens, thus warranting intervention.” Each expert individually performed four ranking exercises, within a 1-hour time period, ranking regimens in order of increasing patient-level medication regimen complexity from lowest to highest. The ranking exercises were conducted in the following order: (i) single-disease cohort (A) claims view and then (B) EMR view, and (ii) multiple-disease cohorts (A) claims view and then (B) EMR view. For each set, the expert ranked the nine regimens (printed as one regimen per page, given an identifier of A through I, with initial ordering of the stack shuffled for each expert) and provided notes regarding their rationale for ranking a regimen as higher or lower than an adjacent regimen. After all four sets had been ranked by each expert, they were brought together to report their individual rankings. A consensus discussion was then moderated to determine an overall consensus ranking of the four sets. The institutional review boards of the SD and the CU approved the study protocol.

Data Analysis

Concordance of MRCI-based rankings with expert rankings (criterion validity) was conducted using the weighted κ agreement statistic and Spearman rank-order correlation coefficient (ρ). The former was used as the gold standard statistic for rated ranks, and the latter was used for comparison to the original MRCI validation exercise.5 Construct validity, correlation of MRCI scores with age, gender, Charlson comorbidity index (unweighted), Chronic Disease Score (CDS), number of comorbidities, and medication count were conducted using Spearman rank-order correlation coefficient (ρ) for continuous variables, and Kendall τ was used for categorical variables. Level of significance was set a priori as ≤ 0.05. Guidelines for interpreting the strength of associations were < 0.20 poor, 0.2–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 good, and 0.81–1.00 very good for the κ statistic and 0.2 minimal, 0.5 moderate, and 0.8 strong for Spearman ρ and Kendall τ.22 The CDS was calculated using American Hospital Formulary Service codes to derive comorbidities for each patient based on the primary use of the medications. SAS version 9.3 (SAS Institute Inc., Cary, NC, USA) was used to conduct all analyses.

Results

The content validity of the MRCI was judged to be good in that all pharmacists agreed the items included were related to and sufficient to describe medication regimen complexity regardless of the disease state. In addition, basic patient-related factors were recommended as additional items to consider when assessing a patient's ability to manage complex medication regimens. Thus, age, gender, and comorbidities that would commonly be available via electronic claims or medical record data were included in the subsequent expert (i.e., clinical pharmacist) panel ranking exercises.

Concordance of expert panel rankings of patient-level medication regimen complexity, when using the claims view, with MRCI scores for the single-disease cohort were significant for six of seven panel members and the consensus ranking (weighted κ 0.48–0.63) (Table2). In contrast, concordance with medication count, a simpler common assessment of medication regimen complexity, was significant for only two panel members (weighted κ 0.18–0.63). For the multiple-disease cohort, concordance of expert panel rankings of patient-level medication regimen complexity with MRCI was significant for four of seven panel members and the consensus ranking (weighted κ 0.25–0.63). Similar to the single-disease cohort, concordance with medication count was significant for fewer panel members (two and consensus; weighted κ 0.25–0.55). There was no difference in the concordance of expert consensus ranking with MRCI ranking when experts used the claims data versus EMR data for the single-disease cohort (DM) (weighted κ 0.55 claim, 0.55 EMR). However, for the multiple-disease cohort, the concordance between expert consensus ranking and MRCI was lower when experts used the additional EMR data versus claims data (weighted κ 0.40 and not significant EMR, 0.63 claim). The values of significant Spearman ρ coefficients in Table2 indicated moderate to strong correlation that was consistent with the interpretation of the κ statistics as moderate to good concordance. Notably, nonsignificant coefficients were in the moderate (Spearman) or fair (κ) association interpretation range. Table3 presents a summary of concordance patterns across study contrasts.

Table 2

Concordance Expert Panel Medication Regimen Complexity with MRCI Ranking and Medication Count Ranking (Claims and Electronic Medical Record Views)

Expert	Claims View				EMR View
	Weighted κ		Spearman's ρ		Weighted κ		Spearman's ρ
	MRCI Score	Medication Count	MRCI Score	Medication Count	MRCI Score	Medication Count	MRCI Score	Medication Count
Single-disease cohort (DM)
1	0.33	0.63*	0.53	0.82*	0.63*	0.25	0.82*	0.48
2	0.55*	0.25	0.82*	0.53	0.48*	0.25	0.78*	0.57
3	0.55*	0.18	0.83*	0.13	0.33	0.33	0.48	0.60
4	0.63*	0.25	0.80*	0.50	0.70*	0.25	0.90*	0.45
5	0.55*	0.33	0.77*	0.60	0.48*	0.48*	0.67*	0.72*
6	0.55*	0.18	0.77*	0.43	0.63*	0.40	0.87*	0.68*
7	0.48*	0.48*	0.77*	0.75*	0.55*	0.55*	0.70*	0.78*
Consensus	0.55*	0.33	0.85*	0.63	0.55*	0.40	0.80*	0.68*
Multiple-disease cohort (HTN, DM, HIV, GD)
1	0.55*	0.40	0.80*	0.60	0.33	0.40	0.65	0.70*
2	0.48*	0.25	0.80*	0.50	0.48*	0.25	0.67*	0.33
3	0.33	0.25	0.48	0.47	0.10	0.63*	0.30	0.83*
4	0.25	0.40	0.50	0.68*	0.25	0.48*	0.48	0.73*
5	0.55*	0.55*	0.77*	0.73*	0.48*	0.55*	0.65	0.83*
6	0.63*	0.40	0.87*	0.70*	0.55*	0.25	0.83*	0.55
7	0.33	0.55*	0.58	0.77*	0.25	0.48*	0.42	0.65
Consensus	0.63*	0.48*	0.85*	0.75*	0.40	0.55*	0.67*	0.77*

MRCI = Medication Regimen Complexity Index; DM = diabetes, HTN = hypertension; HIV = human immunodeficiency virus; GD = geriatric depression.

p<0.05.

Table 3

Summary of Number of Experts with Medication Regimen Complexity Ranking in Concordance with MRCI Ranking

	Data View	Number of Experts of 7 in Concordance	Consensus Ranking in Concordance? Yes or No
Single-disease cohort	Claims	6	Yes
	EMR	6	Yes
Multiple-disease cohort	Claims	4	Yes
	EMR	3	No

Concordance-based weighted κ.

MRCI = Medication Regimen Complexity Index; EMR = electronic medical record.

Construct validity (convergent and divergent) of the MRCI was assessed using patient-level MRCI scores and patient characteristics for the 800 patients from CU and SD sites. Table4 presents MRCI scores and patient metrics for each cohort by site for descriptive purposes. Mean MRCI and other patient metrics for each disease cohort were very similar between sites with the exception of the geriatric depression cohorts where there was a greater difference between sites than other disease cohorts (Table4). The geriatric depression cohorts from CU were slightly older (mean 81.3 ± 6.1 vs 74.3 ± 7.4 yrs) and had a higher Charlson comorbidity index score (mean 1.98 ± 1.57 vs 0.94 ± 1.16), medication count (12.1 ± 4.9 vs 7.1 ± 3.7), number of comorbidities (24.1 ± 9.8 vs 9.2 ± 6.4), and MRCI score (mean 25.4 ± 11.7 vs 17.6 ± 10.0) than did patients from SD. The number of comorbidities was 2–3 times greater for the CU cohorts with the exception of the HIV cohort. Correlation of MRCI scores with medication count were significant and strong across the four disease states and two sites (p≤0.05, Spearman ρ 0.84–0.93). (Table5) Correlation of MRCI scores with CDS and number of comorbidities was in the moderate range across cohorts and 0.46 and 0.47, respectively, for all cohorts combined. Correlation of MRCI was slightly lower for the Charlson comorbidity index (0.37 for all cohorts combined) and minimal for age and gender (0.08 and 0.06, respectively, for all cohorts combined).

Table 4

MRCI Score and Patient Metrics (N=800)

Site-Cohort	Mean (standard deviation)
	MRCI Score	Medication Count	Charlson Comorbidity Index	Chronic Disease Score	Number of Comorbidities	Age	Female Gender (%)
CU, geriatric depression	25.4 (11.7)	12.1 (4.9)	1.98 (1.57)	1.81 (1.88)	24.1 (9.8)	81.3 (6.1)	79
SD, geriatric depression	17.6 (10.0)	7.1 (3.7)	0.94 (1.16)	2.33 (2.90)	9.2 (6.4)	74.3 (7.4)	76
CU, diabetes	23.0 (11.6)	10.4 (5.0)	2.15 (1.13)	4.85 (2.68)	23.4 (13.1)	59.6 (13.5)	51
SD, diabetes	20.6 (11.8)	7.8 (4.3)	1.36 (1.09)	5.75 (3.54)	7.0 (4.6)	60.5 (13.6)	55
CU, HIV infection	21.8 (12.5)	10.8 (5.9)	1.84 (1.00)	1.75 (1.94)	15.7 (8.6)	49.1 (8.9)	18
SD, HIV infection	21.2 (12.7)	8.5 (5.0)	1.65 (0.88)	1.62 (2.10)	15.7 (8.9)	48.0 (10.1)	16
CU, hypertension	17.8 (9.1)	8.3 (3.8)	1.19 (1.24)	1.82 (2.20)	21.2 (12.2)	64.3 (11.7)	46
SD, hypertension	13.2 (9.6)	5.3 (3.6)	0.57 (0.88)	1.95 (2.86)	6.1 (4.4)	64.3 (14.4)	51
All cohorts	20.1 (11.7)	8.8 (5.0)	1.46 (1.24)	2.74 (2.97)	15.3 (11.3)	62.7 (15.3)	49

MRCI = Medication Regimen Complexity Index; CU = University of Colorado Anschutz Medical Campus; SD = University of California, San Diego; DM = diabetes, HTN = hypertension; HIV = human immunodeficiency virus; GD = geriatric depression.

Table 5

Correlation of MRCI Score with Patient Metrics (N=800)

Site, Cohort	Correlation Coefficient
	Medication Count	Charlson Comorbidity Index	Chronic Disease Score	Number of Comorbidities	Age	Female Gender
CU, geriatric depression	0.84*	0.33*	0.43*	0.46*	0.01	0.04
SD, geriatric depression	0.93*	0.21*	0.60*	0.24*	−0.06	0.01
CU, diabetes	0.89*	0.42*	0.31*	0.57*	0.18	0.03
SD, diabetes	0.89*	0.26*	0.49*	0.45*	0.11	0.05
CU, HIV infection	0.92*	0.40*	0.56*	0.64*	0.25*	0.02
SD, HIV infection	0.93*	0.16	0.65*	0.32*	0.20*	0.10
CU, hypertension	0.84*	0.34*	0.40*	0.48*	−0.01	0.11
SD, hypertension	0.89*	0.22*	0.55*	0.38*	0.14	0.13
All cohorts	0.90*	0.37*	0.46*	0.47*	0.08*	0.06*

MRCI = Medication Regimen Complexity Index; CU = University of Colorado Anschutz Medical Campus; SD = University of California, San Diego; DM = diabetes, HTN = hypertension; HIV = human immunodeficiency virus; GD = geriatric depression.

p<0.05.

Discussion

This study broadened the validity testing of the MRCI to be more pertinent to identifying patients for subsequent MTM interventions by using a patient-level MRCI score and clinician ratings of “expected patient difficulty managing medication regimens, thus warranting intervention.” Our investigation revealed moderate to strong concordance between patient-level MRCI scores and expert ranking of patient medication regimen complexity. Expert ranking of medication regimen complexity was more often in concordance with MRCI rankings than rankings based on medication count alone. The addition of more-detailed “administration directions” available in EMR data, but not in claims data, did not improve concordance of expert medication regimen complexity ranking and MRCI ranking. In fact, fewer experts were in concordance with MRCI rankings when the extra EMR information was available for the multiple-disease cohort. The strengths of correlations with expert opinion were comparable to those reported in the original MRCI validation study (Spearman ρ range 0.657–0.943). However, the original MRCI validation study used only prescription medications in the MRCI score, evaluated patients from a single-disease cohort (chronic obstructive pulmonary disease), and used a definition of “medication regimen complexity” that explicitly did not consider any nonmedication factors.5 Convergent validity of the MRCI was supported by strong positive correlations with medication count and moderate positive correlations with measures of morbidity (CDS and number of comorbidities). The minimal level of correlation of MRCI scores with age and gender supported the divergent validity of the MRCI. These findings were consistent for patient cohorts from each site (CU and SD) and were similar to other studies that have included some psychometric testing of the MRCI.12,13,15,19

There are several implications of our findings. First, our investigation used an applied definition of “medication regimen complexity” that was focused on the practical need to identify patients for whom their medication regimen could be problematic and would therefore likely benefit from subsequent intervention. Expert panel members endorsed this concept as one relevant and consistent to that used in their practices. Our definition of “medication regimen complexity” and decision to use a patient-level MRCI were both supported during the consensus session when experts discussed their reasoning for medication regimen complexity rankings to reach a single consensus ranking. During the consensus session, there was extensive discussion of factors that influence assessment of patient-level medication regimen complexity including pill burden, possible drug–drug or drug–food interactions, age and disability issues, and number and types of comorbid conditions, especially as related to cognitive decline or impairment. Our finding that more expert medication regimen complexity rankings were in concordance with MRCI rankings than medication count rankings also supports the multifactorial nature of medication regimen complexity in a clinician's mind and the usefulness of an index that quantifies dosing forms, dosing frequency, and additional administration directions.

A second implication of our study is related to the type of information available from EMR versus claims data. When our expert clinicians had more-detailed information regarding administration directions (essentially the third component of the MRCI) that would be available from an EMR, but not claims data, the number of experts with medication regimen complexity rankings in concordance with MRCI rankings declined. The reason for this is not clear and warrants further investigation. One explanation may be that there is a point of too much information for clinicians to synthesize. If this is true, using the MRCI to rank medication regimen complexity could be particularly useful for clinicians in general practice settings who are treating a wide array of patients. Another explanation may be that there is a point of diminishing return for additional information that may have implications for automation of the MRCI using claims or EMR databases. Because, at least in this study, supplementary EMR data did not consistently improve concordance, the additional effort, time, and expense required to augment databases with more-detailed administration directions may not be cost effective. For example, McDonald and colleagues spent considerable time and effort with multiple committee meetings to first decide how to automate the additional administration directions section of the MRCI and then in the regimen coding process scanning free text fields searching for extra administration directions.18 For the practical purpose of identifying patients for MTM, it may be more useful to augment use of MRCI scores with other easily coded indicators of patients with complicated medication regimens, such as medications on the Beers list23 for elderly populations, medications requiring regular laboratory monitoring, or medications with greater frequency of significant adverse events.

Finally, the MRCI is limited, by definition, to assessing the complexity of a patient's medication regimen. However, in this study, we included other measures that are important in assessing the overall level of “patient complexity.” Overall patient complexity is a broader concept beyond a patient's comorbid disease states and his/her medications that has been described as including socioeconomic, cultural, biology/genetic, environmental, and behavioral factors.24 Similarly, the need to assess elements of patient complexity when considering the need for medication changes has been described as multifactorial and inclusive of many of the same factors such as clinical, comorbidities, complications, socioeconomic, and behavioral.25 In our validation study, experts were considering medication regimens in context of patients' gender, age, and comorbidities, but we did not include other factors, such as socioeconomic status (e.g., ability to access medications) and behavioral (e.g., adherence patterns) that could also be included in efforts to target patients. A recent study created and tested a patient-reported medication user self-evaluation (MUSE) tool that did include nonmedication factors to identify Medicare Part D patients who may benefit from comprehensive medication reviews.26 While four of the seven MUSE elements could be populated via pharmacy claims databases (number of prescription medications, medical conditions, pharmacies, and prescribers), the remainder (forgot to take medications, not filled or stopped taking medication due to cost, hospital admission within past 6 months) must be supplied by the patient or medical claims data. Further study of a variety of broader patient complexity factors and the added value of including patient-reported factors to claims-based factors versus the added cost and effort to obtain should be conducted with the goal of efficiently identifying patients who are likely to benefit most from interventions to improve the outcomes of medication usage.

We acknowledge limitations to this work. This study was conducted using a limited number of patient cases and expert clinicians from clinics from only two universities, so results may differ in different patient populations or using different clinicians. However, our sample size was much larger than any other MRCI validation study and our patients were drawn from multiple ambulatory clinics at each university. Also, our study was limited to patients with one of four possible targeted disease states, although they were not excluded for any presence or lack of comorbidities. Results may differ in other populations and in patients with other disease states, although based on prior work comparing a set of four disease-defined cohorts (GD, DM, HTN, HIV infection), the distributions of complexity scores shifted, but factors comprising complexity were consistent (e.g., prescription medications, dosing frequency).20 Finally, this study used cross-sectional, retrospective data; therefore, we were not able to assess the predictive validity of the MRCI or its relationship with longitudinal estimates, or patient-reported adherence, or other patient-reported variables such as their perceived difficulty in managing their medication regimen.

Future testing should examine the relationship of MRCI scores with adherence, patient cognition, and other patient and health care utilization outcomes, as well as within other care settings (e.g., long-term care facilities) that may have different patterns of MRCI relationships than those observed in ambulatory clinics. Further evaluation of the content validity of the MRCI as a tool to identify patients who may benefit from MTM intervention is needed because the content validity of the MRCI has only been assessed by clinicians, not patients who may or may not endorse the elements of the MRCI, but also may identify other concepts that contribute significantly to their ability to manage their medication regimen. If the tool becomes useful as information at the point of care, developing quick access to on-demand scoring by providers or even patients may have utility to improve care safety and effectiveness.

In conclusion, this study supported the validity of a patient-level MRCI as a tool to identify patients who may benefit from MTM intervention (e.g., to improve adherence, or risk of drug-related events). There was a high degree of concordance between MRCI and expert pharmacist rankings of medication regimen complexity, and convergent and divergent validity of the MRCI was supported using measures of patient complexity and other patient factors. The patient-level MRCI is a valid and useful tool for assessing medication regimen complexity that can be used with data commonly found in claims and EMR databases.