Using Electronic Medical Records and Health Claim Data to Develop a Patient Engagement Score for Patients With Multiple Chronic Conditions: An Exploratory Study.

The study objective was to (1) develop a statistical model that creates a novel patient engagement score (PES) from electronic medical records (EMR) and health claim data, and (2) validate this developed score using health-related outcomes and charges of patients with multiple chronic conditions (MCCs). This study used 2014-16 EMR and health claim data of patients with MCCs from Sanford Health. Patient engagement score was created based on selected patients' engagement behaviors using Gaussian finite mixture model. The PES was validated using multiple logistic and linear regression analyses to examine the associations between the PES and health-related outcomes, and hospital charges, respectively. Patient engagement score was generated from 5095 patient records and included low, medium, and high levels of patient engagement. The PES was a significant predictor for low-density lipoprotein, emergency department visit, hemoglobin A1c, estimated glomerular filtration rate, hospitalization, and hospital charge. The PES derived from patient behaviors recorded in EMR and health claim data can potentially serve as a patient engagement measure. Further study is needed to refine and validate the newly developed score.

Introduction

Controlling health care costs and preventing adverse health outcomes for patients with multiple (2 or more) chronic conditions (MCCs) continue to be a challenge since the management of MCCs are highly influenced by patient behaviors. Recently, patient engagement has become a major focus of health care reform as it has been shown to improve health outcomes and lower health care cost (1 –7). Patient activation measure (PAM) is a validated instrument that has been widely used to measure the level of patient engagement in real-world practice. Higher PAM score indicates greater activation and PAM segments patients into 4 activation levels, including (1) the level that patients are passive and lack confidence, (2) the level that patients become aware, but they are still struggling, (3) the level that patients take action, and (4) the level that patients can maintain behaviors (8). However, routinely assessing PAM is challenging since it requires patients to spend time and respond to the questions (9). In addition, while the level of patient engagement could change over time, especially for chronic conditions (9).

Electronic medical record (EMR) and health claim data have been widely used for several purposes in the US health care. Some EMR and claim data reflect patient engagement behaviors in various health care activities, for example, immunizations, medication refills, and wellness exams. This study intended to explore the use of these behaviors available in the EMR and health claims from a large regional health care provider to determine the level of patient engagement. This might enable health care providers to routinely capture the level of patient engagement without burden to patients. Thus, the objectives of this study were to develop a statistical model that generated a novel patient engagement score (PES) from EMR and health claims of patients with MCCs, and to validate this developed score using health-related outcomes and charges.

Methods

We conducted a cross-sectional study since only the most recent records of the majority of data elements were available. Figure 1 shows the conceptual framework of this study. We built the framework from Andersen’s behavioral model of health service utilization (10 –13). The Andersen’s model suggests that predisposing, enabling, and need factors could influence health-related outcomes (11). Based on various studies (1 –7,14 –16) indicating that patient engagement improved health-related outcomes and lowered health care cost, patient engagement was included in the study framework. Also, previous studies reported that behavioral measures, for example, adherence to medication and self-care behavior, were linked to a high level of patient activation (17 –19). This study proposed to construct a patient engagement measure from these behaviors.

Figure 1.

Conceptual framework adapted from Andersen’s model.

Data Collection

This study obtained data through the Sanford Data Collaborative. The data comprised EMR data from Sanford Health, including 44 hospitals, as well as claim data over a 3-year (calendar years 2014-2016) period. All data were de-identified. The data elements included International Classification of Diseases, Tenth Revision, age, gender, race, ethnicity, marital status, primary insurance, primary care provider, tobacco use, smokeless cigarette use, alcohol use, body mass index (BMI), PAM score, number of immunizations, number of wellness exams, number of chronic conditions, systolic blood pressure (SBP), diastolic blood pressure (DBP), high-density lipoprotein (HDL), low-density lipoprotein (LDL), hemoglobin A1c, number of emergency department (ED) visits, number of hospitalizations, estimated glomerular filtration rate (eGFR), number of missing scheduled visits, clinic charges, and hospital charges. Only cumulative values over the 3-year period were available for the number of immunizations, wellness exams, ED visits, hospitalizations, and missing scheduled visits. Late medication refills were determined from 3 variables, including medication name, the number of day supply, and prescription date, using Sanford Health claim data. We assumed that the late medication refill occurred when the number of days supply was shorter than the number of days between 2 consecutive prescription refill dates. Then, the proportion of late refills was calculated for each patient. This study included patients who were aged at least 18 years old and had MCCs with one of the following primary diagnoses: hypertension, hyperlipidemia, ischemic heart disease, heart failure, diabetes, and chronic kidney disease; since these diseases were among highest chronic condition rates with highest rates of MCCs. This resulted in a total of 147 687 individual patient records.

Data Analysis

We conducted descriptive analyses of all data elements for both patients with PAM and PES scores. To explore the potential use of the EMR and health claim data to determine the level of patient engagement, 3 behavioral measures, including the proportion of late medication refills, the number of missing scheduled visits, and the number of immunizations over the 3-year period were purposely selected, since they likely reflected how much patients engaged in their health care. We then developed the PES score by using data from 5095 patients with complete records of these 3 behavioral measures. A finite mixture of Gaussian model was used to group patients based on the behavioral measures into a collection of nonhomogeneous distributions (clusters) (20,21). The model we assumed a G-component Gaussian mixture given by , where the component density function φ was Gaussian distribution with mean vector μ and covariance matrix Σ . The mixing proportions π had restrictions 0 < π ≤ 1 and . Here, θ contained {π , μ , Σ , g = 1,…, G}. Since, it was assumed that the data came from a mixture of multivariate normal distributions; hence, the 3 variables were log-transformed to have an approximately normal mixture distribution. The expectation-maximization algorithm was used to find the parameter estimates of the model at different numbers of groups. The iterative algorithm was stopped after the relative difference in likelihood value was below a given threshold. The optimal number of groups was identified using the Bayesian information criterion (BIC) (21). After obtaining the component parameters for the best model, maximum a-posteriori estimate was used by which an individual was assigned to a component with the highest posterior probability of belonging was used to assign individuals to clusters. To generate the PES score, the clusters were ranked based on the mean vector by which converting clusters labels to ordered categorical variable.

We used multiple logistic regression analyses to examine the relationship between all outcomes (dependent variable)—SBP, DBP, HDL, LDL, A1c, eGFR (normal vs not normal range), and ED visit and hospitalization (yes vs no) and independent variables—age, gender, race, marital status, primary health insurance, primary care provider, BMI, tobacco use, alcohol use, number of chronic conditions, PAM, or PES score. We used multiple linear regression analyses to examine the relationship between the same independent variables and hospital charges. However, since the hospital charge had a right screwed distribution, we log-transformed the charge before we conducted the analyses. A 2-sided P value of <.05 was considered statistically significant.

Results

Table 1 shows the descriptive statistics of all study variables for patients with PAM scores and patients with PES scores. Only 1442 patient records had PAM scores. The average ages of patients with PAM and PES scores were about 67 and 56 years old, respectively. Approximately 64% and 49% of the patients with PAM and PES scores were female, respectively. More than 90% of the patients were white and non-Hispanic or Latino for both groups. The majority of the patients with PAM scores and the patients with PES scores were married (55.3% and 64.1%, respectively), had primary care providers (94.2% and 94.1%, respectively), and were former or nonsmokers (89.7% and 82.0%, respectively). Although 65.9% of the patients with PAM scores had government-sponsored health insurance, 16.1% of the patients with PES scores had this type of insurance. Less than 40% of the patients with PAM scores, but 52.0% of the patients with PES scores, consumed alcoholic beverages. The average BMI (25.3) of the patients with PES scores were lower than those of the patients with PAM scores (32.3). Over 58% of the patients with PAM scores had the scores at either 3 or 4. Both groups had similar health-related outcomes. The average hospital charges for the patients with PAM scores, and the patients with PES scores were $111 268 and $66 986, respectively.

Table 1.

Patients’ Characteristics, Health Behavior, and Health-Related Outcomes for All Patients, Patients With PAM Scores, and Patients With PES Scores.

Variables	Patients with PAM scores	Patients with PES scores
Average age ± SD years	67.3 ± 12.9	55.8 ± 10.8
	N = 1442	N = 5095
Gender, # (%)
Female	916 (63.5)	2527 (49.6)
Male	526 (36.5)	2568 (50.4)
Race, # (%)
Black	19 (1.3)	122 (2.4)
American Indian or Alaskan Native	50 (3.5)	195 (3.8)
White	1348 (93.5)	4658 (91.4)
Asian	7 (0.5)	55 (1.1)
Native Hawaiian or Pacific	2 (0.1)	7 (0.1)
Islander
Missing	16 (1.1)	58 (1.1)
Ethnicity, # (%)
Hispanic or Latino	12 (0.8)	99 (1.9)
Not Hispanic or Latino	1405 (97.4)	4953 (97.2)
Missing	25 (1.7)	43 (0.8)
Marital status, # (%)
Married	797 (55.3)	3266 (64.1)
Single	217 (15.0)	1168 (22.9)
Others	428 (29.7)	652 (12.8)
Missing	–	9 (0.2)
Primary insurance, # (%)
Government sponsored	950 (65.9)	821 (16.1)
Nongovernment sponsored	470 (32.6)	3912 (76.8)
Missing	22 (1.5)	362 (7.1)
Primary care provider, # (%)
Yes	1359 (94.2)	4793 (94.1)
No	83 (5.8)	302 (5.9)
Missing	–	–
Tobacco use, # (%)
Current every day smoker	149 (10.3)	917 (18.0)
Former smoker	591 (41.0)	1766 (34.7)
Never smoker	702 (48.7)	2409 (47.3)
Missing	–	3 (0.1)
Smokeless cigarette use, # (%)
Current every day smoker	10 (0.7)	143 (2.8)
Former smoker	60 (4.2)	298 (5.8)
Never smoker	1309 (90.8)	4434 (87.0)
Missing	63 (4.4)	220 (4.3)
Alcohol use, # (%)
Yes	557 (38.6)	2651 (52.0)
No	835 (57.9)	2317 (45.5)
Missing	50 (3.5)	127 (2.5)
Average BMI ± SD	32.3 ± 6.7	25.3 ± 7.6
	N = 1399	N = 5070
PAM score, # (%)
Level 1	153 (10.6)	8 (0.2)
Level 2	445 (30.9)	17 (0.3)
Level 3	331 (23.0)	24 (0.5)
Level 4	513 (35.6)	41 (0.8)
Missing	–	5005 (98.2)
Average # of immunizations ± SD	7.1 ± 4.9	6.2 ± 4.9
	N = 1274	N = 5095
Average # of wellness exams	1.3 ± 0.6	1.5 ± 0.8
	N = 3	N = 5095
Average % late medication refills ± SD	55.7 ± 22.4	54.1 ± 23.4
	N = 125	N = 5095
Average # of chronic conditions	3.1 ± 1.2	2.5 ± 1.0
	N = 1442	N = 5095
Average SBP ± SD	127.0 ± 14.9	126.3 ± 14.6
	N = 1441	N = 5094
Average DBP ± SD	72.2 ± 11.0	75.9 ± 10.2
	N = 1441	N = 5092
Average HDL ± SD	47.0 ± 13.2	46.9 ± 13.3
	N = 1380	N = 4601
Average LDL ± SD	93.7 ± 32.1	99.4 ± 33.8
	N = 1367	N = 4559
Average A1C ± SD	7.0 ± 1.3	6.8 ± 1.3
	N = 996	N = 2879
Average # of ED visits ± SD	3.2 ± 2.5	2.5 ± 2.1
	N = 771	N = 2267
Average # of hospitalizations ± SD	2.2 ± 1.4	1.7 ± 1.1
	N = 547	N = 1662
Average eGFR ± SD	61.6 ± 17.0	69.6 ± 15.8
	N = 1259	N = 3619
Average # of missing scheduled visits ± SD	4.1 ± 3.8	3.3 ± 3.3
	N = 924	N = 5095
Average hospital charge ± SD	$111 268.0 ± $208 464.4	$66 986.5 ± $116 841.9
	N = 1396	N = 4379

Abbreviations: BMI, body mass index; DBP, diastolic blood pressure; ED, emergency department; eGFR, estimated glomerular filtration rate; HDL, high-density lipoprotein; LDL, low-density lipoprotein; PAM, Patient Activation Measure; PES, patient engagement score; SBP, systolic blood pressure; SD, standard deviation.

From the Gaussian model, a 3-component mixture was chosen. The best model based on BIC assumes the covariance matrices associated with the groups are symmetric and equal variance for all 3 variables within a group. By inspection of the associated behavioral characteristics, the groups were labeled as low (1), medium (2), and high levels of engagement (22). The mean number of immunizations was 4.632, 4.980, and 5.245, whereas the mean number of missing scheduled visits was 2.627, 2.548, and 1.100 when the patient engagement was at low, medium, and high levels, respectively. Similarly, the mean proportion of late medication refills was 3.568 when the patient engagement was at low level and 0 when the patient engagement was at medium and high levels.

Tables 2 and 3 show the associations between PAM/PES scores and health-related outcomes and hospital charges, respectively. After controlling for all variables, the PAM score was not a significant factor associated with any of the health-related outcomes, except hospital charges. Patients with the lowest level of PAM score had significantly higher hospital charges (β^[SE] = 0.288 [0.145], P = .046), compared to patients with the highest level of PAM score. The patients with medium PES score had significantly higher odds of having abnormal A1c (odds ratio [OR] [95% CI] = 1.340 [1.110-1.617], P = .002), at least one ED visit over the 3-year period (OR [95% CI] = 1.264 [1.093-1.462], P = .002), at least one hospitalization over the 3-year period (OR [95% CI] = 1.323 [1.134-1.542], P < .001), and had higher hospital charges (β^[SE] = 0.291 [0.060], P < .001) than those patients with the highest PES score. For eGFR, patients with the lowest PES scores had significantly higher odds of having abnormal kidney function (OR [95% CI] = 1.278 [1.064-1.534], P = .009), compared to the patients with the highest PES score. For LDL, while the patients with the lowest PES scores had significantly higher odds of having abnormal LDL (OR [95% CI] = 1.182 [1.005-1.390], P = .043), the patients with medium PES scores had significantly lower odds (OR [95% CI] = 0.818 [0.702-0.955], P = .011), compared to the patients with the highest PES scores.

Table 2.

Multiple Logistic Regression Results for the Association Between Either PAM or PES, and Health-Related Outcomes.

Variable	Odds ratio (95% CI), P value
	Systolic blood pressurea		Diastolic blood pressurea
	PAM	PES	PAM	PES
Age	1.008 (0.991-1.025), .361	1.010 ( 1 .001-1.020), .039	0.965 (0.942-0.990), .005	0.969 (0.958-0.980), <.001
Gender
Female Maleb	0.966 (0.666-1.401), .855	1.253 ( 1 .036-1.514), .020	0.586 (0.326-1.053), .074	1.642 ( 1 .267-2.129), <.001
Race
White Nonwhiteb	1.257 (0.609-2.596), .536	1.438 ( 1 .026), .035	1.515 (0.569-4.040), .406	1.173 (0.757-1.817), .475
Marital status
Married Single & othersb	1.041 (0.726-1.494), .827	1.332 (1.092-1.623), .005	1.207 (0.659-2.209), .543	1.282 (0.981-1.674), .069
Health insurance
Government sponsored Nongovernment sponsoredb	0.751 (0.495-1.140), .179	1.191 (0.934-1.517), .158	0.573 (0.298-1.102), .095	1.128 (0.791-1.607), .506
Primary care provider
No Yesb	1.237 (0.591-2.588), .572	1.323 (0.920-1.903), .131	1.601 (0.585-4.383), .359	1.319 (0.818-2.127), .257
BMI	1.003 (0.977-1.030), .806	1.022 ( 1 .008-1.037), .002	0.974 (0.932-1.019), .252	1.020 ( 1 .002-1.039), .032
Tobacco use
Former smoker & Nonsmoker Smokerb	0.889 (0.475-1.665), .714	1.501 ( 1 .195-1.886), <.001	1.480 (0.685-3.197), .319	1.920 ( 1 .448-2.545), <.001
Alcohol
No Yesb	1.530 ( 1 .053-2.224), .026	0.992 (0.822-1.196), .931	0.984 (0.545-1.777), .957	0.992 (0.769-1.280), .949
Number of chronic conditions	1.047 (0.983-1.114), .153	0.982 (0.928-1.038), .520	1.034 (0.925-1.156), .553	0.919 (0.841-1.005), .064
PAM or PES
	1 0.735 (0.390-1.386), .342  2 0.936 (0.618-1.416), .753  3 0.846 (0.531-1.347), .480  4b	1 0.954 (0.756-1.205), .695  2 0.931 (0.747-1.160), .524  3b	1 0.981 (0.348-2.768), .971  2 1.188 (0.601-2.349), .619  3 0.720 (0.318-1.634), .432  4b	1 1.076 (0.792-1.464), .638  2 1.057 (0.783-1.427), .718  3b
N	1316	4465	1316	4465
Naqelkerke R 2	0.018	0.021	0.082	0.021
-2 Log likelihood	938.017	3197.349	407.037	1958.338
Model χ2 (P value)	11.998 (.528)	48.080 (<.001)	30.916 (.003)	96.444 (<.001)
	Odds ratio (95% CI), P value
	HDLa		LDLa
	PAM	PES	PAM	PES
	0.980 (0.986-0.993), .002	0.986 (0.979-0.994), <.001	0.979 (0.967-0.991), .001	0.986 (0.979-0.993), <.001
	0.246 (0.187-0.324), <.001	3.687 ( 3 .160-4.301), <.001	1.657 ( 1 .270-2.162), <.001	0.584 (0.512-0.666), <.001
	1.102 (0.617-1.968), .744	0.875 (0.647-1.184), .388	0.753 (0.424-1.337), .333	0.919 (0.699-1.208), .543
	1.136 (0.859-1.503), .372	1.055 (0.898-1.240), .514	0.854 (0.660-1.105), .230	0.866 (0.750-1.000), .050
	1.358 (0.982-1.879), .065	1.070 (0.873-1.311), .513	0.737 (0.548-0.992), .044	0.865 (0.719-1.041), .126
	1.229 (0.654-2.310), .522	1.347 (0.957-1.894), .087	1.340 (0.742-2.417), .332	0.700 (0.498-0.982), .039
	1.039 ( 1 .018-1.060), <.001	1.052 ( 1 .040-1.064), <.001	0.973 (0.954-0.992), .005	0.994 (0.984-1.004), .238
	1.821 ( 1 .191-2.785), .006	1.696 ( 1 .403-2.050), <.001	0.604 (0.392-0.932), .023	1.122 (0.940-1.340), .203
	1.293 (0.982-1.703), .067	1.474 (1.274-1.707), <.001	0.984 (0.765-1.267), .901	0.911 (0.799-1.039), .164
	1.114 ( 1 .059-1.171), <.001	1.157 ( 1 .110-1.206), <.0001	0.844 (0.795-0.896), <.001	0.839 (0.803-0.877), <.001
	1 1.306 (0.833-2.049), .245	1  1.093 (0.911-1.311), .339	1 0.994 (0.639-1.544), .977	1 1.182 (1.005-1.390), .043
	2 1.289 (0.933-1.781), .124	2  1.178 (0.994-1.396), .059	2 0.791 (0.586-1.069), .127	2 0.818 (0.702-0.955), .011
	3 1.080 (0.759-1.536), .669	3b	3 1.001 (0.727-1.377), .997	3b
	4b		4b
	1263	4060	1253	4024
	0.201	0.176	0.124	0.079
	1380.554	4482.109	1535.516	5306.996
	195.042 (<0.001)	540.820 (<0.001)	119.396 (<0.001)	245.013 (<0.001)
	Odds ratio (95% CI), P value
	ED visita		A1Ca
	PAM	PES	PAM	PES
	0.999 (0.986-1.011), .815	0.985 (0.978-0.991), <.001	0.985 (0.971-0.999), .033	0.980 (0.971-0.988), <.001
	0.989 (0.762-1.284), .935	1.098 (0.969-1.244), .144	0.972 (0.724-1.305), .849	0.986 (0.835-1.165), .872
	1.134 (0.636-2.020), .670	1.238 (0.965-1.589), .093	1.647 (0.863-3.143), .130	1.369 ( 1 .001-1.872), .049
	1.631 ( 1 .266-2.101), <.001	1.192 ( 1 .042-1.363), .011	1.115 (0.835-1.490), .459	1.007 (0.844-1.202), .934
	1.669 ( 1 .242-2.444), .001	1.440 ( 1 .212-1.709), <.001	1.019 (0.721-1.441), .914	0.862 (0.689-1.079), .196
	1.004 (0.575-1.755), .988	0.841 (0.638-1.110), .221	0.974 (0.487-1.948), .941	0.985 (0.65401.482), .941
	0.999 (0.980-1.018), .887	0.988 (0.979-0.997), .012	1.037 ( 1 .015-1.060), .001	1.016 ( 1 .003-1.029), .017
	1.559 ( 1 .013-2.398), .044	1.305 ( 1 .107-1.539), .002	0.971 (0.601-1.568), .904	1.113 (0.891-1.390), .346
	1.794 ( 1 .401-2.297), <.001	1.274 ( 1 .125-1.443), <.001	1.324 (0.992-1.768), .057	1.118 (0.950-1.317), .180
	1.302 ( 1 .219-1.389), <.001	1.209 (1.161-1.260)	1.095 ( 1 .035-1.159), .002	1.177 ( 1 .121-1.235), <.001
	1 1.035 (0.660-1.623), .880	1   1.071 (0.917-1.250), .386	1 0.882 (0.550-1.416), .604	1   1.015 (0.826-1.246), .888
	2 0.823 (0.611-1.108), .198	2 1.264 (1.093-1.462), .002	2 1.292 (0.919-1.817), .141	2 1.340 ( 1 .110-1.617), .002
	3 1.049 (0.761-1.446), .772	3b	3 1.122 (0.775-1.623), .542	3 b
	4b		4b
	1273	4416	901	2507
	0.185	0.061	0.065	0.055
	1558.865	5862.083	1179.358	3368.688
	188.875 (<0.001)	206.639 (<0.001)	44.267 (<0.001)	105.714 (<0.001)
	Odds ratio (95% CI), P value
	eGFRa		Hospitalizationa
	PAM	PES	PAM	PES
	1.072 ( 1 .053-1.090), <.001	1.062 ( 1 .054-1.070), <.001	1.018 ( 1 .005-1.031), .008	1.001 (0.994-1.008), .737
	3.063 ( 2 .078-4.515), <.001	0.542 (0.467-0.629), <.001	0.761 (0.576-1.004), .054	1.236 ( 1 .080-1.413), .002
	1.110 (0.534-2.310), .779	0.725 (0.552-0.953), .021	1.039 (0.583-1.851), .897	0.942 (0.723-1.228), .657
	0.944 (0.629-1.418), .782	0.979 (0.836-1.147), .796	1.371 ( 1 .050-1.790), .020	1.087 (0.941-1.256), .255
	0.896 (0.581-1.382), .619	1.069 (0.862-1.326), .543	1.570 ( 1 .142-2.158), .005	1.647 ( 1 .382-1.963), <.001
	2.063 (0.793-5.372), .138	1.497 ( 1 .053-2.128), .025	1.852 ( 1 .034-3.314), .038	1.386 ( 1 .049-1.832), .022
	1.019 (0.990-1.048), .209	0.988 (0.977-0.999), .027	1.003 (0.983-1.023), .790	0.989 (0.979-0.999), .035
	0.702 (0.422-1.169), .174	0.705 (0.587-0.845), <.001	1.695 ( 1 .100-2.611), .017	1.101 (0.92401.311), .282
	0.567 (0.381-0.845), .005	1.017 (0.879-1.177), .820	1.444 ( 1 .107-1.884), .007	1.424 ( 1 .247-1.626), <.001
	1.129 ( 1 .029-1.239), .011	1.063 ( 1 .014-1.534), .011	1.434 ( 1 .340-1.534), <.001	1.270 ( 1 .218-1.325), <.001
	1 0.570 (0.312-1.041), .067	1 1.278 ( 1 .064-1.534), .009	1 1.052 (0.671-1.649), .825	1  0.960 (0.811-1.137), .638
	2 1.105 (0.686-1.780), .681	2  1.027 (0.867-1.217), .754	2 0.896 (0.653-1.228), .493	2 1.323 ( 1 .134-1.542), <.001
	3 1.013 (0.613-1.674), .961	3b	3 1.059 (0.753-1.489), .742	3 b
	4b		4b
	1305	4298	1257	4434
	0.229	0.144	0.252	0.093
	775.735	4570.752	1435.397	5294.774
	163.454 (<0.001)	449.968 (<0.001)	259.116 (<0.001)	304.429 (<0.001)

Abbreviations: BMI, body mass index; ED, emergency department; eGFR, estimated glomerular filtration rate; HDL, high-density lipoprotein; LDL, low-density lipoprotein; PAM, patient activation measure; PES, patient engagement score.

a Coded as binary variable (normal vs not normal)

b Reference group.

Table 3.

Linear Regression Results for the Association Between PAM/PES and Hospital Charges.

Variable	Ln (hospital charge)a Coefficient (SE), P value
	PAM	PES
Age	0.004 (0.004), .294	0.012 (0.003), <.001
Gender
Female Maleb	−0.063 (0.088), .476	−0.241 (0.051), <.001
Race
White Nonwhiteb	−0.089 (0.189), .638	0.124 (0.104), .230
Marital status
Married Single & othersb	−0.210 (0.086), .015	−0.144 (0.056), .010
Health insurance
Government sponsored Nongovernment sponsoredb	−0.686 (0.102), <.001	−0.327 (0.068), <.001
Primary care provider
No Yesb	−0.141 (0.191), .461	−0.319 (0.114), .005
BMI	−0.006 (0.006), .377	−0.005 (0.004), .223
Tobacco use
Former smoker & Nonsmoker Smokerb	−0.210 (0.141), .137	−0.058 (0.068), .399
Alcohol
No Yesb	−0.303 (0.085), <.001	−0.251 (0.051), <.001
Number of chronic conditions	0.237 (0.016), <.001	0.211 (0.015), <.001
PAM or PES	1c     0.288 (0.145), .046  2c   0.036 (0.101), .719 3c   0.169 (0.109), .123	1c  −0.075 (0.064), .241    2c  0.291 (0.060), <.001
N	1281	3857
R 2	0.251	0.115

Abbreviations: BMI, body mass index; PAM, patient activation measure; PES, patient engagement score.

a Logarithmic transformation.

b Reference group.

c Dummy code (PES score 3 is reference group).

Discussions

In general, the patient characteristics, health behavior, and health-related outcomes of the patients with PAM scores, and the patients with PES scores were similar. However, it is noteworthy that the patients with PAM scores were likely younger, female, had nongovernment sponsored health insurance, consumed alcoholic beverages, and had slightly lower BMI. PAM score was a significant predictor for only hospital charges, whereas PES score was a significant predictor for several outcomes, for example, LDL, ED visit, A1c, eGFR, hospitalization, and hospital charges.

Interestingly, the PAM score was a significant predictor for only hospital charges in this study. Several previous studies showed that patient activation, as a proxy of patient engagement, was a significant predictor of health-related outcomes (2,23 –25). Besides the demographic differences, one of the reasons could be that the PAM score was not fully implemented at Sanford Health during the study period. The records showed that only 1% of all patients with MCCs had PAM scores. These results supported the rationale of this study, which was to develop another patient engagement measurement from existing data in order to reduce any patient burnout or PAM survey administration. In addition, the PAM score was criticized for its inability to fully capture the broad concept of patient engagement (26).

Based on 3 real behavioral measures (average percentage of late medication refills, average number of missing scheduled visits, and average number of immunizations) of patient engagement with their health care, PES was developed in this study. We used a χ2 test to examine the association between PAM and PES scores from the same individuals and found no significant relationship (P = .957). However, the test was based on a small sample size (only 90 individual patients) who had both scores. Despite the lack of statistical association, the PES score was a significant predictor for several outcomes, such as LDL, ED visit, A1c, eGFR, hospitalization, and hospital charges. These results showed evidence that the PES score captured the engagement level of the patients with MCCs and their health-related outcomes. To our knowledge, this was the first study that used existing EMR and health claim data to define the level of patient engagement. These results provided an example for other health care settings to generate PES scores for their patients with MCCs and use them to make decisions or recommendations without the need to administer any surveys, which likely cause patient burnout. Furthermore, the PES score can be used to motivate patients to improve their engagement (eg, their medication adherence), to improve patient experience of care, to make decisions, and to make a recommendation that creates an environment promoting and encourage engagement. However, some relationships between the PES scores and those outcomes in this study indicated room for improvement in the current PES score. For instance, only the patients with medium PES scores had significantly higher odds of having abnormal A1c, ED visit, hospitalization, and had higher hospital charge, relative to those with the highest PES scores, while those patients with the lowest PES score did not; which was a counterintuitive finding. Similarly, the findings related to LDL were counterintuitive.

This project suffered from various limitations. First, this study used secondary data from EMR and health claims. Therefore, all study variables were limited. Second, the PES score was based on only 3 behavior variables due to data limitation. For instance, data measuring other dimensions of patient engagement were not available. Having access to other potential behaviors, including patient access to EMR, rescheduling a visit, and patient-led communication (eg, emails via EMR, reminders, and phone calls), would improve the performance of the PES score. Third, although all 3 variables were captured across the study period, most of the health-related outcomes in this study were based on cross-sectional data. Fourth, an inconsistency between the prescribing dates and the number of day supplies was observed. The late medication refill was assumed to occur when the number of day supply was shorter than the number of days between 2 consecutive prescription dates. This assumption could be too restrictive because patients might still possess some medications from their last refills to cover those late few days. In other words, the late medication refills in this study might be overestimating the actual late refills. Fifth, since the PES was based on EMR and health claims, it could not be used to assess patient engagement in real time for new patients. Last, to assess patient engagement without engaging directly with the patient might result in an incomplete picture and a lack of understanding of other factors that could be driving health outcomes.

Conclusions

In our study, the PES score derived from patient behaviors in EMR and health claim data drastically outperformed the PAM score, which was based on patient survey. It was a significant predictor for various health-related outcomes, such as LDL, ED visit, A1c, eGFR, hospitalization, and hospital charge. Further studies are needed to improve the PES score, and to incorporate its use into clinical settings to promote patient engagement and subsequently improve the care management of patients with MCCs.