Electronic blood glucose monitoring impacts on provider and patient behavior.

OBJECTIVE: Recent technological development along with the constraints imposed by the coronavirus disease 2019 (COVID-19) pandemic have led to increased availability of patient-generated health data. However, it is not well understood how to effectively integrate this new technology into large health systems. This article seeks to identify interventions to increase utilization of electronic blood glucose monitoring for patients with diabetes.
MATERIALS AND METHODS: A large randomized controlled trial tested the impact of multiple interventions to promote use of electronic blood glucose tracking. The total study sample consisted of 7052 patients with diabetes across 68 providers at 20 selected primary care offices. The design included 2 stages: First, primary care practices were randomly assigned to have their providers receive education regarding blood glucose flowsheet orders. Then, patients in the treated practices were assigned to 1 of 4 reminder interventions.
RESULTS: Provider education successfully increased provider take-up of an online blood glucose monitoring tool by 64 percentage points, while a comparison of reminder interventions revealed that emphasizing accountability to the provider encouraged patients to track their blood glucose online. An assessment of downstream outcomes revealed impacts of the interventions on prescribing behavior and A1c testing frequency. DISCUSSION: It is important to understand how health systems can practically promote take-up and awareness of emerging digital health alternatives or those with persistently low utilization in clinical settings.
CONCLUSION: These results indicate that provider training and support are critical first steps to promote utilization of patient-generated health data, and that patient communications can provide further motivation. Published by Oxford University Press on behalf of the American Medical Informatics Association 2022.

INTRODUCTION

The percentage of the US population with diagnosed diabetes increased from 4.4% to 7.8% from 2000 to 2018, with nearly $1 in $4 of health care dollars spent caring for people with diabetes in 2017. There is substantial evidence that improved average blood sugar control (as measured by A1c levels) is associated with significant decreases in the probability of complications from diabetes. Commercially insured patients with type II diabetes who lower their A1c, blood pressure and lipid levels experience significant reductions in total medical costs. Recent research also suggests that reduction in blood glucose variability is associated with reduced risk of complications and mortality independently of average blood glucose and A1c.,

For patients who are insulin-dependent, self-monitoring of blood glucose (SMBG) is a critical aspect of disease management and regulation of blood glucose levels and variability. For noninsulin-dependent patients with type 2 diabetes the literature is somewhat more mixed, although a recent, large meta-study showed positive effects. Two key factors in enhancing the effectiveness of SMBG seem to be patient adherence and physician involvement.

While the existing evidence on SMBG is promising, especially for insulin-dependent patients, nearly all randomized controlled trials of SMBG have had patients monitor blood glucose using either pen and paper or storing on a monitoring device, both needing to be physically brought to an office visit for physician viewing. However, the technology to electronically transmit blood glucose readings is available and ready to be analyzed. What’s more, recent trends in the electronic delivery of health services, particularly as a result of the coronavirus disease 2019 (COVID-19) pandemic, have made monitoring technologies increasingly accessible. In 2019, 43% of health centers were capable of providing telehealth, compared with 95% of the health centers that reported using telehealth during the COVID-19 pandemic, and this trend has extended to SMBGs in particular.

This article speaks to the particular context of utilizing electronic monitoring technologies within a large health system. An established literature suggests that providers face barriers in utilizing health technologies across a range of domains including diabetes monitoring, which in turn influences the degree to which patients are able to access the relevant health services. These barriers include the time burden associated with utilization, technological challenges, as well as hesitancy regarding the value of adoption. Many of these studies also show a correlation between adherence and reduction in A1c, suggesting SMBG is most effective when patients track regularly. This motivates looking at barriers to utilization from the perspective of the patient as well. For instance, the literature in behavioral science on habit formation around healthy behaviors, such as handwashing, sheds light on a crucial mechanism for continued adherence in this setting. Additional research shows that accountability to one’s physician—as engendered via doctor-patient communication—as well as financial incentives are also predictive of patient adherence., Finally, there is a vast literature on time discounting (valuing outcomes more in earlier compared to later time periods) indicating that patients may procrastinate engaging in healthy behaviors, particularly when there is some start-up cost such as understanding a new technology., This insight suggests that even simple reminders may serve to effectively increase long-run utilization.

Our study contributes to the relatively sparse literature regarding how best to address these barriers and promote effective utilization of health technologies., Indeed, we provide evidence from the first randomized trial to our knowledge that tests the causal impact of integrating data from patient blood glucose monitoring into electronic medical records (EMRs) on a wide scale. The first focus of this randomized trial is a practice-level intervention aimed at helping to overcome technological knowledge and awareness constraints around the use of electronic blood glucose tracking functionality among providers with patients who have diabetes. The second goal of this study is to estimate the effect of different reminder messaging approaches aimed at increasing patient use of electronic blood glucose flow sheets. The reminder messages that we test seek to address some of the barriers to utilization of SMBG discussed above. For instance, one reminder message variation tested in our study emphasizes provider accountability through provider engagement and monitoring of flow sheet entries. Another variation offers patients financial incentives through the chance to receive a gift card for every flow sheet entry completed. A third message serves purely to remind patients of the opportunity to track their blood glucose in an effort to increase the salience of SMBG and motivate more immediate action. A total of 4 reminder treatment arms are considered: (1) no reminder, (2) a generic reminder addressed from the medical group, (3) a provider accountability reminder addressed from the patient’s provider, and (4) a reminder addressed from the medical group with a chance to receive a gift card.

We combine and test these provider and patient-focused approaches to encouraging use of electronic blood glucose tracking, evaluating impacts on both utilization of the technology and downstream patient health and treatment outcomes.

MATERIALS AND METHODS

This study was conducted at selected primary care offices part of a multi-hospital system in Northern Virginia through a collaboration with the Office of Evaluation Sciences at the US General Services Administration. The health system’s EMR patient portal, MyChart, has a feature through which patients can track their blood glucose electronically, allowing providers to view entries in real time and be notified of out-of-range results. To streamline the tracking process, within the year prior to study initiation functionality was also integrated to connect Apple’s Healthkit to MyChart. However, for patients to use these flow sheets, it is necessary for their primary care provider to place an order through the EMR system. As part of business-as-usual, providers in all practices were notified via email of new features as they were rolled out in the MyChart electronic health record (EHR) system, but to our knowledge, there were no other regular programs in place to educate practitioners or encourage use. And indeed, at baseline, 0.1% of patients in the study sample had an open order allowing them to track their blood glucose using the flow sheets.

Datasets for this study were compiled monthly from EMR data collected as a normal part of care from January through October 2018. Additionally, a dataset listing active medications associated with a patient’s most recent encounter includes encounters going back to July 2017. A file matching provider IDs to practices, treatment assignment, and practice size strata used for random assignment of practices was also merged into the data described above.

At the time of the study, Inova Health System had 24 practice locations focused specifically on primary care. The total study sample consisted of 7052 patients of 68 providers at 20 selected primary care offices. Most providers practicing at these locations were physicians with MD/DO degrees, but a small number were nurse practitioners (NPs) (<10%). These NPs were included in the study sample. Adult patients of providers at these sites with a current diabetes mellitus diagnosis, no contraindications for tracking of blood glucose, and active MyChart account at time of treatment administration were included in the study. All communications and data collection took place through MyChart, the EMR patient portal. The study received IRB approval through the multi-hospital system (Protocol #17-2642).

This study involved a 2-stage experiment and test of interventions. First, primary care practices were randomly assigned to a treatment and control group. Providers in treated practices received additional tools and training to facilitate batch orders of blood glucose flow sheets. Second, patients at treated practices whose doctor placed flow sheet orders were assigned to 1 of 4 reminder messages to encourage utilization of the flow sheets. Sections “Provider assignment and intervention” and “Patient assignment and intervention” discuss this 2-stage assignment and the relevant interventions in greater detail.

Provider assignment and intervention

The provider-focused intervention was randomized at the practice level. Randomization stratified across practices by number of patients with diabetes (cluster size), dividing practices into 5 strata of 4 practices each. Two practices per strata (50%) were assigned to the treatment or control arm at the outset of the study; that is, practices were allocated in a 1:1 ratio to the treatment and control group. Table 1 reports the practice assignment and the number of providers assigned to each group.

Table 1.

Sample description

Practice assignment	N doc	N doc attend	N	L-name group	N	Reminder	N
Assigned practice treatment	34	23	3411	A–D	866	Gift card	554
				E–K	888	Provider accountability	573
				L–R	895	Basic	589
				S–Z	762	None	466
Assigned practice control	34	0	3641	A–D	926	NA	0
				E–K	882	NA	0
				L–R	1008	NA	0
				S–Z	825	NA	0

Note: This table describes the sample breakdown for the practice and reminder interventions.

Table 2 conducts balance tests on pre-treatment baseline covariates (with no controls) and reports coefficients on the difference across the assigned treatment and control practices. The patients in the treatment and control practices appeared to be well balanced across these measures.

Table 2.

Balance on pre-treatment covariates, treatment and control practices

Covariate	Control	Control	Diff	SE	P
	Mean	SD
Age	58.863	(14.008)	−.049	(1.616)	.976
Male	.538	(.499)	.013	(.024)	.603
Ethnicity, White non-Hispanic	.799	(.401)	.005	(.029)	.873
A1c, baseline	7.218	(1.587)	−.011	(.055)	.849
Days since last A1c test, baseline	188.607	(190.291)	7.225	(11.26)	.539
Days since last appointment, baseline	147.858	(187.32)	7.728	(8.176)	.372
Completed flow sheet, last 14 weeks at baseline	.001	(.033)	−.001	(.001)	.601
Number of patient messages, last 14 weeks at baseline	1.381	(2.819)	.117	(.144)	.438
Number of phone appts, last 14 weeks at baseline	.896	(1.922)	.009	(.157)	.956
Number of in-person appts, last 14 weeks at baseline	1.415	(3.068)	−.008	(.113)	.944
Medication list changed, last 14 weeks at baseline	.222	(.416)	−.015	(.018)	.417
Medication removed, last 14 weeks at baseline	.071	(.257)	−.009	(.008)	.281
Medication added, last 14 weeks at baseline	.217	(.412)	−.014	(.017)	.444
Number of RX orders, last 14 weeks at baseline	6.303	(20.279)	−.518	(.493)	.323
Number of new RX orders, last 14 weeks at baseline	6.298	(20.229)	−.518	(.493)	.323
Number of diabetes RX orders, last 14 weeks at baseline	.86	(2.558)	−.053	(.074)	.490

Note: This table shows the balance between control and treatment practices on pre-treatment characteristics. Control means and standard deviations are reported in the first 2 columns. Columns Diff and SE reflect coefficients and standard errors from a regression on pre-treatment covariates, which estimates the balance between treatment and control practices. P-values are in the far right column.

Primary care practices assigned to the control arm did not receive any intervention. Providers at primary care practices selected for the treatment group were given information, encouragement, and assistance to batch order blood glucose flow sheets for all patients with diabetes with active MyChart accounts. The research team contacted providers and practice managers with an explanation of the initiative and invited them to attend a virtual session within the first 2 weeks of the study to review instructions for completing batch orders and viewing entries through the system. Following the virtual session, providers had the option to request an in-person, hands-on walk through of the order process. In this case, a member of the research team facilitated the batch order and showed the provider how to monitor patient entries. Providers were given a template for a secure smart-text message to send to all patients receiving the flow sheets. This template is shown in Supplementary Appendix Exhibit A1.

Patient assignment and intervention

In the second stage of the experiment, patients at treatment practices whose doctors placed flow sheet orders were assigned individually to be sent 1 of 4 follow-up reminders aimed at encouraging the use of flow sheets: (1) “No Reminder”, in which patients received only an initial reminder notice and no subsequent reminders; (2) “Basic”, in which patients received an additional reminder of the availability and benefits of blood glucose monitoring; (3) “Gift Card”, in which patients received an additional reminder indicating that they would be entered into a lottery to receive 1 of 50 $50 gift cards to Amazon.com for each day they track their blood glucose on MyChart; and (4) “Provider Accountability”, in which patients received an additional reminder indicating that their provider would discuss their results with them at future office visits. These follow-up emails were sent after the initial 2-week administration and roll-out period. The full text of each reminder message can be found in Supplementary Appendix Exhibit A2.

Assignment to reminders was determined alphabetically by first letter of patient last name. Table 1 provides an overview of the sample and assignment structure. Though it would have been ideal to assign patients to reminder groups randomly, it was logistically infeasible to send individual-level patient messaging without sorting on an existing field in the patient’s EMR. There were some concerns that confounding factors such as ethnicity could correlate with assignment based on last name spelling. Indeed, we tested the balance of the pre-treatment covariates across reminder groups and found that the “Basic” reminder group has a slightly lower baseline A1c and was slightly more likely to be White than the “No Reminder” group. All baseline covariates are controlled for in the reported analysis of reminder messaging. Further falsification tests that support a causal interpretation of these results are available upon request.

Timeline

The trial consisted of a 14-week intervention phase with an additional 12-week follow-up phase. Reminders were sent out every 2 weeks for the 12 weeks following the initial order period. The total trial period was 26 weeks. Measurements were undertaken at 3 key time-points in each group: at baseline, directly after completing the 14-week intervention period, and at 6-month follow-up (an additional 12 weeks after the intervention period).

Outcomes

All outcomes considered were pre-specified via ClinicalTrials.gov prior to the launch of the interventions. We present what we consider the most important outcomes in the main text, and we present the full set of outcomes in Supplementary Appendix Exhibits A4 and A5. In particular, we examine the impact of the interventions on flow sheet use, measured in terms of both an indicator for whether patients in a practice had a flow sheet order (“Flow Sheet Orders,” for the practice intervention) as well as an indicator for whether a patient entered any flow sheet data during the measurement period (“Flowsheet Use, Extensive”). The primary health outcomes of interest are the most recent HbA1c test value in EMR records (“A1c”) and an indicator for whether the patient had an HbA1c test during the measurement period (“Had A1c during period”). We look at the total number of MyChart messages sent by the patient during the measurement period (“Number of Messages Sent by Patient”) as one proxy for patient-provider interaction. Finally, we consider 2 patient treatment outcomes, in particular an indicator for whether there was a change to patient active medications during the measurement period (“Any Medication List Change”) and the number of prescription orders for a patient during the measurement period (“Number of Prescription Orders”).

Empirical approach

We specified multivariate regression models to analyze the effects of both the practice-level and patient-level interventions. For the practice-level intervention, we ran a regression to compare patients assigned to providers in control practices (business as usual) to patients assigned to providers in treatment practices, i.e. those which received information, encouragement, and assistance to batch order blood glucose flow sheets. Our first specification includes an indicator for whether the patient belonged to a treatment practice and fixed effects for the 5 strata of 4 practices each. The treatment effect is the estimated coefficient on the indicator for belonging to a treatment practice. We report heteroskedasticity-robust standard errors which are clustered at the practice level to account for the clustered randomization design. In a second version of this specification, we incorporate patient-level controls and apply the Lin covariate adjustment. These controls include patient age (quadratic), sex (categorical), ethnicity (categorical), value of most recent baseline A1c test result (linear), days since most recent baseline A1c test result (linear), and days since most recent appointment at baseline (linear). Additionally, for outcomes referring to prescription medications, appointments, or secure messages, the 14-week baseline value of the outcome was included as an additional control variable.

For the patient-level intervention, we compared patients assigned to each reminder variation to the “No Reminder” group. The regression specification includes a series of indicators for whether the patient was assigned to each reminder group, as well as the list of controls from above. We report the coefficient on each reminder indicator relative to the no-reminder group. A version of this specification was estimated using the full sample of patients from treated practices, as well as one with a sample limited to patients who received flow sheet orders. We report heteroskedasticity-consistent standard errors and apply the Lin covariate adjustment. Supplementary Appendix Exhibit A3 provides further details on all regression specifications.

RESULTS

Practice intervention

Of the 34 providers from 10 practices assigned to the practice intervention arm, 23 attended a virtual orientation session and all but 5 of these requested an in-person session to facilitate the batch orders. Table 3 reports the impact of this practice intervention on flow sheet use, patient health, patient–provider interaction, and patient treatment.

Table 3

Impact of practice interventions on all outcomes

Outcome	Period	Control mean (SD)	TE, no covariates	TE, covariates
Flow sheet use
Flow sheet orders	Weeks 1–14	.001 (.037)	.634 (.138) [.002***] {.601}	.636 (.135) [.001***] {.612}
	Weeks 1–26	.002 (.041)	.634 (.138) [.002***] {.602}	.636 (.135) [.001***] {.612}
Flowsheet use (Extensive)	Weeks 1–14	.001 (.033)	.046 (.007) [0***] {.031}	.047 (.007) [0***] {.045}
	Weeks 15–26	.001 (.033)	.022 (.004) [.001***] {.016}	.023 (.004) [0***] {.025}
Patient health
A1c	At Week 14 (N = 6430)	7.196 (1.601)	−.002 (.052) [.968] {.002}	−.006 (.017) [.751] {.799}
	At Week 26 (N = 6236)	7.179 (1.54)	.043 (.051) [.422] {.002}	.035 (.025) [.202] {.711}
Had A1c test in period	At Week 14 (N = 6423)	.391 (.488)	.012 (.031) [.708] {.005}	.016 (.03) [.607] {.049}
	At Week 26 (N = 6240)	.626 (.484)	−.018 (.026) [.518] {.004}	−.01 (.026) [.704] {.109}
Patient–provider interaction
Number of messages Sent by patient	Weeks 1–14	1.385 (3.016)	.117 (.133) [.403] {.007}	.067 (.071) [.37] {.276}
	Weeks 15–26	1.128 (2.65)	.006 (.113) {.005}	−.022 (.063) {.182}
Patient treatment
Any medication list change	Weeks 1–14	.204 (.403)	−.042 (.018) [.051*] {.008}	−.036 (.015) [.043**] {.074}
	Weeks 1–26	.311 (.463)	−.055 (.024) [.051*] {.008}	−.048 (.02) [.043**] {.094}
Number of prescription orders	Weeks 1–14	5.111 (19.825)	−.289 (.502) [.581] {.003}	−.014 (.422) [.973] {.286}
	Weeks 1–26	9.296 (32.904)	−.708 (.904) [.456] {.004}	−.245 (.848) [.78] {.281}
N			7052	7052

Note: This table displays coefficients and standard errors for regressions (with and without covariates) which estimate the treatment effect of the practice intervention on patient flow sheet use, patient health, patient–provider interaction, and patient treatment. Lin covariate adjustment is used. Standard errors are in parentheses, P-values are in square brackets (*P < .10, **P < .05, ***P < .01), and R2 are in curly brackets.

In terms of flow sheet orders, patients at practices randomly selected for treatment were 63.6 (SE 13.5) percentage points more likely to receive an electronic flow sheet order, meaning that they had the option of tracking their blood glucose measurements through MyChart. They were 4.7 (SE 0.7) percentage points more likely to use the flow sheet at least once in the 14 weeks following implementation, and 2.3 (SE 0.4) percentage points more likely in the 15–26 weeks following implementation. The average patient using the flow sheets made a total of 66 entries over the 26-week study period.

Despite some increase in flow sheet use, results reported in the “Patient Health” panel in Table 3 indicate that assignment to the practice intervention had no effect on A1c. A clinically significant change in A1c of 0.3 is outside of the confidence interval for impacts at either 14 or 26 weeks, indicating a precisely estimated null effect for the reported intent-to-treat estimates. As can be seen in Supplementary Appendix Exhibit A4, alternate transformations of the A1c outcome, including indicator variables for A1c below 7, an ADA-recommended cutoff, and A1c improvement over baseline as well as the probability of receiving an A1c test, also showed no effect.,

Another outcome of interest is how electronic tracking impacts patient–provider interaction. Though it appears patients in the treatment group may have sent slightly more secure messages in the initial 14-week period, the increase of 0.067 (SE = 0.071) messages was not statistically significant, as shown in the “Patient-Provider Interaction” panel in Table 3. This alleviates some concerns about indirect impacts of monitoring technology on provider burden through increased patient communication and interaction. However, the messaging outcome reported does not include automated notifications generated as a result of the flow sheet orders.

The “Patient Treatment” panel in Table 3 reports the effect of the practice intervention on prescription orders and changes to active medications. We observe significantly lower rates of change to active medications, which is primarily driven by lower rates of addition. Patients in the treatment group were 4.8 percentage points less likely to see a change to their active medications within the 26 weeks following the start of the intervention. The effect of the practice intervention on medication list changes remains significant after correcting for multiple hypothesis corrections (shown in Supplementary Appendix Exhibit A6).

Patient intervention

Due to quasi-random assignment of patients to reminder treatment groups based on first letter of last name, causal interpretation of these results requires that grouped last name spelling is not independently related to these outcomes. In addition to controlling for pre-treatment patient characteristics and baseline outcome values, we also conduct a series of placebo tests by re-running the analysis on these same last name letter designations but with patients in the Control practices, who did not receive flow sheet orders or reminders. No significant differences between last name groupings were found for any of the outcomes (results available upon request).

Table 4 below shows how different reminder messaging impacts patient use of the flow sheets. Neither the “Gift Card” reminder nor the “Basic” reminder resulted in use rates significantly different from the “No Reminder” group. However, we saw a 3.1 (SE 1.6) percentage point (52%) higher take-up rate among patients receiving the “Provider Accountability” reminder relative to the “No Reminder” group. This effect persisted even after reminders were no longer being sent out, in the period 15–26 weeks after implementation. Patients in the “Provider Accountability” group had a 2.0 (SE 1.2) percentage point (63%) higher flow sheet use rate in the post-reminder period relative to the no-reminder group. Possible explanations for this persistence include long-term influence of this particular reminder on patients’ evaluation of the value of tracking, and habit-formation.

Table 4

Impact of patient interventions on all outcomes

	Full sample					Patients with FS orders
Outcome	Period	Control	TE,	TE,	TE,	Control	TE,	TE,	TE,
		Mean (SD)	Basic	GC	Phys	Mean (SD)	Basic	GC	Phys
				Flow sheet use
Flow sheet use(Extensive)	Weeks 1–14	.035 (.185)	.014 (.01) [.165] {.001}	.013 (.01) [.187] {.001}	.021 (.01) [.043**] {.001}	.058 (.234)	.016 (.015) [.289] {.030}	.016 (.015) [.294] {.030}	.031 (.016) [.052*] {.030}
	Weeks 15–26	.02 (.139)	0 (.007) [1] {.002}	.003 (.007) [.666] {.002}	.014 (.008) [.074*] {.002}	.032 (.177)	−.002 (.011) [.843] {.025}	.002 (.011) [.89] {.025}	.022 (.012) [.076*] {.025}
				Patient health
A1c	Week 14 (N = 3097)	7.276 (2.823)	.013 (.039) [.741] {.852}	.055 (.044) [.205] {.852}	.067 (.04) [.092*] {.852}	7.352 (3.418)	.03 (.057) [.602] {.857}	.081 (.063) [.197] {.857}	.089 (.057) [.121] {.857}
	Week 26 (N = 3004)	7.327 (2.894)	−.057 (.051) [.269] {.763}	.022 (.056) [.692] {.763}	.043 (.054) [.431] {.763}	7.39 (3.493)	−.052 (.067) [.438] {.796}	.087 (.074) [.24] {.796}	.06 (.071) [.402] {.796}
Had A1c test in period	Week 14 (N = 3099)	.434 (.496)	−.026 (.025) [.311] {.055}	−.008 (.025) [.758] {.055}	−.054 (.025) [.028**] {.055}	.464 (.499)	−.029 (.032) [.372] {.063}	−.012 (.032) [.719] {.063}	−.08 (.031) [.011**] {.063}
	Week 26 (N = 3008)	.618 (.486)	−.016 (.026) [.544] {0}	.009 (.026) [.742] {0}	−.017 (.026) [.517] {0}	.632 (.483)	.004 (.032) [.895] {.002}	−.001 (.032) [.973] {.002}	−.052 (.032) [.104] {.002}
Patient–provider interaction
No. messages sent by patient	Weeks 1–14	1.407 (2.872)	.268 (.131) [.041**] {.307}	.079 (.12) [.512] {.307}	.16 (.123) [.194] {.307}	1.715 (3.291)	.159 (.178) [.373] {.328}	−.074 (.166) [.657] {.328}	−.035 (.166) [.832] {.328}
	Weeks 15–26	1.226 (2.803)	−.044 (.122) [.722] {.210}	−.155 (.117) [.187] {.210}	−.092 (.116) [.425] {.210}	1.356 (3.109)	−.104 (.16) [.513] {.225}	−.223 (.154) [.147] {.225}	−.239 (.15) [.11] {.225}
Patient treatment
Any medication list change	Weeks 1–14	.177 (.382)	−.018 (.018) [.333] {.069}	−.02 (.018) [.267] {.069}	−.022 (.018) [.232] {.069}	.189 (.392)	−.04 (.023) [.082*] {.077}	−.035 (.023) [.122] {.077}	−.021 (.023) [.37] {.077}
	Weeks 1–26	.26 (.439)	−.024 (.021) [.254] {.096}	−.006 (.021) [.77] {.096}	.02 (.021) [.336] {.096}	.277 (.448)	−.04 (.026) [.125] {.106}	−.031 (.026) [.238] {.106}	.02 (.027) [.462] {.106}
Number of prescription orders	Weeks 1–14	5.912 (19.041)	−.727 (.7) [.299] {.248}	−.907 (.727) [.212] {.248}	−1.009 (.699) [.149] {.248}	5.83 (18.405)	−.079 (.877) [.928] {.248}	−.599 (.919) [.515] {.248}	−1.824 (.701) [.009***] {.248}
	Weeks 1–26	10.554 (33.799)	−1.211 (1.247) [.332] {.248}	−1.854 (1.278) [.147] {.248}	−1.65 (1.171) [.159] {.248}	9.987 (32.454)	−.152 (1.53) [.921] {.258}	−.683 (1.631) [.676] {.258}	−2.104 (1.33) [.114] {.258}

Note: This table displays coefficients and standard errors for regressions which estimate the treatment effect of different types of reminder messages received by patients in the treatment practices on flow sheet use and patient health outcomes, relative to no reminder. The left panel includes the full sample, while the right panel limits the sample to patients who received a flow sheet order. Lin covariate adjustment is used. Standard errors are in parentheses, P-values are in square brackets (*P < .10, **P < .05, ***P < .01), and R2 are in curly brackets.

Table 4 additionally presents results for how different reminder messages impact downstream outcomes including patient health, patient–provider interaction, and prescription medications. Because higher flow sheet use rates were observed for patients receiving the “Provider Accountability” reminder, we may expect to see differences in health and healthcare among this group. However, we find no discernible differences in patient A1c across the reminder groups. Though some alternate transformations as shown in Supplementary Appendix Exhibit A5 are marginally significant, these are not sustained through the 26-week outcome period. Notably, there does appear to be a statistically significant reduction in the probability of having a A1c test among the “Provider Accountability” reminder group (relative to the no-reminder group), particularly during the initial 14-week period. Patients in this group are nearly 8 percentage points less likely to get an A1c test during this time frame. One explanation is that the information entered in the flow sheets is seen by providers as a substitute for information gained from an A1c test. Note that any change in probability of A1c testing potentially confounds results evaluating A1c as an outcome.

Supplementary Appendix Exhibit A5 also shows no statistically significant differences in patient messaging or appointments among the reminder groups, with the exception of patient messaging at 14 weeks, for which we estimate a slightly higher rate in the “Basic” message group relative to control. However, this impact is no longer significant when considering only patients who received flow sheet orders, and is not sustained through Weeks 14–26.

Finally, we observe significant reductions in prescriptions for the “Provider Accountability” reminder group. Patients in this reminder group who received a flow sheet order saw 31% fewer prescription orders during the initial 14-week outcome period than patients assigned to the group that did not receive reminders. At least some of the change appears to be driven by a reduction in orders for diabetes-related prescriptions, which were 28% lower relative to the no-reminder group.

Importantly, while the un-adjusted P values show significant impacts of the Provider Accountability reminder on A1c test frequency and diabetes prescription orders in comparison to the no-reminder group, estimates adjusted for multiple hypothesis corrections are no longer statistically significant. This suggests caution in interpreting the effect of this reminder on specific healthcare use outcomes. However, the result of a summary index test shows a negative statistically significant impact of the reminder on healthcare use outcomes overall.

DISCUSSION

We find that a provider training and support intervention was successful at increasing utilization of the flow sheets, and that patient reminder messaging focused on accountability to the patient’s provider was most successful at increasing patient tracking. This increased use of flow sheets led to significant downstream reductions in medication changes for patients in treated practices as well as prescription orders for patients assigned to the reminder group associated with provider accountability. This result is somewhat surprising and seems to contradict the hypothesis that an increase in health data would result in updates to a patient’s treatment plan. One explanation is that providers see SMBG as a substitute for additional prescription medications, or want to hold off on making changes to medications while the patient gathers tracking data. This result may also represent a downstream outcome of increased flow sheet usage and resulting lifestyle management acting as a substitute for prescription medications. It is also possible that negative but statistically insignificant point estimates of the effect of the interventions on appointments could be driving this result.

We also documented slightly lower rates of A1c testing for patients in this reminder group, suggesting a possible substitute-ability of patient-generated health data for formal lab testing. Importantly, however, we did not observe changes in downstream patient health. It is important to note that imperfect compliance with provider ordering and limited first stage impacts on flow sheet use mean we cannot interpret this as a direct failure of electronic tracking to impact downstream health. Rather, roll-out of electronic tracking in a system-wide setting with voluntary provider education and patient participation fails to produce measurable A1c changes.

Technologies for patients to collect and share their health data in real time are increasingly available, particularly given the increased use of digital health alternatives in response to the COVID-19 pandemic. Continuous Glucose Monitor (CGM) technology, which is not integrated with the EHR, is a notable example that we see as both distinct from and complementary to the MyChart flowsheet functionality in this study. At the time of our study, compatible digital CGM devices existed but were not prevalent in our patient population. Of patients who completed any flowsheet, 8.9% made automated entries using Apple Health Kit, averaging 121 entries over the study period per patient (compared to 66 for the entire population). Existing literature indicates a relatively low perception of CGM burden among patients, and we expect that the proliferation of CGM will make it even easier and more common for patients to actively share data with their providers and EHR via the flowsheet technology used in this study or other emerging technologies. Our study broadly informs communication strategies for encouraging patients to use health data sharing applications, as well as possible implications for patient outcomes as utilization of these technologies continues to grow. There are a small number of other studies that describe the institutional logistics of integrating CGM data with an EHR, but we are not aware of any existing research that evaluates the impact of sharing CGM data with providers on patient outcomes.

Use of self-monitoring technologies in a clinical setting remains limited. Prior to this study, just 0.1% of patients in our sample were tracking blood glucose data through their EMRs. While many patients at treatment practices eventually stopped tracking, about one-third of those who tracked continued to do so well after reminders were no longer sent out. We interpret this as evidence that low utilization of remote monitoring technology is at least in some cases attributable to lack of awareness or the barriers to trying a new technology for the first time rather than low value assessment. Thus, even in light of the fact that we do not observe an impact of the increased use of flow sheets on patient health, our results can be applied more broadly to practically promote take-up and awareness of new technologies. In particular, an important contribution of our findings is that they point to provider training and support as well as emphasizing engagement and accountability to healthcare providers to increase utilization of effective health technologies. Future research could explore other low-cost strategies for enhancing patients’ intrinsic motivation, such as automated dynamic responses to entered data, as well as strategies for overcoming the myopia that poses barriers to first-time use and promoting habit formation to facilitate longer-term use.

Finally, one limitation of this study is that it focused only on patients who had already activated online patient portals. Studies show that patients who are non-English speaking, non-White, older, or lower socioeconomic status are less likely to be portal users., Future research could focus on how to encourage use of patient-generated data and remote monitoring as well as to develop alternative tools for these harder to reach populations.

CONCLUSION

This study evaluates multiple strategies for encouraging utilization of electronic blood glucose tracking and measures the downstream impact of a system-wide implementation of this tool. We find that a provider education intervention and a patient reminder emphasizing accountability to the provider led to an increase in patient tracking as well as changes in prescribing behavior. These findings point to the promise of strategies that can contribute to the effective integration of emerging digital health technologies into our health care systems.

FUNDING

This work was supported by the Abdul Latif Jameel Poverty Action Lab’s Health Care Delivery Initiative Pilot Fund.

AUTHOR CONTRIBUTIONS

All authors contributed substantially to this paper. AR conceived of the idea and was involved in implementing the randomized controlled trial as well as writing the manuscript. CC, SM, and HA were all involved in implementing the randomized controlled trial. MT was involved in writing the manuscript.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

Click here for additional data file.