Impact of a Mobile Health Application on User Engagement and Pregnancy Outcomes Among Wyoming Medicaid Members.

BACKGROUND: Pregnancy and birth outcomes are a critical area of healthcare, yet negative outcomes like C-sections and preterm births remain widespread. Studies show that early and ongoing prenatal care can improve outcomes; however, in-person care is difficult to deliver in rural areas. This article examines the impact of mobile health technology on user engagement and birth outcomes in a Wyoming pilot study. The pilot did face some limitations; namely, the small app user group size and scant demographic information collected from users.
MATERIALS AND METHODS: Wyoming Medicaid contracted with Xerox State Healthcare to launch WYhealth Due Date Plus, a pregnancy application by Wildflower Health. Pregnant Medicaid members registering for the app and providing a Medicaid ID were assigned to the app user group (N = 85). The non-app user group consisted of other pregnant Medicaid members with delivery outcome records (N = 5,158). Downloads and utilization frequency were tracked to gauge user engagement. Among pregnant Medicaid members, data were collected on app usage and four outcomes of interest-6-month or more prenatal visit, C-section, low birth weight, and Neonatal Intensive Care Unit (NICU) admission-to examine the association between app use and pregnancy/birth outcomes. Chi-square tests were conducted to analyze associations. A Kolmogorov-Smirnov test was used to assess potential confounding.
RESULTS: Strong user engagement was observed with over 1,730 downloads. App users had a statistically significant association between app usage and completion of a 6-month or more prenatal visit (p = 0.022). There was a borderline significant association between app use and decreased incidence of low birth weight (p = 0.055). Maternal age was not a possible confounder.
CONCLUSIONS: Preliminary data indicate that Due Date Plus attracted an engaged user base and that app usage was associated with improvements in prenatal visit completion and reduced incidence of low-birth weight delivery. These promising results suggest broader implementation and further study of mobile applications for prenatal support.

Introduction

Maternal health and pregnancy outcomes are among the most important areas of healthcare. About four million women give birth each year in the United States.[1] Complications during pregnancy and delivery can result in a number of negative impacts on families, the healthcare system, and society as a whole.

In the short term, complications often lead to increased costs, which are borne by consumers and public or private insurers. Both Medicaid and commercial payers paid more than 50% more for cesarean births (C-sections) than vaginal births in 2010.[2] A large-scale 2007 study found that hospital costs for preterm (<37 weeks)/low-birth weight (<2,500 g) deliveries represent 47% of all infant hospitalization costs.[3] Adjusting for inflation, the study found that while hospitalizations for uncomplicated newborns cost $800 on average (2015 dollars) per stay, hospitalizations for preterm/low-birth weight infant stays average $20,200.[3,4]

In addition to short-term costs for delivery and perinatal care, negative pregnancy outcomes such as preterm birth and low birth weight carry important long-term consequences. Prematurity is a major driver of pediatric morbidity and disability, and it is associated with nearly 50% of all childhood neurodevelopmental disorders.[3] In a 2006 report, the Institute of Medicine (IOM) estimated an annual societal cost of $26.2 billion ($31.8 billion in 2015 dollars) associated with preterm birth in the United States, of which 2% stemmed from early intervention services, 4% from special education services, and 22% from lost household and labor market productivity.[5]

In light of these wide-ranging negative consequences, the persistence of birth complications and preterm deliveries in the United States is highly problematic. In 2014, 32% of U.S. infants, including over 50% of preterm babies, were delivered through C-section. Nearly 10% of infants were born preterm, and 8% were born at a low birth weight in 2014; the latter number has remained stable since 2006, trending downward to less than 5%.[6] Moreover, significant racial disparities persist. For example, while the incidence of low birth weight infants is less than 7% among non-Hispanic whites, it exceeds 13% among non-Hispanic blacks.[6]

Research aimed at improving U.S. birth outcomes has found that one of the most important determinants of birth outcomes is access to prenatal care. Studies have found that even after adjusting for other differences like socioeconomic status and maternal age, infants born to mothers who receive no prenatal care weigh considerably less, on average, than those whose mothers receive prenatal care.[7] In contrast, early prenatal care in the first trimester (the first 90 days) has been shown to have a significant positive effect on birth outcomes; for instance, a sample of white teenagers showed a 27% reduction in low-birth weight births when provided with early prenatal care.[8]

To be effective, prenatal care programs must appropriately educate, monitor, and screen for behavioral risk factors and provide emotional support—intensive support when needed—for pregnant women. Effective programs can be difficult to implement anywhere, but the task is especially challenging in states like Wyoming, where population density is low and opportunities for in-person support are limited. Yet, it is in these states that supportive programs are most acutely needed. In 2014, 9.2% of Wyoming babies were born at a low birth weight—15% more than the national average; 11.2% were born preterm—17% more than the national average.[6] WY also ranks among the most unequal states in terms of racial/ethnic disparities in birth outcomes, as the rate of preterm births among Native Americans exceeds 14%.[9]

Technology offers innovative ways to overcome these challenges and deliver supportive prenatal care to at-risk women, even in remote areas. This article uses case study methodology to examine the effectiveness of one such technology-driven intervention: WYhealth Due Date Plus, a mobile application (app) rolled out to pregnant women on Medicaid in Wyoming. The case study evaluates the extent to which a mobile app can align patient behavior around prenatal care to best practices, thereby improving pregnancy outcomes.

Materials and Methods

Overview of the WYhealth Due Date Plus Intervention

In March 2013, Wyoming Medicaid and Xerox State Healthcare LLC, which provided the Total Population Health Management Program for Wyoming Medicaid clients, partnered with Wildflower, a mobile health company, to bring a smartphone application called WYhealth Due Date Plus to the state's Medicaid clients. The goal of the partnership was to use technology to identify women with pregnancy risk factors and refer them to evidence-based interventions, including case management, as well as behavioral health programs and state-based health resources. Wyoming Medicaid hoped to engage women as early in pregnancy as possible through the app and to keep them engaged throughout their pregnancy and postpartum.

The WYhealth Due Date Plus application includes information on over 70 health risk factors in pregnancy to identify at-risk clients needing more intensive care management. Clinical content is based on national guidelines from organizations such as American Congress of Obstetricians and Gynecologists (ACOG) and is reviewed yearly and updated when guidelines change. The app also offers women a pregnancy timeline tool, with information on key milestones, a weight tracker, and reminders for upcoming appointments. It further includes a number of Wyoming-specific resources, including access to click-to-call nurse support and contact information for a variety of related support programs. shows a screenshot of the application's home screen. displays the Wyoming Medicaid Benefits screen. shows the app's Community Resources Finder and presents an example of the app's educational content on smoking during pregnancy.

The WYhealth Due Date Plus application's main home screen.

The Wyoming Medicaid Benefits screen.

Look-up screen to find free community resources by zip code.

Educational content around smoking, a target risk factor.

Participants and Recruitment

Wyoming Medicaid began offering the WYhealth Due Date app in January 2014. Despite its focus on the Medicaid population, the app was made available to all pregnant women statewide, regardless of payer source, to facilitate effective recruitment. Women could opt to include their Medicaid ID during enrollment. While this broad approach limited the number of users for whom the authors were able to obtain Medicaid-based outcome data, it allowed the state to maximize access to the app across many outlets.

Wyoming Medicaid has promoted the app through various channels. Women were targeted through grassroots strategies using nurse and case management referrals. County health departments coordinated with Xerox to sponsor local Medicaid pregnancy events, known as “Baby Showers,” to introduce the app to expectant mothers. $25 Gift card incentives were given to women visiting all mandatory promotional booths at Baby Showers; women would be entered into a drawing for an additional $25 card if they downloaded the app. Maternity care providers were informed about the app and provided with referral materials for their patients. Additional channels have included promotion on relevant web sites, printed materials, and targeted e-mail and text messaging campaigns.

Outcomes and study measures

The goals of analyzing data on the WYhealth Due Date Plus app in Wyoming were twofold. First, the authors sought to understand whether the app could drive meaningful user engagement and be accepted by target users as a regularly utilized tool. The outcomes of interest were (1) application downloads and (2) frequency of screen views/feature use within the app after downloading.

Second, the authors sought to examine whether use of the app was associated with significant changes in prenatal care and birth outcomes among the Wyoming Medicaid population. Specifically, app use was analyzed in the context of four prenatal care and birth outcomes as follows: (1) prenatal visit at least 6 months before the delivery date; (2) C-section during delivery; (3) low birth weight, defined as <2,500 g; and (4) Neonatal Intensive Care Unit (NICU) admission following delivery.

Datasets

To examine user engagement with the WYhealth Due Date Plus app, Wildflower Health collected data on downloads of the app in Wyoming, among Medicaid, as well as non-Medicaid users, between January 2014 and December 2015. Along with download volumes, this dataset tracked the frequency of screen and feature utilization by user over the 2-year time period. Specific screens included the following: Wyoming Medicaid Benefits, providing an overview of Wyoming Medicaid with links to important resources and phone numbers; Health Milestone screens highlighting significant stages of gestation and the first 3 months postpartum; and What's Happening this Week screens with clinical content and ultrasound videos for each week of pregnancy. For app features, data were collected on usage of the Hot List, a user-defined list of clinical and behavioral topics that app users can share with their healthcare professional at their next appointment, and Baby Boost, a centering tool that helps expectant mothers connect with their babies.

To analyze the relationship between app use and prenatal/birth outcomes, the authors conducted a pilot study using a smaller Medicaid-only dataset. Specifically, pregnant Medicaid members who registered to use the app and also provided a Medicaid ID were assigned to the app user group (N = 85). The non-app user group consisted of other pregnant Medicaid members across Wyoming with delivery outcome records (N = 5,158). The only maternal demographic information collected was the mother's age during delivery.

Data on the four outcomes of interest—6-month prenatal visit, C-section, low birth weight, and NICU admission—were collected using Medicaid claim information rather than medical records. For the two postdelivery outcomes (low birth weight and NICU admission) claim-based birth outcome information was collected by matching the Medicaid ID of the mother to the Medicaid ID of the newborn.

Statistical analyses

Descriptive statistics was used to characterize the data on user engagement, with specific focus on app use over time and utilization of particular features/screens. For the pilot study on prenatal care and birth outcomes, a Chi-square test for independence was conducted to examine the association between app use and each of the four outcomes of interest. Given the small size of the app user group providing a Medicaid ID, all calculations included Yates' correction for continuity. For any associations found to be statistically significant, the authors calculated Cramer's V to determine the effect size. Then, to assess the role of maternal age as a possible confounder, two-sample Kolmogorov–Smirnov tests were used to examine the equality of distribution of age between groups of interest.

The statistical analyses were completed using the RStudio program (version 0.99.467).

Results

User engagement

Since the WYhealth Due Date Plus mobile application's launch in January 2014, the program has gained significant adoption among Medicaid providers and the pregnant population. During the reporting period, over 1,730 people downloaded the application. Seasonal differences in download volumes were observed, with increases in both the spring and autumn. On average, registered Medicaid users returned to the app 6.4 days per month and 41% continued to utilize the app from month to month in a rolling 6-month window. An overview of application downloads over time is shown in .

WYhealth Due Date Plus downloads between January 2014 and December 2015.

Women who downloaded the app varied in their frequency of engagement with it. Across app users, pregnancy and postpartum Health Milestone screens were utilized the most, as summarized in .

Table 1.

Summary of User Engagement by Application Screen/Feature

SCREEN/FEATURE NAME	PERCENTAGE OF MONTHLY ACTIVE USERS VIEWING THE SCREEN/FEATURE
Wyoming Medicaid Benefits screen	52
Health Milestone screens	62
What's Happening this Week screens	55
Hot List feature	24
Baby Boost feature	23

Prenatal and birth outcomes

In the pilot study, four outcomes of interest were tracked for the app user group and the non-app user group. summarizes the outcomes for each group.

Table 2.

Summary of Outcome Measurements Between App and Non-App User Groups

OUTCOME MEASURE	APP USER GROUP (N = 85), N (%)	NON-APP USER GROUP (N = 5,158), N (%)
Six-month or more prenatal visit	60 (70.6)	2,974 (57.7)
C-section	23 (27.1)	1,405 (27.7)
Low birth weight	2 (2.4)	458 (8.9)
NICU admission	4 (4.7)	289 (5.6)

NICU, neonatal intensive care unit.

Chi-square tests for independence were conducted to estimate the association between use of the WYhealth Due Date Plus application and each of the four outcomes of interest. reports the results, along with p-values and an odds ratio for each association. There was a statistically significant association between app use and completion of a prenatal visit at least 6 months before delivery (p = 0.022). Based on this association, a Medicaid member was estimated to be 76% more likely, on the odds scale, to have a 6-month prenatal visit if she was a user of the WYhealth Due Date Plus application. A borderline significant association was found between app use and low birth weight (p = 0.055). As Yates' correction can produce overly conservative results, a Pearson Chi-square statistic was also calculated for this association, yielding statistically significant results (χ2 = 4.45, p = 0.035). A Medicaid member was estimated to be only 25% as likely, on the odds scale, to have a low-birth weight baby if she was an app user (although the 95% confidence interval includes 1.0, as expected for a borderline significant association). No associations were found between use of the application and rates of C-section or NICU admission.

Table 3.

Chi-Square Tests for Independence Between App Use and Outcomes, by Outcome Measure

OUTCOME MEASURE	CHI-SQUARE[a]	P[a]	ODDS RATIO[b]
Six-month or more prenatal visit	5.22	0.022	1.76 (1.10, 2.82)
C-section	0.007	0.933	0.99 (0.61, 1.61)
Low birth weight	3.67	0.055	0.25 (0.06, 1.01)
NICU admission	0.014	0.906	0.83 (0.30, 2.29)

Reported Chi-square values and p-values reflect Yates' correction for continuity.

Odds ratios include a point estimate and a 95% confidence interval.

To assess effect size, Cramer's V was calculated for the associations between app use and 6-month prenatal visit and low birth weight, respectively. V = 0.033 for the association between app use and 6-month prenatal visit; V = 0.029 for the association between app use and low birth weight. With one degree of freedom, a V less than 0.3 is considered a small effect size, indicating that the observed association between app use and outcome, although statistically significant, was weak in both cases. This is, in part, attributable to the small sample sizes used in the pilot study.

In addition, the authors investigated the impact of a potential confounder: maternal age at delivery. As stated earlier, this was the only piece of demographic information collected in the pilot study. presents the results of a series of two-sample Kolmogorov–Smirnov tests for age distribution equality. Age distribution did vary significantly (p < 0.0001) between mothers who had a 6-month prenatal visit and those who did not, as well as between mothers who had a C-section delivery versus a vaginal delivery. Because a significant association was found between app use and the rates of 6-month prenatal visit completion, it was important to ascertain whether that association might be confounded by maternal age. However, the Kolmogorov–Smirnov test found that the distribution of maternal age did not differ significantly between app users and non-app users (p = 0.532). Because both groups in the study had a similar age distribution, confounding by age was not a concern.

Table 4.

Two-Sample Kolmogorov–Smirnov Test Results for Age Distribution Equality

AGE DISTRIBUTION COMPARISON	K-S STATISTIC	P
Six-month or more prenatal visit/no 6-month prenatal visit	0.069	< 0.0001
C-section/no C-section	0.114	< 0.0001
Low birth weight/no low birth weight	0.038	0.561
NICU admission/no NICU admission	0.058	0.306
App use/no app use	0.088	0.532

Discussion

Wyoming Medicaid launched the WYhealth Due Date Plus mobile application in January 2014 with the goal of using technology to engage and identify women with pregnancy risk factors early in their pregnancies and connect them to evidence-based interventions and supportive resources. The state saw the app as a potential tool to help reduce the incidence of negative prenatal and birth outcomes, which carry high financial and social costs in both the short and long term. For this case study, the key questions were: (1) Did the application drive meaningful user engagement? (2) Was use of the application associated with improved prenatal and birth outcomes among Wyoming's Medicaid population?

Data on app downloads between January 2014 and December 2015 showed that engagement with the technology was strong. User views of the Health Milestones and What's Happening this Week screens that were personalized to the mother's due date held steady over time. In addition, the consistent popularity of the Wyoming Medicaid Benefits screen by over half of users underscored its relevancy in connecting users to local resources. Nevertheless, some pitfalls were observed as well. The cyclical nature of user enrollment was a challenge and it was difficult to sustain interest in the app without continued marketing campaigns. The cost to the state during the pilot's reporting period was an average of $55K per year. Commercial and marketing best practices for driving uptake in a population similar to Wyoming would require at least $100–150K per year to ensure successful adoption.

Preliminary data from a pilot study of Medicaid participants suggest that use of WYhealth Due Date Plus was associated with a small but statistically significant increase in the completion of prenatal visits at least 6 months before delivery—an important milestone in ensuring a healthy pregnancy and delivery. Analysis also found a borderline significant association between use of the app and lowered incidence of low-birth weight delivery. However, no associations were found between app use and the rates of C-section or NICU admission.

The pilot study had a number of limitations. The sample size of the Medicaid app user group was small, limiting the robustness of the statistical analyses. Confounding remained a significant concern. Because maternal age was the only piece of demographic information collected on users, the results may have been biased by a number of untracked systematic differences between the user and nonuser groups. These include the following: previous pregnancies and outcomes; drug, alcohol, or tobacco use; participation in other resources or support programs; and marital status. There was also potential self-selection bias stemming from Medicaid participants' choice to download the WYhealth Due Date Plus app and register with their Medicaid ID. Furthermore, the pool of app nonusers may have inadvertently included Medicaid members who were using the app, but did not provide their Medicaid ID during registration. To the extent this “mixing” of the pools was present, the reported results underestimated the association between app use and outcomes.

The collection of outcome data was also subject to limitations. Because outcomes were based on ICD-9, ICD-10, and Current Procedural Terminology (CPT) code claim data rather than direct observation, they may not have been accurate, or sufficiently detailed, in all cases. As one example, the claim data did not offer any way to determine whether a C-section was elective or medically necessary. The integration of app use and claim data presented challenges. There was, for instance, no way to determine whether the 6-month prenatal visits occurred before or after installation of the application. Finally, the Chi-square analyses conducted in this study point only to association, not causality.

Despite these limitations, the preliminary results reported in this study help establish an emerging evidence base for the use of technology to support healthy pregnancies and ensure positive birth outcomes. Maternal and perinatal outcomes remain a key target for improvement in the United States. Research on best practices has demonstrated the importance of healthcare consumer engagement and supportive resources in preventing poor outcomes. However, in-person engagement is especially difficult in rural/frontier states like Wyoming. Spanning nearly 98,000 square miles, WY has a population of only 586,000, 200,000 of whom live in shortage areas with inadequate access to primary care.[10] In addition, the whole state is designated as a shortage area for mental healthcare. Given the high proportion of low-birth weight and preterm births in Wyoming, it is critical to find novel ways to engage pregnant women in this and similar states.

Technology and mobile apps, in particular, give providers and government agencies a new tool for driving engagement, especially in remote areas. Supportive pregnancy apps are an emerging field, and little information is available on their effectiveness in attracting users and impacting birth outcomes. For this reason, the early evidence on the effectiveness of WYhealth Due Date Plus reported in this study is significant. Strong user engagement numbers and promising results from the Medicaid pilot study suggest that broader implementation of the application and further study of its effectiveness are warranted. Wyoming Medicaid continues to enroll women in the app and plans to update the sample size of the Medicaid app user group and refresh this analysis down the road.