Utilization and effects of mobile electronic clinical decision support on pediatric asthma care quality in the emergency department and inpatient setting.

OBJECTIVE: To determine utilization and impacts of a mobile electronic clinical decision support (mECDS) on pediatric asthma care quality in emergency department and inpatient settings.
METHODS: We conducted an observational study of a mECDS tool that was deployed as part of a multi-dimensional, national quality improvement (QI) project focused on pediatric asthma. We quantified mECDS utilization using cumulative screen views over the study period in the city in which each participating site was located. We determined associations between mECDS utilization and pediatric asthma quality metrics using mixed-effect logistic regression models (adjusted for time, site characteristics, site-level QI project engagement, and patient characteristics).
RESULTS: The tool was offered to clinicians at 75 sites and used on 286 devices; cumulative screen views were 4191. Children's hospitals and sites with greater QI project engagement had higher cumulative mECDS utilization. Cumulative mECDS utilization was associated with significantly reduced odds of hospital admission (OR: 0.95, 95% CI: 0.92-0.98) and higher odds of caregiver referral to smoking cessation resources (OR: 1.08, 95% CI: 1.01-1.16). DISCUSSION: We linked mECDS utilization to clinical outcomes using a national sample and controlling for important confounders (secular trends, patient case mix, and concomitant QI efforts). We found mECDS utilization was associated with improvements in multiple measures of pediatric asthma care quality.
CONCLUSION: mECDS has the potential to overcome barriers to dissemination and improve care on a broad scale. Important areas of future work include improving mECDS uptake/utilization, linking clinicians' mECDS usage to clinical practice, and studying mECDS's impacts on other common pediatric conditions.

LAY SUMMARY

Childhood asthma is a leading cause of emergency visits, hospitalizations, missed school days, and missed work days for caregivers. Our team developed and launched a mobile decision support tool (app) as part of a national quality improvement (QI) project focused on pediatric asthma care. The application provided evidence-based decision support for management of asthma exacerbations for children in the emergency department and inpatient setting. Children’s hospitals and sites with greater overall QI project engagement were more likely to use the application. Cumulative application use was associated with improvements in pediatric asthma care, including reduced odds of hospital admission (OR: 0.95, 95% CI: 0.92–0.98) and higher odds of caregiver referral to resources for quitting smoking (OR: 1.08, 95% CI: 1.01–1.16). To our knowledge, our study is the first to link app use to clinical outcomes using a national sample and controlling for important potential confounders (time trends, patient characteristics, QI project engagement). Decision support in app form has the potential to overcome barriers to dissemination and improve care quality on a broad scale. Important areas of future work include improving app utilization, linking clinicians’ app usage to clinical practice, and studying app’s impacts on other common pediatric conditions.

BACKGROUND AND SIGNIFICANCE

New medical evidence takes an average of 17 years to enter into widespread clinical practice. Although healthcare institutions try to expedite the dissemination and implementation of evidence-based practices through the production of clinical decision support (CDS) tools, CDS development is resource-intensive, with limited portability across institutions. Even simpler decision support available in most electronic health record (EHR) systems, such as order sets, faces many barriers to and delays to implementation. Mobile electronic clinical decision support (mECDS) tools, when deployed through a freely downloadable app, have the potential to be an effective and scalable resource for improving quality of care and health outcomes. These tools also have been shown to integrate well into clinical workflow and reduce provider cognitive demand, improve medication dosing accuracy, aid with symptom recognition, and increase diagnostic and triage accuracy.

While mECDS tools have the potential to broadly and efficiently improve care quality, studies to date have left important knowledge gaps. Most studies to date assessing the impacts of mECDS on clinical practice have been conducted in simulated settings. A few studies that have assessed mECDS impacts on real-world practice consisted of simple pre-post analyses, which are subject to confounding biases from overall trends in healthcare delivery and changes in patient severity/case-mix over time. Prior randomized-controlled trials of mECDS have not quantified the cumulative effects of mECDS utilization on clinician practice or explored the effects of mECDS at the hospital/facility level. In a recent observational study, members of our team leveraged aggregate mECDS utilization data to determine the effects of mECDS on site-level care quality for infants with fever in the American Academy of Pediatrics (AAP) Value in Inpatient Pediatrics (VIP) network quality improvement (QI) project, REVISE. However, this study did not account for overall site project engagement (eg, other QI activities that could impact care quality), and the study only examined impact on practice in the emergency department (ED) setting.

Childhood asthma is a leading cause of emergency visits, hospitalizations, missed school days, and missed work days for caregivers, with total estimated direct costs of approximately $6 billion annually in USA. In 2018, VIP launched a new national QI project entitled PIPA, Pathways for Improving Pediatric Asthma Care. The project supported clinical pathway implementation with the global aim of “improving the value of care delivered to children with asthma.”, PIPA included a diverse sample of EDs and inpatient wards across the country.

OBJECTIVE

Our team developed and launched a mECDS tool as part of PIPA, a multi-dimensional QI project. The mECDS tool provided evidence-based decision support for both inpatient and ED management of asthma exacerbations in children. The PIPA project collected data on site-level project engagement (other QI/implementation activities) and pediatric asthma care quality monthly. Our team used these data to achieve our objective—to determine the impact of mECDS use on pediatric asthma care quality in the ED and inpatient setting.

MATERIALS AND METHODS

Study design and setting

We conducted a longitudinal, observational study using data from the PIPA project. Recruitment of PIPA sites occurred via 3 e-mails to VIP electronic mailing lists (listservs). These listservs include clinicians from over 250 EDs and hospitals in the USA that range widely in size, type (eg, free-standing children’s hospitals, community hospitals), ownership model (eg, private, non-profit), and location (eg, rural, urban). To adequately support the QI project, VIP had PIPA sites to initiate the QI project in 2 waves, with half starting improvement activities in January 2018 and half starting in April 2018 (completing in December 2018 and March 2019, respectively). Our mECDS tool was released to PIPA sites in late August 2018. Core elements of this multi-dimensional QI project were designed using existing QI and implementation frameworks (the Institute for Healthcare Improvement’s Model for Improvement and the Consolidated Framework for Implementation Research)., Participating EDs and hospitals were provided a pediatric asthma pathway implementation toolkit, which included sample evidence-based pathways and sample order sets based on pathway content. Each site designated a local physician implementation leader. These leaders recruited and then worked with local multidisciplinary teams to tailor and implement the pathways to fit local needs and context. Sites were provided several additional resources for implementation support: external practice facilitators, QI training, monthly audit and feedback, and educational seminars (eg, evidence-based asthma care).

Development and function of the mECDS tool

The PIPA mECDS tool (Figure 1A) was developed using the human factors methods including heuristic analysis and iterative usability testing. The tool consisted of ED and inpatient pediatric asthma pathways (Figure 1E) that calculated a patient’s severity score based on clinical parameters (eg, respiratory rate, wheezing, breath sounds, etc.) specified by the user at the time of assessment (Figure 1B). The pathways then provided evidence-based management recommendations based on the calculated severity score. The ED pathway provided criteria for ordering chest radiography and reminders to promptly administer steroids as indicated (Figure 1C). The inpatient pathway included guidance on MDI dosing and administration as well as reminders and tools for screening for secondhand tobacco exposure (Figure 1D). The mECDS tool also provided a selection of other tools that reinforced clinician adherence to the selected quality measures (Figure 1F) including links to smoking cessation resources (Figure 1F) and MDI administration tutorials (Figure 1G).

Figure 1.

PIPA mECDS pathways and other tools. (A) mECDS within the overall app PedsGuide; (B) severity score calculator; (C) example ED pathway end screen; (D) example inpatient pathway end screen; (E) pathway selection screen; (F) other resource selection screen; (G) smoking cessation resource; (H) MDI administration tutorial.

Deployment of the mECDS tool within PIPA

The tool was released in August 2018 as an available update to the pre-existing PedsGuide app. The PedsGuide app was released in 2015 for use in a prior VIP project. The app is free to download from both the iOS and Android app stores and requires no registration to begin using. The free mECDS tool release was announced to PIPA sites both via videoconference and email during project launch. However, there was then a 6 month delay before the tool was released leaving uptake of the tool to be largely driven by passive deployment methods including word of mouth and users having already downloaded the app for the prior VIP project.

PIPA site-reported data collection

Participating site characteristics such as hospital size and location (rural vs urban) were collected at project initiation via electronic survey. Site project engagement (QI activities) was assessed monthly via electronic survey of each site’s physician implementation leader. Surveys collected data on QI/implementation activities, specifically the state of implementation of key clinical pathway elements (eg, criteria for ordering chest radiography). Responses were converted into binary indicator variables that indicated whether each pathway element was implemented and in-use during the respective intervention month. Clinical practice data, including patient characteristics, adherence to performance metrics, and balancing metrics, were collected via chart review. A trained clinician from each site entered the chart review data on each ED visit/hospital admission into a secure, web-based electronic database (REDCap) maintained by the University of California, San Francisco. Sites collected chart review data on ED visits/admissions that occurred from January 2017 to March 2019. Most fields used in this analysis were required for chart submission by sites. Less than 2% of charts had missing data for the few non-required fields, and these were excluded from the analysis.

mECDS tool utilization data collection

mECDS tool utilization data were collected from release (August 23, 2018) to the end of the study period (March 31, 2019) using Google Analytics. Google Analytics automatically records mECDS utilization in terms of distinct devices on which the overall app has been opened (users), number of times the tool has been used (sessions), and what pages were viewed/buttons were clicked within the tool (events). Time stamps of the hour and geolocation of each session by city are also recorded. City-level usage data was linked by study site location for comparisons across sites. There were no cities with more than one site. For this analysis, we analyzed users, sessions, and events related to the newly-developed asthma mECDS tool within PedsGuide. Events were dichotomized according to whether they led the user to view quality metric-related content (MetricHits).

Primary predictor

In the prior REVISE study, we found cumulative mECDS utilization was associated with care quality for infants with fever. Cumulative utilization may reflect accumulated knowledge gained by use of the tool over time; thus, this cumulative utilization was used as the primary predictor in this study (cumulative metric hits). We determined “cumulative metric hits” by summing metric-related screen views by all asthma mECDS tool users in each city to date, from the city’s first month of usage data through the month immediately preceding the index month. For example, the value for October was computed by summing the city’s metric hits from mECDS release through September. The measure was set to zero for the first release month (August 2018).

Outcomes

Study performance and balancing metrics were selected through a consensus process among the national expert panel assembled for this study by the AAP. Performance metrics in the ED setting included decreasing the utilization of chest radiography (chest X-ray); increasing the use of severity assessment at ED triage (triage assessment); and decreasing the time from ED arrival to systemic corticosteroids administration (time to steroids). The balancing metric for the ED was not increasing ED length of stay. In the inpatient setting, the performance metrics were decreasing length of hospital stay; increasing early administration of metered dose inhalers (MDI, early defined as MDI at admission or first ordered at 1–2 h frequency); increasing documented screening for secondhand tobacco smoke exposure (Smoke Screening); and, when positive, increasing referral of caregivers to smoking cessation resources (Smoke Referral). The balancing metric was not increasing hospital readmission within 7 days of discharge (7-day readmit).

Statistical analyses

We analyzed the relationship between patient characteristics and cumulative metric hits in the city-month in which they were seen using an ANOVA test for patient age and chi-squared tests for all others. The relationship between site characteristics and cumulative metric hits in each site’s city in the final intervention month was analyzed using Fischer’s exact tests. Crude case adherence to each metric was tested using chi-squared tests for binary metrics and Mann–Whitney U-tests for continuous metrics.

To determine associations between cumulative metric hits and quality metrics, we used generalized mixed effect logistic regression models (1 per outcome/quality metric). The cutoff to derive odds ratios of case adherence by city-month cumulative metric hits was determined empirically by using the upper quartile of city-site use (5+ cumulative metric hits). A binomial distribution was used for binary metrics and a Gaussian distribution was used for continuous metrics. A log-link was used to compute ORs for continuous metrics (eg, odds of longer/prolonged LOS between kids seen in cities and months with 5+ cumulative metric hits vs <5 cumulative metric hits). Given clustering of encounters within sites, a random site intercept was included in each model. To account for potential confounders including secular trends/time, patient case-mix, and overall QI engagement, we included the following explanatory variables: cumulative metric hits, study month, site characteristic variables (eg, location, teaching status), site project engagement variables (eg, implementation of QI interventions), and patient characteristics (insurance type, sex, age, and prior use of inhaled corticosteroids). Metric Models: CaseAdherence = CumulativeMetricHits + Month + SiteCharacteristics + SiteProjectEngagement + PatientCharacteristics + RandomSiteIntercept.

All analyses were conducted using R v. 3.4.1 (Vienna, Austria), and p-values < .05 were considered significant.

RESULTS

Overall mECDS utilization

A total of 89 sites were recruited for the PIPA study and 75 sites completed the study. Reasons for not completing included lack of support from hospital leadership/administrators, difficulty obtaining local institutional review board (IRB) approval, difficulty obtaining chart review data, staff turnover, difficulty due to competing QI projects and very low patient volumes.

In total, the tool was used on 286 devices, 335 times, incurring 4191 events, of which 922 (22%) were page views of quality metric-related content (MetricHits). Usage trends from release until the end of the intervention are depicted in Figure 2. Users of the tool consistently engaged with it about once per month on average and had about 1 metric hit per session. Overall, the inpatient pathway and ED pathway were used roughly the same number of times (205 vs 190). The metric-related content most often viewed, particularly in early months, was secondhand tobacco smoke screening tools, followed by chest x-ray criteria. Guidance on ED management/timely steroid administration were second most viewed overall and surpassed smoke screening in some of the later months. Guidance on metered-dose inhaler dosing was consistently viewed least often. Site locations and city-level cumulative metric hits are depicted in Figure 3.

Figure 2.

Monthly tool utilization during the PIPA project. (A) Unique users (number of devices on which the tool was used), sessions (number of times the tool was used), and MetricHits (views of metric-related content) in each month; (B) number of sessions involving the ED pathway, inpatient pathway, and other tools in each month; (C) number of times content related to each ED metric was viewed in each month; (D) number of times content related to each inpatient metric was viewed in each month.

Figure 3.

Map of PIPA sites by cumulative metric hits in the intervention period.

Patient-level predictors of mECDS utilization and quality metric adherence

Patient characteristics, care setting, and crude adherence to metric are depicted in Table 1 by the level of cumulative use in the city and month in which their encounter occurred. Higher levels of cumulative metric hits were associated with patient-level prior prescription of an inhaled corticosteroid, having a payor type of “other,” and being seen in the inpatient setting.

Table 1.

Patient characteristics and metric adherence by cumulative use in the city and month of encounter.

				Cumulative metric hits
		Total		0		1–4		5+
		n	%	n	%	n	%	n	%	P-value
Total patients		34 121		29 484	86%	2216	6%	2421	7%
Age (years)a	Mean (SD)	7	4	7	4	7	4	7	4%	<.001
Sexb	n (%)									.585
	Male	20 577	60%	17 804	60%	1337	60%	1436	59%
	Female	13 544	40%	11 680	40%	879	40%	985	41%
Insurance	n (%)									<.001
typeb	Public	13 039	38%	11 242	38%	897	40%	900	37%
	Private	5512	16%	4716	16%	432	19%	364	15%
	Tricare	191	1%	168	1%	14	1%	9	0%
	Other	1857	5%	1583	5%	85	4%	189	8%
Prior prescription of inhaled corticosteroidb	n (%)									<.001
	Yes	15 724	46%	13 409	45%	1042	47%	1273	53%
	No	18 397	54%	16 075	55%	1174	53%	1148	47%
Settingb
ED	n (%)	22 109	65%	19 198	65%	1440	65%	1471	61%	<.001
Case adherence
Chest X-rayb	n (%)	6138	28%	5410	28%	433	30%	295	20%	<.001
Triage assessmentb	n (%)	19 866	90%	17 183	90%	1395	97%	1288	88%	<.001
Time to steroids (min)c	Median (IQR)	49	(30–81)	49	(30–82)	50	(32–81)	43	(28–69)	<.001
Admissionb	n (%)	4219	19%	3598	19%	359	25%	262	18%	<.001
ED LOS (min)c	Median (IQR)	148	(102–208)	149	(102–209)	144	(103–199)	147	(103–207)	.343
Inpatient	n (%)	12 012	35%	10 286	35%	776	35%	950	39%
Case adherence
Inpatient LOS (h)c	Median (IQR)	29	(20–42)	29	(20–42)	31	(21–44)	28	(18–41)	<.001
MDIb	n (%)	6295	52%	5161	50%	393	51%	741	78%	<.001
Smoke screeningb	n (%)	9755	81%	8328	81%	659	85%	768	81%	.024
Smoke referralb	n (%)	1323	11%	1075	10%	113	15%	135	14%	<.001
7-day readmitb	n (%)	276	2%	232	2%	12	2%	32	3%	.032

ANOVA test.

Chi-squared test.

Mann–Whitney U-test.

(min)

(min)

(h)

Site-level predictors of mECDS utilization

Most cities with a study site (61%) had at least some use, but cities with free-standing children’s hospitals had higher levels of use than those with community hospitals or non-freestanding children’s hospitals (Table 2). Sites with higher QI project engagement had significantly higher mECDS utilization, specifically those that implemented the pathway elements MDI dosing guidance, bronchodilator protocol, and discharge criteria.

Table 2.

Hospital/ED characteristics by cumulative metric hits in the final intervention month.

	Total sites		Cumulative metric hits
			0		1–4		5+
	n	%	n	%	n	%	n	%	P-valuea
Site-level factors	75		29	39	24	32	22	29
Hospital location									0.08
Urban	33	44	10	34	9	38	14	64
Suburban	37	49	17	59	14	58	6	27
Rural	5	7	2	7	1	4	2	9%
Hospital type									0.07
Community	40	53	21	72	12	50	7	32
Non-freestanding children’s	23	31	6	21	9	38	8	36
Free-standing children's	12	16	2	7	3	13	7	32
Hospital teaching status									0.32
Yes	68	91	27	93	23	96	18	82
No	7	9	2	7	1	4	4	18
Hospital bed size									0.92
Large (≥250 beds)	46	61	19	66	13	54	14	64
Medium (100–249 beds)	21	28	7	24	8	33	6	27
Small (<100 beds)	7	9	2	7	3	13	2	9
QI project engagement (pathway elements implemented): ED
CXR criteria	36	48	13	45	11	46	12	55	0.92
Severity scoring tool	51	68	16	55	16	67	19	86	0.16
Order set for corticosteroids	27	36	8	28	9	38	10	45	0.54
QI project engagement (pathway elements implemented): inpatient
MDI dosing guidance	60	80	17	59	22	92	21	95	0.01
Bronchodilator protocol	51	68	14	48	19	79	18	82	0.04
Discharge criteria	57	76	16	55	21	88	20	91	0.01
Tobacco screening reminder	63	84	20	69	23	96	20	91	0.06
Cessation tool referral reminder	62	83	20	69	23	96	19	86	0.06

Fisher’s exact test.

Associations between mECDS utilization and care quality

City-level cumulative metric hits were associated with one ED quality metric (Figure 4). Children seen in a city and month with 5 additional cumulative metric hits were 5% less likely to be admitted to the hospital (OR: 0.95, 95% CI: 0.92–0.98). City-level cumulative metric hits were associated with 2 inpatient quality metrics (Figure 5). Children seen in a city and month with 5 additional cumulative metric hits had a reduction in odds of prolonged hospital length of stay (OR: 0.99, 95% CI: 0.98–0.99) and were also more likely to have a caregiver referred to smoking cessation resources (OR: 1.08, 95% CI: 1.01–1.16).

Figure 4.

Effects of 5 additional cumulative metric hits on ED quality metrics. Odds of case adherence to each ED metric in a given month and city with 5+ cumulative metric hits versus <5 cumulative metric hits (adjusted for site characteristics, site engagement, patient case mix, study month, and clustering by site).

Figure 5.

Effects of 5 additional cumulative metric hits on inpatient quality metrics. Odds of case adherence to each inpatient metric in a given month and city with 5+ cumulative metric hits versus <5 cumulative metric hits (adjusted for site characteristics, site engagement, patient case mix, study month, and clustering by site).

DISCUSSION

To our knowledge, this is the first study of the cumulative effects of a mECDS tool on care quality that uses a large, national sample and robust methods to address potential confounding biases (secular trends, case-mix differences, concomitant QI efforts). We found that the mECDS tool was used in most cities with a project site and that cumulative mECDS utilization was associated with improvements in the quality of asthma care for children, including reduced odds of hospital admission, reduced inpatient length of stay, and higher odds of referral of caretakers to smoking cessation resources. These findings align with those of our prior study of a mECDS tool for management of infants with fever, in which the tool was also associated with improvements in 3 quality metrics. Our findings also reinforce existing evidence from randomized controlled trials that mECDS tools can have positive impacts on clinicians’ guideline adherence, and build upon this prior work by providing real-world data on care quality for a high-prevalence condition among children, asthma.

We found that cumulative utilization of the mECDS tool was associated with improvements in 3 quality measures: hospital admission from the ED, inpatient length of stay, and referral of caretakers to smoking cessation resources. Guidelines recommend timely administration of bronchodilators and systemic corticosteroids for children with asthma exacerbations because timely administration decreases time to recovery and risk of hospital admission. Although we did not detect statistically significant changes in timely systemic corticosteroid administration, the mECDS tool may have supported other aspects of more timely care, such as severity assessment or administration of bronchodilators, thus driving our finding of lower hospital admission risk. It is also possible that use of the tool increased clinician recognition of moderate to severe patients speeding up the triage process and decreasing admission risk. Bronchodilator weaning protocols/pathways and standardized discharge criteria can also decrease hospital length of stay. Clinician use of these resources within the tool may have contributed to our findings of decreased length of inpatient stay. Lastly, we included a section within the mECDS tool that could be shared in real time with caretakers to provide smoking cessation resources, and this resource was highly utilized compared to others within the tool. This resource may have helped drive our findings of increased smoking cessation resource referral.

The overall effects of the mECDS tool were small compared to prior studies of QI interventions, including the REVISE study that examined associations between cumulative mECDS utilization and care quality for febrile infants. Recent pediatric asthma studies have shown larger magnitude improvements in rates of hospital admission from the ED (OR 0.53 [95% CI: 0.37–0.76] shown by Bekmezian et al, OR 0.63 [95% CI: 0.40–0.99] shown by Walls et. al) length of inpatient stay (decreases of approximately 8–9%,) and guideline adherence measures, such as referral of caretakers to smoking cessation resources. These findings underscore the importance of our analysis accounting for site project engagement/other QI and implementation activities. However, such QI activities, particularly those that involve implementation within the EHR, are labor intensive and not as easily disseminated across institutions as mECDS tools. Thus, mECDS tool may still play an important role in QI when adequate resources for QI interventions are not available, as well as a supplemental tool for increasing the effects of QI interventions.

Although utilization was similar overall for the ED and inpatient pathways, the volume of patients seen in the ED was much higher (∼22 000 in ED vs ∼12 000 in the inpatient setting), perhaps indicating that the inpatient pathway had more uptake. This finding aligns with the greater impact of the tool on inpatient metrics. Free standing children’s hospitals are often repeat participants of VIP’s QI projects; thus, clinicians at these sites may have already had had the app the tool was deployed within. Sites also had higher overall project engagement/implementation of key pathway elements in the inpatient versus ED setting (68–84% vs 36–68%). Previous studies have shown that such project engagement is often unmeasured or only available via project leader self-report. The association found between self-reported project engagement/implementation and mECDS utilization indicates that mECDS utilization may be a good proxy for project engagement. This finding also supports the validity of self-reported project engagement.

This study had several limitations. First, despite harnessing using human factors methods (heuristic analysis, iterative usability testing) to develop the mECDS tool and widely disseminating it, mECDS tool uptake and usage was low. This may have been due to delays in launching the tool (about 6 months after the launch of the QI intervention), concomitant delivery of multiple QI interventions, or lack of explicit application of behavior change theory to mECDS use (though theory/frameworks were used in design of the overall QI intervention)., Our team did also conduct a mixed-methods study to better understand barriers to utilization of the QI interventions offered, including the mECDS tool. We found a potential barrier was lack of awareness, as many QI interventions were introduced within a relatively short time frame. When asked why they did not use the mECDS tool, participants reported they did not know it was available. The delayed launch also meant we were only able to track data for 6 months after launching the mECDS, so we did not evaluate sustainability of mECDS utilization. However, the eventual rollout of the mECDS tool reached a large, diverse sample of both EDs and inpatient wards from all regions of the country and strengthens the generalizability of our results.

Second, the delay led to mECDS tool use to only in the Fall and early Winter. Seasonal differences in availability could have contributed to differences seen, as asthma cases overall are higher in the Fall and the Winter. Also, since many of the sites were teaching hospitals, this could lead to practice differences given the release’s correspondence with the beginning of the academic year with a new influx of staff and students. Unfortunately, we only had patient demographics that are correlates of patient severity and no direct measures of clinical severity, so we could not account for differing severity driven by season. However, models were adjusted for both time and teaching status, and we had comparison/control sites gathering outcome data at the same time/season (because of differing usage levels between sites).

Third, we were unable to directly tie usage to study sites or clinicians, but rather city of use. Thus, not all use in a city can be assumed to have come from the project site. However, this methodology has been previously used by us and others to study mECDS effects., Additionally, the tool was released freely and required no registration to use. While this method of deployment allowed for easier scalability across the diverse network of sites, it also prevented us from being able to analyze user characteristics or definitively tie their use to the project.

Finally, since our study links geographically based usage to aggregate practice level changes in care quality, the exact mechanism by which mECDS use led to these improvements cannot be determined. At the individual user level, the mECDS tool may have provided real-time tailored decision support on evidence-based practices and/or may have provided education that led to longlasting user behavior change at both the user and the site levels. While there may also be residual confounding bias from overall site QI engagement (which correlated with cumulative mECDS usage), we tried to account for this by including measures of project engagement in the adjusted models.

CONCLUSIONS

Mobile electronic clinical decision support has the potential to overcome barriers to dissemination and improve quality of care and health outcomes across institutions. To our knowledge, this report is the first of its kind to attempt to link clinical practice to mECDS utilization on a national scale that controls for important potential confounders including case-mix differences and concomitant local QI efforts. We found cumulative mECDS utilization was associated with improvements in multiple measures of pediatric asthma care quality. Important areas of future work include improving mECDS uptake/utilization, linking clinicians’ mECDS usage to behavior, and studying mECDS’s impacts on other common pediatric conditions.

ETHICAL CONSIDERATIONS

The PIPA study was approved by the AAP Institutional Review Board. Teams at each participating site obtained local institutional review board approvals, as necessary. This analysis was deemed non-human subject research by the University of Nebraska Medical Center Institutional Review Board.

FUNDING

This work was supported by the Office of the Director of the National Institutes of Health (grant number UG1OD024953) through support of Ms. Kerns and Dr. McCulloh’s time, and also by the Agency for Healthcare Research and Quality (grant number K08 HS024592) through support of Dr. Kaiser’s time.

Conflict of interest statement

None declared.

AUTHOR CONTRIBUTIONS

EK drafted the study design, helped to develop and test the mECDS tool, performed the analysis, and drafted the manuscript. RM participated in the study design, helped to develop and test the mECDS tool, reviewed the analysis, and revised the manuscript. SF participated in the study design, led the testing and development of the mECDS tool, reviewed the analysis, and revised the manuscript. CM participated in the study design, helped test the mECDS tool, reviewed the analysis, and revised the manuscript. LK helped test the mECDS tool and revise the manuscript. PL helped test the mECDS tool and revise the manuscript. SK designed the PIPA quality improvement project, participated in study design and data analysis, and reviewed and revised the manuscript. All authors read and approved the final manuscript.

DATA AVAILABILITY STATEMENT

The data underlying this article cannot be shared publicly due to its potential to identify hospitals in association with their performance in the project described. The data will be shared on reasonable request to the corresponding author.