A Learning Collaborative Approach to Improve Mental Health Service Delivery in Pediatric Primary Care.

BACKGROUND: Pediatric primary care practitioners (PPCPs) report inadequate training in the care of children with common mental health conditions. Although additional training is needed, system changes are also necessary to support improvements in care.
METHODS: We developed the Building Mental Wellness Learning Collaborative to assist PPCPs in delivering better mental health services in primary care by targeting 5 focus areas: mental health promotion; early identification and screening; practitioner skills; collaboration and community linkages; and medication management. Aims were developed for each area.
RESULTS: Twenty-one practices and 50 practitioners completed the collaborative in 2 seven-month waves. For mental health promotion, ≥85% of charts showed documentation in 3 of 4 preselected areas. For early identification/screening, screening increased, but the ≥85% goal was not met. For practitioner skills, a ≥20% increase in the proportion of children/youth ≥1 visits for anxiety or depression was achieved, from 0.70% of children/youth in the 12 months preintervention to 1.09% children/youth in the 12 months after. For collaboration/linkages, mental health referral completion was unchanged and below the 60% goal. For medication use, a ≥15% increase in selective serotonin reuptake inhibitor prescribing by Building Mental Wellness (BMW) practitioners was achieved from 0.72% children/youth with office visits pre-BMW to 0.92% post. Prescribing did not decrease for atypical antipsychotic medication use or for psychotropic medication use in children younger than 6 years, although there was a trend toward more appropriate prescribing.
CONCLUSIONS: The BMW Learning Collaborative was effective in helping PPCPs implement certain aspects of a comprehensive approach to the delivery of mental health services in primary care.

INTRODUCTION

Thirteen to 20% of children have a diagnosable mental health disorder,[1] and pediatricians are increasingly tasked with the diagnosis and management of childhood mental health conditions.[2] Many pediatricians have limited training in mental health, are not comfortable treating these disorders, or believe it is not their responsibility to treat mental health conditions in children.[3-7] Training in mental and behavioral health conditions is often limited in pediatric residency programs,[4,5] resulting in variable familiarity and comfort with assessment tools and evidence-based treatment.[8] Not surprisingly, pediatric primary care practitioners (PPCPs) report low comfort levels managing mental health conditions, including using standardized assessment tools, screening, and ongoing management.[3] It is also unclear to what extent pediatricians feel that management of mental health disorders is within their scope of practice. A 2008 survey of pediatricians found that the majority believed they have the responsibility to identify and refer patients with anxiety, depression, and behavior problems, but not to treat these disorders.[7]

To address these areas of need, several national organizations have developed standards to improve care for children with mental health concerns in the primary care setting. In 2009, the American Academy of Pediatrics (AAP) stressed the need for improved pediatric mental health care training and ongoing education by defining 6 key competency areas for practitioners: systems-based practice, patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, and professionalism.[3] Similar principles have been outlined by Family Medicine organizations[9] and the American Academy of Child and Adolescent Psychiatry.[10]

Comprehensive practice change will likely be necessary to realize these improvements in care. Regional and state-wide efforts to improve mental health care are underway, both in terms of improving pediatric residency training[11] and improving service delivery models for current practitioners.[8,12] Several states have developed resources to support primary care practitioners through telephone-based decision support (see www.nncpap.org) and/or websites containing tools and other materials. Although there is evidence of the effectiveness of these interventions in some settings,[13,14] comprehensive practice change has not been well studied.

The Building Mental Wellness (BMW) learning collaborative was designed to help PPCPs implement practice-based mental health services based upon the AAP’s mental health competencies using a quality improvement (QI) approach to evaluate their progress. The BMW project was developed in part to address both national and statewide concerns regarding inappropriate use of psychotropic medication, including overuse of atypical antipsychotic medication (AAM) and overuse of medication in young children.[15,16] Using the Chronic Care Model framework[17] and the AAP competencies as a guide, we conceptualized mental health care as involving 5 foci on which practices could concentrate improvement efforts: mental health promotion; screening and early identification; practitioner skills;[18,19] collaboration and community linkages; and medication management. Aims for each focus area were developed based upon best practice guidelines,[20-27] expert opinion, and available Medicaid/OCHIP claims data, balancing goals for improvement with feasibility (see Table, Supplemental Digital Content 1, available at http://links.lww.com/PQ9/A53, which describes learning collaborative aims by focus area). Key stakeholders including government agencies, mental health professionals, and primary care practitioners participated in the development of the conceptual framework, measures development, and proposed interventions. We hypothesized that practices would meet goals for each focus area they selected.

METHODS

Setting and Participants

The project was approved by the Institutional Review Board at Nationwide Children’s Hospital (IRB13-00398), and the requirement for informed consent was waived for participating practitioners and families. Recruitment was conducted in 2 waves from April 2013 to June 2013 and from August 2014 to November 2014 through the Ohio Chapter, AAP and Partners for Kids, a physician hospital organization affiliated with Nationwide Children’s Hospital. Due to funding requirements, practices with a payer mix of at least 30% Medicaid/OCHIP were preferentially recruited. Once enrolled, participating practices identified an improvement team comprised of at least 3 core team members: a physician leader, a nurse/nurse practitioner or medical assistant, and an administrative staff/office manager. Core team members were expected to educate other participating practitioners from their practice on the collaborative content. Practitioners were awarded American Board of Pediatrics (ABP) maintenance of certification part 4 credit and continuing medical education credit for participation.

Intervention Design

The BMW learning collaborative was modeled after the Institute for Healthcare Improvement’s Breakthrough Series Collaborative model for its QI process.[28] This model was designed to facilitate the spread and adaptation of existing knowledge to multiple settings. In accordance with the model, BMW participants attended a one-and-a-half day face-to-face learning session, where they learned about topic areas from faculty experts. Didactic content was informed by best practice guidelines and policy statements from the AAP, the American Academy of Child and Adolescent Psychiatry, and related organizations.[20,22,24,25,27,29-33] Participation consisted of attendance at the learning session by at least 1 member of the practice, monthly data collection, and monthly webinars for additional didactics and data review over a 7-month period (see Table, Supplemental Digital Content 2, available at http://links.lww.com/PQ9/A54, which describes content covered during the learning session and monthly calls).

All practices were required to engage in improvement activities for collaboration and community linkages and were required to choose at least 1 additional focus area (from Table, Supplemental Digital Content 1). Practices were given tools to further their activities, including handouts for mental health promotion that were developed as part of the learning collaborative (available at https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/EBCD/Pages/Patient-Handouts.aspx), screening tools, and templates for resource directories. QI support was provided to each practice through 2 state-supported QI coordinators assigned to the BMW project. As per ABP requirements and the Institute for Healthcare Improvement model, practices were required to complete 3 Plan-Do-Study-Act (cycles) to implement change and measure improvement. Examples of Plan-Do-Study-Act cycles include process changes to increase screening implementation or handout documentation and implementation of a referral log to track mental health referrals.

Data Collection, Measures, and Analysis

Representatives from each practice completed the AAP’s Mental Health Readiness Inventory (MHRI)[34] to assess their practice’s readiness to deliver mental health care and to plan their improvement activities. This was repeated at the end of the collaborative, when the practices were also asked the question, “What barrier(s) do you anticipate that might make it difficult to sustain the BMW changes you have made?”.

To obtain baseline chart review data, all practices were asked to review charts for the presence or absence of specific elements corresponding to measures for each focus area. Practices reviewed 10 charts for children seen for a 36 month health supervision visit; 10 charts for adolescents (13–18 years of age) seen for a health supervision visit; and 5 charts for patients who had been prescribed AAM’s. After the baseline period, practices collected monthly chart review data only for their focus areas. Data were entered into a web portal platform designed by the Ohio Chapter, AAP for QI reporting. Where possible, data were plotted on a run chart. Standard probability-based rules were used for interpretation of the run chart.[35]

Finally, family satisfaction data were collected through Family Surveys, completed by parents/caregivers electronically using Survey Monkey on tablets provided by BMW at project start and at the end of the their wave of the collaborative. Family satisfaction with care was measured using the Family Survey, a 10-item questionnaire developed for the BMW project and patterned after items from the National Survey for Children with Special Health Care Needs. Sample questions included, “During the past six months, have you felt that you could have used extra help in arranging or coordinating care among different health care providers or services?” Responses were assessed using a 5-point Likert scale and were de-identified. All practices were asked to complete at least 5 surveys per practitioner after health supervision visits.

Focus Area 1: Mental Health Promotion.

Measures included chart documentation of anticipatory guidance in 4 areas: importance of parental mental health, parents being tuned in to their emotions, use of time in/time out, and caregivers verbally labeling emotions.

Focus Area 2: Early Identification and Screening.

Measures included percentage of patients less than 2 months old with documentation of postpartum depression screen, percent of patients less than 2 months old with documentation of a family health risks screen, percent of patients 9–18 years old with documentation of an alcohol screen, and percentage of patients 13–18 years old with documentation of a depression screen. The family health risk screen is a nonstandardized tool developed for the BMW project to assess parental alcohol use, intimate partner violence, and family functioning using published tools and/or guidance.[21,22,36] These data were collected by chart review performed by office staff, as described above.

Focus Area 3: Practice-based Interventions.

Medicaid/OCHIP paid claims data were used to assess the proportion of children/youth who had one or more office visits that were billed with an anxiety or depression International Classification of Diseases, Ninth Revision (ICD-9) code (list of ICD-9 codes available on request). Change in the proportion of children/youth with one or more office visits for anxiety or depression in a month was chosen to assess practice change for this focus area, given that pediatricians report lack of comfort with these conditions.[4] A 2-proportion z-test was used to test the hypothesis of no difference in the proportion of children/youth with one or more office visits for anxiety or depression in the 12 months before (“pre”) and after (“post”) the BMW intervention. To track change over time, the proportion of children/youth with diagnoses in months pre-, during, and postintervention was plotted on a control chart. Standard probability-based rules were used to calculate the mean and control limits. In addition, the number of contacts with the Pediatric Psychiatry Network (PPN), Ohio’s psychiatric decision support program for primary care, was measured. PPN is a state-funded program that allows for telephone-based consultation between primary care practitioners and Child and Adolescent Psychiatry. Calls from all BMW practitioners to PPN were tracked on a monthly basis.

Focus Area 4: Collaboration and Community Linkages.

Time to mental health appointment was chosen as a proxy measure to assess collaboration and community linkages. Measures for this focus area included percentage of mental health appointments scheduled within 8 weeks of referral (wave 1). Because this measure did not fully capture the desired outcome (eg, completed appointments), practices in wave 2 were asked to track visits attended within 8 weeks of referral. Referrals were tracked by office staff through use of a referral log, which was provided to the practices. All practices were provided with a resource directory and were required to add additional local resources pertinent to their community.

Focus Area 5: Medication Management.

Data for metabolic monitoring for children prescribed an antipsychotic medication by a BMW practitioner were obtained by chart review (eg, presence of order for lipids and glucose before medication start). Medicaid/OCHIP paid claims data were used to assess changes in 3 areas of prescribing through analysis of psychopharmaceutical medications prescribed by BMW providers and filled by clients in the month following their office visit(s): (1) selective serotonin reuptake inhibitor (SSRI) prescribing for all children/youth with an office visit in month; (2) AAM prescribing for all children/youth with an office visit in month; and (3) psychotropic medication prescribing for children under 6 years of age with an office visit in month. A list of the generic medications and corresponding National Drug Codes included in each class, modeled after the Mental Health Resource Network categories described by Hacker et al,[37] is available upon request. Differences in pediatricians’ prescribing patterns in the 12 months pre- and postintervention were identified using 2-proportion z-tests. The proportion of children/youth prescribed each class of medication by a participating BMW provider following a visit in the months pre-, during, and postintervention was in turn plotted on a run chart. Analyses were completed using the statistical software Stata.[38]

Data for waves 1 and 2 were combined for the purpose of analysis (with the exception of Focus Area 4: Collaboration and Community Linkages, due to differences in data collection).

RESULTS

After receiving informational mailings, 39 practices expressed interest in participation and completed the MHRI. Twenty-two practices participated in the learning session. One practice dropped out, citing time commitment as a barrier to participation (Fig. 1). In total, 50 practitioners in 21 practices completed the learning collaborative in 2 sequential waves over a period of 7 months. Practice characteristics are summarized in Table 1.

Fig. 1.

Enrollment, focus areas, and drop out for participating practices.

Table 1.

Practice Characteristics

Fifteen practices elected to focus on mental health promotion, as presented in Figure 2. Increased chart documentation was noted in 4 key areas: parental mental health, caregiver emotions, specific parenting practices (use of “time in” and “time out” to promote positive relationships and manage challenging behavior), and verbally labeling emotions. Seventeen practices chose to focus on early identification and screening, which was measured by documentation of postpartum depression, family health risks, alcohol use, and depression screening, as presented in Figure 3. Practices improved from baseline in 3 of 4 areas, with the exception of alcohol screening, which remained high throughout the collaborative. Medicaid/OCHIP claims data corresponding to the practice-based interventions focus area were available for all practices. A 0.38% point increase [95% confidence interval (CI), 0.29–0.49; P = 0.000] in the proportion of children/youth with one or more office visits for anxiety or depression was noted per goal, from approximately 487 out of 68,619 (0.70%) children/youth in the 12 months before the intervention to 697 out of 64,151 (1.09%) children/youth in the 12 months after (Fig. 4). Rates of contact with PPN was collected for all practices and increased by 50% (from 10 to 17 calls per month) for wave 1, per goal, but not for wave 2 (from 27 to 31 calls per month). Participants cited preference for a consistent PPN provider and delayed response times as reasons for limited use.

Fig. 2.

Run chart showing changes in documentation of mental health promotion during BMW participation.

Fig. 3.

Run chart showing changes in documentation of mental health screening during BMW participation.

Fig. 4.

Control chart showing changes in proportion of children/youth with one or more office visits in a month related to anxiety/depression showing UCL and LCL by month and grand mean. LCL = lower control limit; UCL, upper control limit.

All practices were required to focus on collaboration and community linkages. The percentage of mental health appointments scheduled (wave 1) or completed (wave 2) within 8 weeks of referral did not improve over the course of the collaborative, from 67% to 48% in wave 1 (n = 3 to 39 patients tracked per month) and 33% to 26% in wave 2 (n = 2 to 59 patients per month). By tracking referral disposition in wave 2, practices found that in most cases (79%), family and other ecosocial factors contributed to the unattended referral (eg, no longer being interested in the appointment and transportation barriers).

Five practices chose to focus on medication management. Improvements in this area were measured by documentation of metabolic monitoring (baseline lipid levels, glucose levels, and body mass index) for patients who were prescribed AAMs by BMW participants. Improvements were noted, but results were limited by small numbers of patients (≤11/mo).

Medicaid/OCHIP pharmacy claims data were available for all practices. Prescribing trends for children/youth having one or more office visits with a BMW provider in a month are presented in Figure 5. Per goal, the overall proportion of children/youth who were prescribed an SSRI by a BMW provider increased 0.20% points (95% CI, 0.10–0.30; P = 0.000) from 0.72% (491/68,619) in the 12 months pre-BMW compared with 0.92% (587/64,151) in the 12 months post-BMW. AAM prescribing did not decrease per goal, instead remaining steady at about 0.47%. The goal of decreased psychotropic medication prescribing by BMW providers for children under age 6 was also not met, increasing slightly from 0.60% (253/42,084) in the 12 months pre-BMW to 0.85% (323/38,048) post (P = 0.000). However, there was a notable increase in the proportion of children/youth prescribed an AAM with bipolar disorder, autism, or ID/DD, increasing 9.46% points (95% CI, 1.35–17.6; P = 0.022) from 28.4% (86/303) pre- to 37.8% (84/222) during, before dipping slightly to 33.2% (106/319) post. This may be an indicator of more appropriate AAM prescribing. Diagnoses and ICD-9 codes corresponding to each class of mental health condition are available from the authors upon request.

Fig. 5.

Trends in psychotropic medication prescribing by BMW providers for children/youth who had office visits before (before month 0), during (months 0 through 6), and after (month 7 onward) the intervention.

The majority of Family Survey respondents reported being highly satisfied with care, both at baseline and at the end of the collaborative. Due to this ceiling effect, the Family Survey data analysis did not provide meaningful results. Data from the MHRI was not used for comparison because initial survey contained data from practices who did not participate in the collaborative. Twenty-five respondents completed the sustainability question. Thirty-six percentage (n = 9) cited no barriers to sustainability. Systems issues/time (36%, n = 9), staff issues (16%, n = 4), and patient/family cooperation (12%, n = 3) were cited as barriers.

DISCUSSION

A learning collaborative approach was effective in helping Ohio PPCPs implement certain aspects of a comprehensive approach to mental health care in their practices. Although pediatricians frequently report a low level of comfort managing mental health conditions, Medicaid/OCHIP claims data suggest an increase in primary care assessment and management of mental health conditions (ie, for anxiety and depression) over the course of participation. Significant improvements were also noted in documentation of key aspects of mental health promotion and mental health screening.

Other aspects of care were less responsive to improvement. Limited improvements were noted in the timeliness of mental health referrals, both for scheduled and attended visits. Data gathered by practices cited family and other ecosocial factors as reasons for missed appointments, suggesting that both systems issues and individual determinants of health-related behaviors represent barriers to prompt referrals. Although not possible for this learning collaborative, integration of a behavioral health provider into the primary care office may have increased completed referral rates.[39] Use of PPN, Ohio’s psychiatric decision support program for primary care, did not change over the course of the collaborative, despite multiple efforts to increase use. Programs like PPN have been highly successful in other states,[14,40] especially when paired with facilitated referrals. It is possible that low PPN usage in our study was due to perceived barriers such as lack of a consistent PPN provider contact and response time. Additional activities to further integrate PPN consultants into the practice may have been beneficial.

The BMW project also aimed to improve medication prescribing. Based on available state data, it was hypothesized that use of psychotropic medications in young children and AAMs in all children/youth would decrease and that SSRI use for all children/youth would increase. Figure 5 shows that while SSRI prescribing increased, as per goal, AAM and psychotropic prescribing in young children did not decrease, but rather remained stable or increased slightly. It is likely that with increased screening, paired with education and support, participants identified more children/youth with anxiety and depression and felt more comfortable prescribing medication. For AAM prescribing, a higher percentage of children/youth were seen for a potentially related condition in the weeks before the medication being filled, which may indicate a trend toward improved quality. Of note, the AAP’s updated guidelines for Attention-Deficit/Hyperactivity Disorder assessment and management[33] were published just before the start of BMW, although typically guideline publication does not lead to rapid practice change.[41] Although medication management didactics were offered to all practices throughout the collaborative, only 5 practices chose to work on this focus area specifically. It is possible that our results may have been more robust if more practices chose to concentrate their improvement efforts on this focus area.

Our study contained certain limitations. Participating practices represented a select group, notably Ohio primary care practices who volunteered for participation and who may represent “early adopters” or “high performers” rather than typical primary care practices. Active participation in BMW was typically limited to the practice’s improvement team. It is difficult to determine if these changes disseminated to all staff and to what degree they might be sustainable. As in most learning collaboratives, chart review data were collected by participants, and inter-rater reliability estimates were not obtained. Chart review data capture only the documentation of care provided, rather than assurance that quality care was actually delivered. Similarly, differences in care measured by claims data may reflect changes in coding practices rather than changes in care delivery. Our study primarily evaluated process change and service outcomes; patient outcomes were not assessed. We were also not able to examine other external factors that may have influenced mental health care, such as changes in insurance coverage or likelihood of presentation with mental health concerns.

A third wave of BMW has been built upon this study’s successes and limitations. Due to concerns about the potential for “QI fatigue,” a tiered level of participation has been developed, such that PPCPs can participate solely in informational, on-demand CME learning modules on common mental health problems or in the multifaceted learning collaborative. To support adoption of mental health care behaviors among both clinical and nonclinical staff, site visits based on Wissow’s common factors adaptation of the chronic care model to primary care have been developed to teach enhanced communication and influence skills.[42] These site visits work in tandem with learning sessions and action period calls to actively engage the entire office staff in fostering an organizational context that supports mental health care. Wave 3 practices are also encouraged to build relationships with community mental health partners and to increase their level of mental health/primary care integration, as measured by the Substance Abuse and Mental Health Services Administration’s “Standard Framework for Levels of Integrated Healthcare.”[43]

In summary, the BMW learning collaborative represents a promising effort to improve PPCP’s delivery of mental health services within their practice. Changes in physician behavior representing service improvement were noted in certain aspects of care. Aspects of care that involved coordination with other systems (eg, mental health) or were associated with lack of perceived benefit were less responsive to change. Waves 1 and 2 of this project built a strong foundation for comprehensive practice change via learning collaborative participation. BMW wave 3 has been redesigned to build on strengths and address barriers to mental health promotion at multiple ecosocial levels. Further study is needed to better understand the core elements of training and support needed to bring about improvements in mental health care delivered in the pediatric primary care setting.

ACKNOWLEDGMENTS

The authors acknowledge Mrs. Cathy Jaworski and Mr. Sean O’Hanlon for their assistance in the implementation of the learning collaborative. Building Mental Wellness was supported by the Ohio Colleges of Medicine Government Resource Center with funding from the Ohio Department of Health and the Ohio Department of Medicaid. The project described was also supported by Funding Opportunity Number CMS-1c1-12-0001 from Centers for Medicare and Medicaid Services, Center for Medicare and Medicaid Innovation. Views stated in the report are those of the researchers only and are not to be attributed to the study sponsors, the Ohio Department of Medicaid, or to the federal Medicaid Program.

DISCLOSURE

The authors have no financial interest to declare in relation to the content of this article.