Quality and performance improvement: what's a program to do?

A confluence of forces is challenging traditional approaches to issues of quality in substance abuse care. The availability of effective, research-based interventions, the Federal emphasis on performance measurement and outcomes, and national initiatives to improve quality and data infrastructure are driving a transition from a static, compliance-oriented approach to a more dynamic performance improvement model. This new way of achieving and documenting quality will produce better outcomes for consumers and greater confidence in the value of substance abuse services, but first it will require new behaviors from all parties involved in the delivery of substance abuse prevention and treatment services. This article describes some of the shifts already under way and offers advice on how organizations can get ready for the coming changes.

Consider the following situations: While cleaning your 16-year-old daughter’s room, you come across a stash of pills. You’ve been aware of several changes in her behavior and attitude over the past 6 months. Her grades are down, and she has become surly and secretive; but you’ve been telling yourself not to worry, as surely all teenagers go through these phases. Now, standing in her room, you have the sinking feeling that your child is in trouble. After a heated confrontation with your daughter, you decide to call your managed care plan, which gives you the names of several programs and therapists available under your coverage. But how do you choose among them? You feel you may be confronting one of the most significant moments in your child’s young life, but how do you decide whom to call, with your daughter’s future possibly hanging on the decision?

You’re the assistant secretary for human services in the Governor’s office. You’ve been a thoughtful advocate for the needs of people with behavioral health disorders. This year’s budget is going to require cuts in the substance abuse service delivery system. You dislike the “percentage cut across the board” approach to system management. You’ve visited programs and have a feel for which programs appear to work better, but you realize that your subjective opinion based on a limited number of site visits will hardly carry the day if programs are cut selectively. How can you manage the downsizing process so that it has the least possible impact on the overall effectiveness of the system and the care delivered to people in need?

A lack of quality information isn’t the only challenge facing parents or system managers. Often, the available information is contradictory or lacks sufficient rigor to be useful for making decisions. In an article in The New York Times Magazine, David Sheff described his experience seeking help for his methamphetamine-addicted 19-year-old son, Nick (Sheff, 2005). As Sheff sought to determine the best course of action, he was often given conflicting advice. For example, one program advised Sheff to have Nick arrested, while another warned him not to do anything that might alienate his son. What advice has merit, Sheff wondered, and how can I distinguish? Sheff concluded that the substance abuse service system “must be the most chaotic, flailing field of health care in America.”

The Institute of Medicine (IOM), a nonprofit scientific organization chartered by Congress to inform and advise policymakers, addressed the need to improve the quality of substance abuse treatment nationwide in a milestone report entitled Bridging the Gap Between Practice and Research: Forging Partnerships With Community-Based Drug and Alcohol Treatment (Lamb, Greenlick, and McCarty, 1998). The report recommended that system and program managers promote the use of evidence-based practices (EBPs) in community treatment settings through funding and other incentives. A subsequent publication, Improving the Quality of Health Care for Mental and Substance-Use Conditions (Institute of Medicine, 2005), offers a broad vision of system transformation for the 21st century. It promotes a patient-centered system that is an equal partner in the country’s health care system, one that is grounded in the application of EBPs and committed to performance improvement based on objective, transparent measurement. Although the IOM recognizes the formidable obstacles to realizing this vision, it also defines the components of system transformation and challenges the field to mobilize the resources to move forward on an agenda for quality (see “Ten Rules To Guide the Redesign of Health Care”).

AGENDA FOR IMPROVEMENT

Implementing a quality and performance improvement (QPI) agenda at a system or individual provider level can be compared to building a three-legged stool. The three supports upon which the system will rest are the implementation of improvements in the delivery of care, including the adoption of empirically tested, effective clinical and administrative practices (the content); the measurement of results (the data); and a QPI method to establish and sustain the advances in quality (the process).

Some people would argue that a fourth leg may be the availability of human and financial resources commensurate with the breadth of the undertaking. A review of needs and available resources is required prior to building the stool and, hence, is implicit in this discussion.

Leg One: Evidence-Based Practices (the Content)

The substance abuse services community has not always recognized the contribution of well-designed research studies to improvements in the quality of substance abuse services. Although the world-class studies of the neurochemistry of addiction have fascinated the field and the general public, the products from the more mundane studies of effective treatment interventions have not been as widely distributed or as warmly received. Perhaps a relative shortage of user-friendly tools or the complexities of implementing an EBP with fidelity have diminished enthusiasm.

Programs have at their disposal several sources that offer information and, in some cases, technical assistance in selecting and implementing EBPs. Among these sources are the Substance Abuse and Mental Health Services Administration’s National Registry of Evidence-Based Programs and Practices, the National Quality Forum, and NIDA’s Clinical Trials Network.

National Registry of Evidence-Based Programs and Practices

The U.S. Substance Abuse and Mental Health Services Administration (SAMHSA) recently expanded its National Registry of Evidence-Based Programs and Practices (NREPP) to include treatment as well as prevention interventions. Treatment providers can consult the NREPP Web site for guidance in selecting the EBP that is best suited to their patients’ needs and program structure. The NREPP divides the EBPs into three categories: promising practices, effective practices, and model programs. In addition to a rigorous review of the research on each model program and practice, the NREPP now assesses its readiness for dissemination and adoption (e.g., are user-friendly materials and technical assistance available to providers or practitioners?). The NREPP will enable a program’s leadership to make thoughtful decisions about the suitability of a particular EBP and to consider what resources may be available to assist staff in implementing the practice. SAMHSA plans to have a redesigned NREPP Web site operational by spring 2007. To learn more about what the NREPP will offer, visit www.samhsa.gov.

National Quality Forum

The National Quality Forum (NQF) is another prominent group working to identify EBPs for use in community-based health and substance abuse treatment settings. The NQF, established in 1999 pursuant to Federal legislation, is a standards-setting organization for health care whose 300 members represent all aspects of the industry, including consumers, purchasers, providers, researchers, and quality improvement organizations. In its initial report on substance abuse, Evidence-Based Treatment Practices for Substance Use Disorders (Power, Nishimi, and Kizer, 2005), the NQF recommended seven categories of practice for consensus-building and quality measurement efforts: (1) screening, (2) initial brief intervention, (3) prescription for services, (4) psychosocial interventions, (5) pharmacotherapy, (6) patient engagement and retention, and (7) recovery management. A draft consensus report on these practices should be available by spring 2007. To view the NQF’s workshop recommendations, visit www.qualityforum.org/projects/ongoing/sud.asp.

Clinical Trials Network

NIDA’s Clinical Trials Network (CTN) brings together researchers and practitioners to study the effectiveness of well-researched interventions when they are implemented in community-based treatment settings. NIDA’s support of the CTN recognizes that practices shown to be efficacious under strictly controlled research conditions need to be tested in the real-world settings of community-based treatment programs, whose patients are more diverse and whose capacity to implement an EBP may vary considerably. The CTN’s “effectiveness research” provides an important laboratory for researchers and clinicians to study and evaluate the most promising of the interventions before wider distribution and dissemination efforts are undertaken. Nationwide, the CTN includes more than 130 community treatment programs and 17 research centers (www.nida.nih.gov/CTN).

Even in the CTN, however, EBP implementation remains somewhat more controlled and draws on more resources than community-based treatment programs typically enjoy. If the CTN’s research studies find a practice or intervention to be highly effective, the task of translating the intervention into clinically useful tools and disseminating them to the field remains. The Blending Initiative of NIDA and SAMHSA aims to meet this need. It brings together CTN researchers and the technology transfer experts of SAMHSA’s Addiction Technology Transfer Centers (ATTCs) to create the clinician-friendly products needed to move a practice out of the research arena and into wider use in treatment settings. The Blending teams have completed products for three EBPs to date: training of addiction professionals in buprenorphine treatment, use of buprenorphine for short-term opiate withdrawal, and use of the Addiction Severity Index in treatment planning. Teams are currently working on packages to support two other EBPs: clinical supervisors’ use of motivational interviewing and promotion of awareness of motivational incentives. For more information on Blending Initiative products, visit www.nida.nih.gov/blending.

Care is based on continuous healing relationships. Patients receive care whenever they need it and in many forms. The health care system is responsive 24 hours a day, every day; it is accessible over the Internet, by telephone, and by other means in addition to face-to-face visits.

Care is customized to patient needs and values. The system of care meets the most common types of needs and is able to respond to individual patient choices and preferences.

Patients receive full information about their condition and care options. Clinicians and patients communicate effectively and share information.

Patients exercise the degree of control they choose over the health care decisions that affect them.

Clinical decisions are evidence-based. Patients receive care based on the best available scientific knowledge. Care does not vary illogically from clinician to clinician or place to place.

Safety is built into the care system. The system includes mechanisms to prevent and mitigate errors and keep patients safe from iatrogenic injury.

Consumers have open access to information on health care organizations’ practices and performance. Patients and their families can obtain adequate information to make informed decisions when selecting a health plan, hospital, or clinical practice. The information includes data on organizations’ safety records, evidence-based practices, and patient satisfaction scores.

The health care system anticipates patient needs, rather than simply reacting to events.

Waste decreases continuously. The health care system does not waste resources or patient time.

Clinicians and institutions actively collaborate and communicate to ensure an appropriate exchange of information and coordination of care.

Adapted from .

The ATTCs have independently developed a wealth of resources to assist programs in selecting and implementing an EBP. The ATTC Web site (www.nattc.org/index.html) provides access to products from all 17 ATTC regional centers, covering topics as diverse as clinical interventions (e.g., motivational interviewing, addiction medications) and management strategies (e.g., optimizing use of Web time, managing change).

Leg Two: Performance Measurement (the Data)

Whether programs are implementing an EBP or tweaking a current practice to improve the quality of care, they need valid and reliable data to measure their performance and track their progress. Purchasers need solid, objective information to ensure that their investment yields sufficient value; regulators need it for accountability purposes; and consumers cannot make informed decisions about their choice of providers without it.

Performance measurement means developing, specifying, and testing measures to compare providers and service systems in core performance areas. It is “the selection and use of quantitative measures of program capacities, processes and outcomes to inform the public or a designated public agency about critical aspects of a program, including its effects on the public” (Perrin, Durch, and Skillman, 1999). Among sources programs may look to for help in choosing and implementing appropriate performance measures and standards are the Washington Circle and SAMHSA’s National Outcome Measures.

The Washington Circle

In March 1998, the Center for Substance Abuse Treatment convened a multidisciplinary group of providers, researchers, managed care representatives, and public policy representatives to develop performance measures for substance abuse services. The Washington Circle (WC), as the group came to be known, established two broad goals (McCorry et al., 2000):

Develop and pilot-test a core set of performance measures for use in the public and private sectors; and

Collaborate with a broad range of stakeholders to ensure widespread adoption of substance abuse performance measures by health plans, private employers, public payers, and accrediting organizations.

The WC has completed the specification and testing process for three measures: identification, initiation, and engagement in care in managed care settings. These can be computed from existing administrative encounter and billing data (Garnick et al., 2002); readers can obtain detailed programming specifications at www.washingtoncircle.org. The WC measures have been adapted by several national and state organizations to evaluate their members’ performance in delivering the early phases of treatment. For example, the National Committee on Quality Assurance (NCQA) has incorporated the WC measures into its standardized performance measurement system, the Health Plan Employer Data and Information Set (HEDIS). The performance of NCQA-accredited health plans on these measures is published and available, at a cost, to purchasers of care. Consumers can request information on specific HEDIS scores by contacting their health plan representative. This transparency is a boon for consumers dealing with alcohol or drug problems. The publication of the HEDIS results also motivates health plans to improve their performance to remain competitive with other health plans.

The WC is working with a dozen states to adapt the three specified measures to public sector systems, which, unlike managed care plans, have no “enrolled population” to determine their success in identifying and engaging individuals in need of treatment. Initial testing and modification of the WC specifications to fit the data capacities of block-grant-funded state systems have yielded promising results (Garnick, Horgan, and Chalk, 2006). Participating states continue to shape the WC measures to account for the multiple pathways through which a person in need may be identified and initiated into block-grant-supported care.

The WC currently is working on three new measures to expand its core measurement set in the prevention, treatment, and maintenance of treatment effects domains. The measures are: screening for alcohol and other drugs, medication-assisted treatment, and recovery management services. Specification and pilot-testing information on these measures will also be available on the WC Web site in spring 2007.

Another source of information on a provider’s or plan’s performance in delivering substance abuse services—besides administrative databases—is the consumer population. The WC is working with the Forum on Performance Measures in Behavioral Healthcare to develop and test a consumer survey that elicits patients’ treatment experiences across the domains of access, quality/appropriateness, and outcome/improvement. The survey consists of two modules: one is specific to substance abuse treatment and recovery, and the other is equally effective in surveying consumers of substance abuse treatment services and consumers of other mental health services. Testing of both modules is nearly completed, and the final survey will be posted on the WC Web site in spring 2007.

National Outcome Measures

The IOM report on mental health and substance abuse recognized that individuals with substance use disorders and other mental illnesses face similar challenges—for example, the neurobiological embedding and chronicity of their problems, stigma, and public skepticism about the effectiveness of treatments (Institute of Medicine, 2005). In an effort to bring some consistency to measuring outcomes for consumers with these conditions, SAMHSA, in collaboration with the state directors of mental health and addictive disorders, has identified a common set of measures for both fields. The National Outcome Measures, although they may be specified somewhat differently for mental health and substance abuse services, capture treatment outcomes that meaningfully reflect recovery in both conditions (see “SAMHSA National Outcome Measures for Substance Abuse Treatment and Prevention”). A majority of states are already reporting on some of these measures; SAMHSA anticipates that all states will have comprehensive reports by October 2007, resulting in the first “national picture” of outcomes. That picture, in turn, should provide the basis for a concentrated effort to improve performance in settings whose profiles indicate poorer performance. For an in-depth look at the National Outcome Measures and SAMHSA’s performance management strategy, visit www.nationaloutcomemeasures.samhsa.gov/outcome/index.asp.

Leg Three: Quality and Performance Improvement Methods (the Process)

Why do science-based practices seem to work their way into everyday practice so slowly? Why do some well-researched practices never become routine?

The lag time between discovery and widespread application of an innovation is not a new issue: The British Admiralty did not mandate that Navy diets include citrus fruits until nearly 200 years after the initial findings that they prevent scurvy. Recently, however, the emerging field of implementation science has begun to make progress in understanding the factors that facilitate or inhibit adoption of innovations. An excellent monograph by Fixsen and colleagues (2005) describes the implementation of innovations as a multilevel process involving strategies (e.g., the availability of coaching to the users of the innovation), organizational dynamics (e.g., leadership endorsement and organizational readiness to change), and external influences (e.g., state agency regulations and reimbursement practices). Together, these three forces interact to promote or doom the innovation. Fixsen and colleagues’ review of the literature suggests that even the most effective, well-researched practices will fail without an adequate implementation strategy.

Achieving the promise of improved outcomes through the implementation of innovative practices, including EBPs, and the transparent measurement of performance will require partnerships among government, purchasers, and providers. The Network for the Improvement of Addiction Treatment, a collaborative project between SAMHSA’s Center for Substance Abuse Treatment and The Robert Wood Johnson Foundation, has been a leader in fostering these partnerships.

The Network for the Improvement of Addiction Treatment

For busy chief executive officers or clinicians, the thought of changing business or clinical practices can seem daunting. They may worry that the change exercise will add burdens—more paperwork, more meetings, more goal-setting—and produce uncertain or disappointing results. Past experience with state quality improvement regulations and accreditation requirements may have reinforced the notion that QPI is just a writing exercise, extraneous to the real work of the organization.

The Network for the Improvement of Addiction Treatment (NIATx), which is based at the University of Wisconsin, is collaborating with providers and state agencies across the United States to overcome these potential pitfalls. The NIATx has identified a set of critical ingredients for successful organizational change and is assisting providers and state agencies in identifying practical, efficient ways to improve patient outcomes and provider bottom lines. Specifically, the NIATx and its collaborating programs are focusing on practices that can reduce treatment seekers’ waiting times and no-shows and increase admissions and early retention rates.

Dr. David Gustafson and colleagues at the University of Wisconsin have developed a rapid-cycle quality improvement process to implement the changes in the access and retention practices of participating providers. Their “Plan-Do-Study-Act” process does not bog down in endless reports and meetings; rather, it emphasizes a planning, implementation, and evaluation cycle that is simple, fast, and inclusive. The method features ongoing support for the change process through learning collaboratives facilitated by expert coaches and peer mentors who have already implemented changes in their treatment agencies. Since the project’s inception in 2003, participating providers have reported a 35 percent reduction in wait times, a 33 percent drop in the number of no-shows, a 22 percent increase in admissions, and a 23 percent improvement in early retention rates. These results have improved staff morale and the providers’ bottom lines. The NIATx Web site, www.NIATx.net, offers a wealth of information on actual improvements implemented by treatment agencies, as well as introductory materials that interested agencies can use to get started.

GETTING STARTED

What should a busy chief executive officer, clinical director, counselor, or state agency manager do to formulate and carry out an agenda for quality? There are a number of steps a manager can take to move the quality agenda to his or her organization’s front burner.

Provider Clinical and Executive Leadership

Begin To Quantify and Measure Performance

Many people believe performance measurement is a complicated business requiring specialized skills and complex formulas. Nothing is further from the truth. To measure how your program is doing in key performance areas like initiation and early retention, follow the specifications on the WC’s Web site or the access and retention measures on the NIATx Web site, and start measuring how many clients come back for a visit within 14 days and how many return twice more within 30 days of their first visit. Grab a baseline rate from a quick review of when patients came into your program and when they left. The data may be readily available, as your program is probably already reporting this indicator to the state government. Discuss the baseline results with your staff or clinical leadership over a brown bag lunch. Together, determine how you can improve the numbers by simple adaptations of current practices. Consult the NIATx Web site for descriptions of how community-based public sector providers have already improved access and retention.

You may find that simply paying more attention to your program’s initiation and early retention processes improves your performance indicators and that there is no need to make any formal changes. If so, pat yourself and your staff on the back, because your willingness to consider improvements may have increased your patients’ chances of succeeding in that crucial first month of treatment when so many drop out or give up. If the change exercise doesn’t improve your organization’s performance, don’t fret. Consult with staff involved in implementing the change, and try something else. Make sure you have a strong change leader and executive-level endorsement for the change, so you can count on some support when the implementation process gets a little bumpy.

The French philosopher Voltaire wrote, “The best is enemy of the good.” Don’t let apprehensions about the quality of the data stop you from acting. Use what you have to initiate a change process. The NIATx Web site also provides guidance on how to do this.

For some programs—especially large, hospital-based programs—the problem may be too much data. It’s not unusual, for example, for the substance abuse and the mental health clinics of the same hospital to be unable to share data with each other or even for each clinic to be unaware of what data the other has at its disposal. The trick to avoid paralysis by such technical challenges is to be practical and concrete. Get started with data you can manage, perhaps with some informal help from the management information systems department or a friendly researcher in a local university or college.

Select an area for change that has some real traction with staff, and identify an “idea champion” who is respected by staff and interested in improving this particular aspect of care. Use what has already been collected as much as possible, rather than burdening staff with extra paperwork. Involve the staff who will do the work when organizing the QPI project. If additional reporting is necessary, make sure your idea champion and your staff understand and accept the need for the extra burden, with the stated purpose that the temporary increase in paperwork will lead to better patient outcomes or reduced burdens down the line.

Use the Data You Collect

Nothing frustrates clinicians more than spending their limited counseling time on paperwork that seems to disappear into the black hole of the administration or a funding agency. To secure staff support for performance improvement efforts, make sure the change exercise is relevant, the data get used, and the findings are reported to the people collecting the data. Challenge your funding agency to examine the necessity of the current data collection requirements. Make the data as user-friendly as possible, and make sure that it benefits the clinical process in tangible ways.

Partner With a Researcher

A research partner can greatly facilitate your efforts to develop a measurement-driven QPI culture. For example, when scoring a proposed research study for funding, both NIDA and the National Institute on Alcohol Abuse and Alcoholism consider the project’s potential impact on patient outcomes and look for evidence of earlier pilot studies or other research for the timeliness and relevance of the research question. Take advantage of the opportunity these policies afford by partnering with a researcher on a QPI project whose results and data may also serve as a backdrop for a more formal research application.

Create a Crisis

Get a copy of SAMHSA’s National Outcome Measures, and ask the staff person in charge of information technology how the organization is going to capture these data and how the data will look to consumers and regulators when reported. Do the same with your board of directors and other stakeholders—alumni, patient advisory boards, and elected officials. Share the problem. Do not wait to be told that your program’s (and your clients’) future depends largely on your organization’s ability to collect, analyze, and use data to improve performance. The literature consistently identifies leadership as a key variable in the success of innovative, adaptive organizations. Take this role seriously, and gather the support you need to transform your organization’s culture.

State Agency Managers

Single State Agencies (SSAs) will not achieve desired quality outcomes through regulation alone. Regulations may bring compliance, but compliance scores are not equivalent to quality outcomes. Nevertheless, SSAs have powerful tools at their disposal to assume a leadership role in QPI work.

Exercise Leadership

Improving quality and patient outcomes is the shared responsibility of every manager and unit in the state agency. SSA leaders should express this formally to employees. Reinforce this value through agency-wide briefings on the principles of QPI and its importance to the SSA’s overall mission. Many of the ongoing functions of a state agency can support a QPI agenda, if the agency personnel are trained to identify opportunities for QPI activities within their ongoing responsibilities.

Form an Interdivisional Committee

The agency’s program, budget, auditing, and technical assistance functions must recognize their individual and collective roles in fostering and managing QPI activities. There often are unrecognized opportunities to promote a QPI agenda during contracting and review activities that do not affect funding levels, but communicate a powerful message about the importance of QPI to the State. For example, the Interdivisional Committee might agree to a standardized approach to a common problem confronting managers and providers, such as screening all new clients for mental health problems. In collaboration with the provider community, the Inter-divisional Committee would design a Plan-Do-Study-Act series to identify and implement a standard approach to screening. Members of the committee would:

Review ATTC and other SAMHSA resources in selecting a screening approach;

Ensure that all components of the agency had input into the design of the Plan-Do-Study-Act series and are prepared to support its implementation; and

Collect data to evaluate the impact of the screen on identification and early retention rates for persons with co-occurring disorders, ensuring that the value, or lack of value, in adopting this practice is known.

Partner With Providers and Local Government

Performance improvement must be an SSA-endorsed, provider-driven process. If the State is to be the guardian of quality, the providers must be its managers. The patient-centered system of care envisioned in the IOM report is impossible without an active partnership among government, providers, and consumer representatives. The State is best positioned to convene and manage the collaboration. A survey of providers and counties on their use of EBPs and QPI processes can generate interest as well as information on improving outcomes by improving quality.

Contract for Performance

Pay for performance is a relatively new construct in the substance abuse services field. Marton, Daigle, and de la Gueronniere (2005) identified three types of “purchasing levers” among states promoting the use of EBPs in substance abuse treatment: standardization of criteria, contractual requirements, and performance incentives. Incentives include financial payments, regulatory relief, competitive bidding process advantage, and infrastructure support. The State of Delaware has incorporated payment incentives into its provider contracts in three areas: engagement/utilization, attendance, and program completion (Kemp, 2006). An evaluation of the effects of the incentive payments on provider performance is currently under way.

Performance-based contracting and reimbursement practices for substance abuse services are in the early stages of development. Pilot studies of incentives within existing State-provider contractual relationships for QPI activities and improved outcomes will open a window on a key factor in sustaining QPI processes over the long term.

CONCLUSION

The substance abuse service system is in the early stages of a major transition to a more accountable, consumer-centered system of care, where the use of EBPs is the norm and performance is measured and transparent. The tools to effect this transition—the EBPs, performance measures, and implementation methods—are available now. Quality and performance improvement standards will take hold only if all parties in the delivery of substance abuse services collaborate to align policies and practices that support these values.

SAMHSA NATIONAL OUTCOME MEASURES FOR SUBSTANCE ABUSE TREATMENT AND PREVENTION

DOMAIN	OUTCOME	MEASURES
		Substance Abuse
		Treatment	Prevention
Reduced morbidity	Abstinence	Change in frequency of use at date of last service compared to date of first service	Changes in 30-day substance use prevalence, age of first use, and perception of the risks of drug use
Employment/education	Increased/retained employment or return to/retention in school	Change in percentage of clients employed or in school at date of last service compared to first service	Change in alcohol- and drug-related termination, suspension, and expulsion rates; change in attendance, enrollment, and employment rates
Crime and criminal justice	Decreased criminal justice involvement	Change in number of arrests in past 30 days from date of first service to date of last service	Change in number of alcohol-related car crashes and injuries; change in prevalence of alcohol- and drug-related crime
Stability in housing	Increased stability in housing	Change in percentage of clients in stable housing situation at date of last service compared to date of first service	Not applicable
Retention	Increased retention in treatment	Changes in length of stay from date of first service to date of last service	Total number of evidence-based programs and strategies; change in percentage of clients seeing, reading, watching,or listening to a prevention message
		Unduplicated count of persons served

This table has been modified from the original, which is available at .