Quality assurance for a program of comprehensive care for older persons.

Quality assurance (QA) for comprehensive programs like the Program of All-inclusive Care for the Elderly (PACE) requires a special strategy. The assessment phase should be capable of looking across the usual subdivisions of care to recognize the contributions of various disciplines, and to focus on the effects of that care on the patient. Measures should thus include both problem-specific and patient-focused elements. The tracer technique which follows the care of specific problems provides an opportunity to look at both the process and outcomes of care. An outcomes focus which looks at patient functioning as well as condition-specific parameters can include specific sentinel events whose presence suggests untoward developments. Quality assurance implies more than assessment. It represents a commitment to act responsibly on the information obtained to improve the care rendered. It includes a strategy for proactive involvement where caregivers are prompted to consider pertinent information in a timely fashion, and a retrospective remedial approach where the data are analyzed and presented in a format that can be readily understood and which suggests next steps to improve care.

Introduction

Current pressures for health care reform have created renewed interest in managed care and its predecessor, capitated care. The challenge in any system of capitated care is to provide more efficient services, that is, to achieve comparable health outcomes within a fixed budget. Because the economic pressures encourage underservice, quality of care efforts must ensure that the services provided are adequate. This challenge is difficult enough when the services are restricted. When the service package is broad and is targeted at frail older persons, the task becomes formidable. This article describes an approach to assuring quality of care for a program that seeks to provide a comprehensive set of acute and chronic care services to a population of frail older persons.

QA implies both a means of assessing the level of quality and the services provided, and a commitment to taking effective steps to correct deficiencies uncovered. The PACE model is one of the few capitated programs that offer comprehensive coverage for both acute and long-term care exclusively to frail older persons. (Kane, Illston, and Miller, 1992) The PACE model is designed to provide and manage a set of comprehensive services to a disabled elderly population, and is based on the success of the On Lok Senior Health Services program in San Francisco. The PACE sites provide services in varied organizational settings, but have several common elements. Each site: (1) offers the same array of acute and long-term care services, either directly or through contract; (2) is financed through both Medicaid and Medicare funds; (3) uses a multidisciplinary team to provide services; and (4) is based on a day health care model that is integrated with primary care. The challenge of the current project is to develop an integrated approach to quality assessment that can be used across the divers sites to reflect the concerns relevant to both acute care and long-term care services provided under capitation, and to the specific needs of a frail elderly patient population. Ideally, such a system should reflect the patient-centeredness which is a hallmark of the On Lok approach.

Specifically, this program will provide for an externally conducted review of quality of care with provisions for feeding back information to each site to facilitate improvement in the care provided. This review process is undertaken to provide a more consistent approach to assessing quality across the various States in which the sites are located. As a HCFA-sponsored project, the program will be conducted in coordination with the State agencies responsible for assuring the quality of health care programs like PACE.

Project Design

The design for the external QA program was developed after reviewing the literature on QA, especially as it pertained to care of older persons and the implementation of QA in prepaid settings. From the outset, the project was designed to involve the PACE sites as active partners. A general strategy of targeting issues that transcended usual medical care and focused on frail older persons was accepted at a meeting of PACE staff. The centerpiece of the design was the development of tracer conditions, for each of which specific positive and negative outcomes could be identified. The PACE staff were asked to suggest possible tracers. Their list was augmented by the project staff and outside consultants, and distributed to the PACE sites for further comment and refinement. A subsequent version with potential sequelae was again distributed. A final version with refinements was again circulated for review and comment. Finally, the total package was presented to the PACE programs at a national meeting.

Factors to Consider in Quality Assessment

Selecting appropriate indicators of the quality of care in the PACE model is difficult as there are many overlapping areas of quality concerns to consider. Areas of quality issues specific to the PACE project include capitated systems for an elderly population (e.g. underservice), the potential problems for overservice using the onsite day health model, the clinical problems unique to an elderly patient population, and the linking of acute and long-term care systems in an integrated service delivery model.

Underservice

Any capitated approach to financing provides incentives for underservice. The pressure to keep costs below the capitated payment amount introduces incentives to provide fewer services, particularly expensive inpatient hospital services. The majority of research on the effects of capitation with an elderly patient population is based on the experience of the Medicare Tax Equity and Fiscal Responsibility Act of 1982 (TEFRA)-risk health maintenance organizations (HMOs). Several areas of concern have been raised about the failure of the Medicare HMOs to provide for the comprehensive needs of a frail elderly population.

Evaluations of the Medicare HMO risk contracts have shown that Medicare HMOs decrease their costs by admitting fewer patients to hospitals, and by shortening the length of hospital stays for those they do admit (Langwell et al., 1987; Greenlick et al., 1984). Gillick (1987) expressed concern that HMOs serving a frail elderly population with poor functional status and at high risk for complications from illness may not be able to achieve the expected cost savings by decreasing hospitalizations. Many of Gillick's suggestions for refining the Medicare HMO service delivery system to meet the needs of a frail elderly population have been incorporated as an integral component of the PACE model: to expand the use of comprehensive geriatric assessment, to utilize case management services, and to provide more supportive home care and social services.

Despite concerns about the incentives for underservice in the Medicare risk contracts, several studies demonstrate that the quality of care in these programs, when measured in terms of patient satisfaction and the technical process of care, is equal to or better than traditional fee-for-service (FFS) patients (Rossiter et al., 1989; Retchin and Brown, 1990a; Retchin and Brown, 1990b; Retchin, 1991; Preston and Retchin, 1991; Carlisle et al., 1991). However, the favorable selection associated with Medicare HMOs may have contributed to the positive results of these studies.

Overservice

The onsite day health model used in the PACE program permits more extensive patient monitoring, and provides a climate conducive to overservice. There exists a potential risk to “medicalize” the process of care for the enrolled elderly population. The ease of access to physicians and other medical personnel makes it possible to treat every minor infraction or concern, and runs the risk of excess use of medication.

The onsite treatment model complicates assessment of overservice. One may want to look at how often a patient is evaluated by the doctor, and develop some standards or norms with which to compare these rates. Yet defining and quantifying the many informal contacts that occur throughout the day will be difficult and onerous. Alternatively, if no increased costs or untoward outcomes result from frequent contact, one may not need to worry about its occurrence.

Geriatrics

Siu, Brook, and Rubenstein (1986) cite several disadvantages to serving an elderly population in an HMO. They suggest that the HMOs' general lack of a geriatric focus may be detrimental to an elderly patient population. For example, they identify older patients who are “subtly sick,” that is, patients with minimal or atypical symptoms. In an HMO, where the focus is on diagnosis for a younger population, such patients may be misdiagnosed as being confused, weak, or just aging. They also suggest that in an HMO environment, the “non-complaining sick” (patients who do not report their illness for any number of reasons, including lack of transportation to the clinic, depression, or general lack of attention to the problem) are often overlooked. Again the PACE model is designed to overcome such concerns using the day health care model. The regular use of geriatric assessments to focus on the varied health and social needs of an elderly patient and the onsite clinical program provide the opportunity to monitor patient conditions in order to detect subtle differences in health status, functional condition, or general effect.

Geriatric care has two basic functions. The first function is in the area of prevention, including chronic care management with the goal of avoiding complications, or detecting problems at an early stage in development. Because prevention involves a front-end investment to detect and treat problems uncovered, it may cost more initially and may be efficient only when viewed over a longer period of time. The proof of the value of this investment lies in its ability to reduce subsequent service use. The second area of geriatric care lies in its role in times of acute care crisis and the decisionmaking capabilities based on information about the benefits and potential adverse consequences of alternative therapies and treatment strategies. For example, hospital stays might be shortened by more intensive attention to functional and management problems early in the episode of illness. Discharge planning may both be improved as well as expedited by such an approach. In both prevention and acute care treatment, diagnosis does not always tell the full story of the patient's condition. A broader functional approach is needed. Each of the common problems of geriatrics may have a variety of etiologies (Kane, Ouslander, and Abrass, 1989; American Medical Association Council on Scientific Affairs, 1990).

The value of geriatric assessment in the HMO context has not been established, and it will be difficult to assess the impact of geriatric assessment on patient outcomes in a quality assessment context. In an evaluation of geriatric assessment in a staff model HMO, patients referred to the geriatric team had more new diagnoses identified, medication changes recommended, psychosocial evaluations performed, and home and community services provided. However, there were no significant differences in outcomes at 3 months and at 1 year after the initial assessment as measured by hospitalization, mortality, and functioning (Epstein et al., 1990).

Geriatrics is not a conspicuous feature of even the TEFRA risk-based Medicare HMOs. Interviews with the medical directors of a national sample of these organizations revealed that one-half have no geriatrician, and the extent of any organized geriatric activity ranges from 58 percent with some type of general health information questionnaire to 12 percent with structured geriatric assessment forms (Friedman and Kane, 1993).

Integrated Service Delivery

The PACE service delivery model for acute and long-term care services in a multidisciplinary onsite service setting provides several advantages to initiating a quality assessment program. The long-term care channeling demonstration was designed to expand Medicaid payments to cover additional case management and home care services in an attempt to reduce institutionalization of elderly at risk of nursing home placement. The quality measures employed in this program were primarily structural ones directed at staffing needs and staffing concerns. Quality indicators included the type and level of staff training received, supervision, tardiness, absenteeism, and client complaints regarding staff (Kemper, 1990).

The social/health maintenance organization (S/HMO) demonstration is an attempt to provide comprehensive acute and long-term care services to Medicare patients under a prepaid capitated payment arrangement. An evaluation of the initial experiences of the S/HMO demonstration by the University of California, San Francisco, provides some insights into these programs (Newcomer, Harrington, and Friedlob, 1990; Harrington and Newcomer, 1991). Unlike the TEFRA HMOs, the S/HMOs limited the enrollment of high-risk patients, especially those at risk for institutional long-term care services, to a proportion equivalent to the community, and relied on active case management to mobilize resources and services. The orientation was on the organization and financing of care rather than on the specific aspects of clinical service delivery. The On Lok model being reproduced in the PACE projects represents a departure from the TEFRA risk HMOs and even from the S/HMO model. The focus on the frail elderly incorporating the day health model provides a unique opportunity to evaluate the effects of comprehensive geriatric care on patient outcomes.

Quality Assessment of Prepaid and Ambulatory Care

Quality assessment efforts in prepaid and ambulatory care have not been as well developed as they have in hospital settings. Nonetheless, some useful lessons can be learned from the efforts that have transpired to date.

HMO Studies

The RAND Corporation collaborated with 12 HMOs (the HMO Quality Care Consortium) as part of the national peer review organization (PRO) reform initiatives to meet congressionally mandated requirements for change in the structure of PRO activities to monitor care provided to Medicare beneficiaries. This consortium of HMOs was formed to test the reliability and validity of quality-of-care measures. RAND prepared several studies based on specific clinical conditions. The focus, however, limited the assessment of quality to the technical component of care in terms of appropriate performance and subsequent outcomes. This quality assessment process makes sense only when there are interventions with demonstrated efficacy for a particular health problem. Where there is a lack of clinical consensus or a lack of information on a particular intervention, no quality assessment can be forthcoming.

Similar to the RAND effort, three HMOs in Minnesota worked together along with the Minnesota Department of Health to develop and test an approach to quality assessment for the Medicare risk contracts (Solberg et al., 1990). The goal was to develop an explicit chart review screening system based on a list of indicators or sentinel screens that might be used by the local PRO as part of the new review for HMOs with Medicare risk contracts. Fifteen indicators were selected partly on the basis of available data, clear coding, and high enough frequency to provide a pool of data. An algorithm was developed through a consensus review process to identify cases that failed one or more of the quality categories. Cases that failed one of the categories were forwarded to a physician for an implicit review.

Medical Outcomes Study

The medical outcomes study (MOS) included more than 22,000 adult ambulatory care patients with chronic diseases such as diabetes, hypertension, arthritis, coronary artery disease, congestive heart failure, chronic lung problems, back problems, gastrointestinal disorders, angina, and depression, of whom about 2,300 were part of a longitudinal study of changes in functioning (Tarlov, Ware, and Greenfield, 1989). These patients included persons 65 years of age or over. Measures of outcomes covered both condition-specific measures and a more general measure of well-being. The well-being measure included physical functioning, role functioning, social functioning, mental health, health perceptions, and bodily pain. Those with chronic conditions frequently had poorer levels of well-being than those without chronic conditions (Stewart et al., 1989). The work emanating from this study provides a useful set of tools for examining the outcomes of care for adults, including older adults (Stewart and Ware, 1992).

Ambulatory Care Studies

The project to develop and evaluate a method to promote quality of ambulatory care (DEMPAQ) uses explicit criteria applied to office-based medical records to provide feedback about patterns of performance averaged over many instances of care, many patients, and many physicians. The quality assessment initiative involves two basic components: claims profiles and medical record review. The program will use Medicare claims data from the common working file. The medical record component involves review of the adequacy of documentation, and broad-based assessment of clinical performance for common diagnoses, conditions, test, treatment, and procedures. Personal computer software will be used to abstract information from photocopies of medical records.

The medical record review uses explicit, detailed, written review criteria that have been developed through an elaborate peer review and consultative process. These practice guidelines are transformed into review criteria that are then applied to the ambulatory clinical records of participating physicians. The intent of the program is to move away from the problem-oriented approach of traditional PRO review, and to focus on patterns of performance for a physician or group of physicians (DEMPAQ, 1991).

Claims-Based Data

The functions developed by the DEMPAQ program resemble those developed by Weiner et al. (1990) to specify 40 indicators to monitor and assess the quality of care of large populations, either an enrolled or insured population, using claims-based data. They defined specific measures of quality under each indicator to address quality of care concerns. The process to specify the list was based on consensus building among the authors. Information on quality of care indicators and measures was collected from the literature, and additional guidance was provided by clinical, quality assessment, and experts for guidance. Each measure was coded using three identifying features: (1) whether the item was either a process or outcome measure; (2) whether the item represented a sentinel event or rate-based indicator (used in reference to some reference point derived either empirically or normatively); and (3) whether the event represents a positive (desirable) event or a negative (undesirable) event.

QA Reform Initiative

Medicaid's commitment to prepaid care has generated the development of QA approach tailored to that audience. The quality assurance reform initiative (QARI) for Medicaid managed care, undertaken by HCFA's Medicaid Bureau, represents an effort to redirect QA for Medicaid managed care to create better accountability. As part of a strategy to displace the current requirement that Medicaid managed care programs enroll at least 25 percent of their clients outside Medicaid and Medicare, this program when implemented will have a direct bearing on PACE projects. At present, the program is still being developed, and will be tested in a few States the latter part of 1993. The focus of the QARI program is currently on the population that uses Aid to Families with Dependent Children (AFDC), but there is at least some recognition of the other groups covered by Medicaid.

QARI includes attention to structure, process, and outcomes of care. It calls for review of quality, both internally by the managed care program and externally by an independent agency. Certain studies are mandated (e.g., prenatal care and immunizations); others can be selected from a predetermined list. The studies include measures of process (largely adherence to appropriateness guidelines) and some indicators of outcomes. However, sample sizes may make the latter difficult in many instances.

Defining Quality of Care

The definition of quality will influence the objectives of the QA program and the type of criteria used to assess quality concerns. Quality assessment refers to the process involved in identifying problems or areas of inadequate quality, which, in turn, includes establishing performance criteria, standards or norms, and a process for detecting quality problems. QA subsumes quality assessment, and implies a program designed to address the quality deficiencies identified. The QA component includes recommendations for change, a strategy to implement the recommendations, and followup measures to indicate whether the problems have been addressed.

The literature contains many and varied definitions of quality. Brook and Lohr (1985) propose a definition of quality that compares the patient health status achievable in a given provider setting (effectiveness) with that achievable under ideal circumstances (efficacy). Donabedian (1988a) provides a more general definition of quality as the ability to achieve desirable objectives using legitimate means. Kane and Kane (1988) suggest that the definition of quality for long-term care includes both elements of care and the outcomes of care that are meaningful to the patient.

Perhaps the most comprehensive definition is that set forth by the Institute on Medicine's (1990) committee to design a strategy for quality review and assurance in Medicare:

“Quality of care is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with professional knowledge.”

The Institute of Medicine's definition of quality reflects the contemporary interest in patient outcomes, but acknowledges the need to attend to process, in terms of both patient activities and preferences, as well as the technical aspects of the practitioner's performance. In essence, quality can be considered as doing the right things well. Their approach also incorporates a conception of probable outcomes, recognizing that health outcomes may be represented by different distributions of results.

The definition of quality developed by the Institute of Medicine, which focuses on process and outcomes, can be adapted to the PACE quality assessment process. Their definition may be rewritten replacing the term “individuals and populations” with “disabled elderly persons and PACE enrollees” to focus the health outcomes requirement on an elderly patient and the monitoring of the technical components of care in the field of geriatrics.

The success of the On Lok model, upon which PACE was based, was partly because of the strength of the positive interaction between the patients and the providers of care. The PACE QA program must include specific elements of practitioner performance as well as the patient-provider relationships in terms of the assessment, diagnosis, and treatment decisions. The Institute of Medicine's approach also incorporates the assessment of outcomes in terms of broad health status measures such as quality-of-life, patient satisfaction, and well-being, so vital to the health of an elderly person who may already be living with a disabling or chronic condition; but the definition can also reflect outcomes relevant to the management of a specific condition. Another critical element of the definition involves the patient's role in decisionmaking, and suggests that the health outcomes pursued are those with which the patient agrees.

Quality Assessment

The quality assessment process includes operationalizing the definitions of the quality measures used, including the selection of criteria that indicate appropriate performance, the standards or expected levels of achievement, and a strategy for data collection and analysis.

Quality of care has traditionally been assessed according to the Donabedian formulation of structure, process, and outcome (Donabedian, 1988b). The structure of care refers to the setting in which care is provided, and includes material resources, human resources, and organizational structure. Examples of structure elements typically include required staffing levels and facility licensing requirements. The process of care includes the activities involved in the practitioner's assessment, diagnosis, and development of treatment recommendations. Process may also include the patient's activities in seeking care, receiving care, and following treatment recommendations. Outcomes refer to the effect of the care received on the health status of patients, and may be assessed in terms of observable changes in health status, or as measures of general patient satisfaction.

In specifying the goals or the “health outcomes” of the program, “health” must be defined, and the extent of the program's responsibility for maintaining health must be clearly delineated (Donabedian, 1983). Health outcomes for a PACE-type program are better specified and monitored at the individual patient level, and should be designed to examine the care of the patient in the context of the interaction of multiple problems. However, for an elderly population, specific health outcome goals are not always well defined. The complexities of the health problems of an elderly population make it difficult to define specific algorithms to address all the possible contingencies and comorbidities in terms of specifying process criteria or to control for all the variables that affect outcome measures. In addition, the PACE model relies on a multidisciplinary team of clinical and social service personnel with diverse backgrounds and expertise. The interdisciplinary approach to service delivery, in addition to the complex interactions of the clinical and social problems faced by an elderly population, provide a context conducive to the use of more global health outcomes that do not rely on specific algorithms. The global measures useful with an elderly population include patient satisfaction, general health and well-being, levels of functioning, and self-perceived health status.

Because it is difficult to rely solely on global health outcomes, the QA program should also include more traditional indicators of quality problems and condition-specific tracers. Indicators are quantitative measures that can be used to monitor and evaluate the quality of patient care (Joint Commission of Healthcare Organizations, 1989). Examples of past indicators of quality of care include such indexes as mortality rates and hospital readmission rates. However, these and other traditional indexes do not work well with an elderly population to determine quality of care because of the prognosis and condition of a medically frail and often disabled population (Kane and Kane, 1988).

Tracers represent a condition whose management and outcome of care illustrate the general pattern of quality provided (Kessner, Kalie, and Singer, 1973). Specific conditions are identified along with positive and negative process and outcome criteria that indicate what constitutes good or bad care (or outcomes). The use of tracers is well suited to the PACE program as it provides an opportunity to focus on a limited number of conditions, and can be specifically targeted to the needs of a frail elderly patient population. Tracers have been successfully used in other geriatric studies. In a study of the quality of ambulatory care for the elderly, Heller et al. (1986) used data from the National Health Nutrition Examination Survey to look at five conditions (i.e., tracers) selected to reflect the general quality of ambulatory care. These five conditions included angina-type chest pain, dyspnea on mild exertion, hypertension, hearing loss, and depression. The quality of care was defined in terms of the technical components of care based on a predetermined set of minimal criteria of care. The approach to quality in this study also included a patient-level process component by looking at barriers to seeking treatment when specified conditions were present, and also controlling for intervening factors such as race, income, and urban-rural differentials in quality.

The tracer approach can be extended to identify certain events whose very existence represents a sign of unsatisfactory care. Rutstein et al. (1976) called these “sentinel events” after the epidemiological term referring to cases that foretold the beginnings of an epidemic. Examples of sentinel events could include deaths from neglect (e.g., malnutrition), new decubiti, unanticipated deaths (or perhaps a higher than usual rate of these), family or patient complaints about problems of access to expensive care, and disenrollments.

Each of these incidents would have to be investigated individually and a determination made of the extent to which it constituted an indictment of the care being given. This sort of generalization from a single case can be very misleading, especially because it relies so heavily on implicit judgment and often major assumptions about what was done and why.

The Pace Application

The proposed PACE quality assessment uses both a client-centered and a problem-centered approach to quality assessment. A client-centered approach will look at individual patient-level information to see if there are deviations from predetermined goals or courses of action. We refer to this process as patient goal attainment. We also propose to use more general measures of patient outcome, such as patient satisfaction, general well-being, and level of functioning, to reflect the effects of various problems. The problem-centered approach focuses on how a specific condition is managed. Tracers are often used in a problem-centered approach to help identify conditions or diagnosis for more indepth review. The patient-centered approach is used to monitor the individual “health” of the patients. The problem-centered approach monitors quality issues associated with the service delivery system or technical processes of care.

General health measures transcend individual conditions or problems, and can be looked at as reflecting the sum of all a person's problems. Measures like these are especially appropriate to a patient-centered approach. They include items that address quality of life and function. Specifically, they cover physiologic health (e.g., normal blood pressure, blood sugar levels), pain and discomfort, activities of daily living (ADLs), psychological well-being, social participation, and satisfaction with care.

The client-centered approach can be implemented in two ways. The general health status items can be used to create one or more outcome scores which form the dependent variables for predictive equations across all PACE sites. The predictive models estimate the separate effects of patient characteristics and treatment. The former include both clinical variables, such as diagnosis, duration of problem, trajectory of problem, and severity and social variables, such as age and social support. The coefficients from these predictive equations are then used to calculate expected values for each client. The expected rates are then compared with the observed rates for each person at each site to create a measure of overall success at that site relative to all sites. This provides, in essence, an adjusted measure of performance for a given site. An even more client-centered approach uses a form of goal attainment in which the interdisciplinary teams establish individualized goals (expressed in function status terms that correspond to the measures available) for each client. The extent to which the goal is attained is compared with the results from the first approach to see whether there is a tendency to underestimate goals in order to improve the risk of achieving them. Likewise, the goals set can be directly compared with expected values for each client generated, as previously noted.

In contrast, the condition-specific approach uses tracers to examine how critical steps in the care of specific clinical problems are managed and what outcomes result. The tracers proposed for PACE are shown in Table 1. A number of aspects of care can be examined, including both the process of care and the outcomes, or sequelae, that result. In some cases, the dealing with the problem begins with its presentation. In other circumstances, the rate at which a problem is detected and the steps taken to prevent it are relevant.

Table 1

Potential PACE Tracers

Problem	Preventing and Detecting	Treating1	Positive Sequelae	Negative Sequelae
Diabetes	Periodic screening.	Fasting blood sugar checks (finger sticks).Diet and drugs.Foot care.Retinal exam.	At least moderate blood sugar control.	Undetected cases.Hypoglycemia.Hyperosmotic, non-ketotic coma.Dehydration.Skin ulcers, gangrene.End organ damage (neuropathy, nephropathy, retinopathy, cataract).
Hypertension	Periodic screening.	Blood pressure checks.Diet and drugs.Retinal exam.	Blood pressure control (systolic less than 160, diastolic less than 90 for those under 80 years of age).	Undetected cases.Falls (orthostatic hypotension).Stroke, transient ischemic attack (TIA), myocardial infarction.Electrolyte Imbalance.2Depression.2Confusion.2Constipation.2
Urinary Incontinence	Periodic inquiry.Evaluation of symptoms.	Diagnosis.Kegel exercises.Timed toileting.Behavior modification.Drugs.	Fewer episodes.Dry.Active.Socially engaged.	Undetected cases.Urinary tract infections.Isolation.Decubiti.Premature catheterization.
Foot Problems	Questions regarding pain. Observed walking.	Foot care.	Walking.	Foot pain.Immobility.Instability, falls.Ingrown toenails.Infections.Ulcers.
Depression	Routine questions (depression scale).Evaluation if positive.	Diagnosis.Psychotherapy or anti-depressant trial.	Improved appetite.More active.Improved attitude.Improved cognition.	Isolation.Sleep disturbance.Lassitude.Suicide.
Disruptive Behavior		Other approaches tried before using psychoactive drugs or restraints.Behavioral therapy.		Falls.Wandering.Tardive dyskenesia.2Impaired cognition.2Paradoxical agitation.2
Constipation	Periodic inquiry.	No excessive laxative use (use in absence of symptoms).	Regular bowel movements.	Fecal impaction.Bowel obstruction.Fecal incontinence.
Malnutrition	Periodic weighing.	Dietary advice.Active weight reduction program.Nutritional supplements if needed.	Body mass index within 20 percent of ideal body weight.	Obesity.Cachexia.Low albumin.Poor wound healing.Decubiti.
Adverse Drug Effects	Periodic medication review.Updated listing of all drugs (prescribed and over-the-counter).	Limited number of drugs.Pharmacist review for cases of more than 5 drugs taken daily.Computerized drug interaction detection system.Drug review before each new prescription.	Potential drug interactions.3	—
Congestive Heart Failure	—	Documented need for digoxin, levels monitored.Electrolytes monitored.Weight monitored.	Improved exercise tolerance.Less shortness of breath (SOB).	Edema, weight gain.SOB, dyspnea.Hypokalemia.2Hospitalization.Isolation.Immobility.Depression.2
Arthritis	—	Cautious use of non-steroid anti-inflammatory drugs (NSAIDs).Gastrointestinal (GI) protection if NSAIDS used.	Physically active.Socially engaged.	Pain, discomfort.Poor ADLs.GI bleeding.2Isolation.Depression.2Immobility.Instability, falls.
Dementia	Periodic screening (uniform instrument, e.g., minimental status examination)	Evaluation to exclude treatable causes (protocol?)Diagnosis.Caregiver support.No restraint use.No psychoactive drugs.Advance directives.	—	Agitation.Rage reaction.Falls.2Premature placement.
Parkinson's Disease	—	Monitored drug use.	Physically active.Socially engaged.Clear speech.Active intellect.	Tremor.Immobility.Slurred speech.Confusion.Depression.Instability.
Stroke	Anticoagulation after TIA (aspirin) (depending on etiology).Modify risk factors (smoking, hypertension).	Rehabilitation plan carried out.Home modification.	ADLs, IADLs. Speech and understanding.Social interaction.	Depression.Social isolation.Immobility.Slurred speech.Contractures.
Hip Fracture	Osteoporosis prevention in women (estrogens, exercise, calcium supplementation).Home safety check.	Rehabilitation plan carried out.Home modification. Assistive devices.	Ambulation.Functioning independently.	Bed bound.Contractures.
Falls	—	No restraint use.	Walking safely with assistive devices.Transfers with minimal or no assistance.	Hip fractures or dislocation.Bruises.Subdural hematomas.Immobility from fear of falling.
Patient-directed Care Goals	—	Clearly documented patient and family wishes.Evidence of: Discussing types of care available and outcomes of each. Eliciting preferences for outcomes. Advance directives obtained by persons outside the care providing organization.	Patient wishes followed.Pattern of decisions that reflects both giving and withholding care.	Pressure to forego care.Excessive care.Client preferences not considered.
Case Management	—	Client priorities obtained.Comprehensive assessment of problems.Appropriate actions taken.Patient course monitored.	Client satisfied.	Provider clearly in charge.Client preferences ignored.No follow through on plan.No knowledge of outcomes.

We have not yet attempted to develop specific criteria for elements of treatment. Definitions of what constitutes adequate care will be needed. As noted in the text, we recommend that these criteria be established in conjunction with the PACE sites.

Drug side effects.

Potential drug interactions include oral anticoagulants/salicylates; oral anticoagulants/oxy-phenylbutazone; oral anticoagulants/disulfiram: Cimetid; oral anticoagulants/antithyroid dugs; oral anticoagulants/thyroid; haloperidol/methyldopa; levodopa/phenothiazines; lithium/thiazide diuretics; digitalis/kaluretics; theophyillines.cimetidine; oral antidiabetics/salicylates; guanethidine/tricyclic antidepressants; quinideine/cimetidine; chlorpromazine/metoprolol, propranalol; aminoglycosides/loop diurietics; indomethacine/furosemide; furosemide/clofibrate; oral antidiabetics/oxy-phenylbutazone; indomethicin/lithium; clonidine/tricyclic antidepressants; thyroid/cholestyramine; digoxin/quinidine: quinine:verapamil; indomethicin/corticosteroids. (Ahem, et al.: Medicine, Health and Aging: Reducing the Risk of Prescription Drug Interactions, and Estimating Risk Prevalence of Prescription Drug Interactions in Pennsylvania's Pharmaceutical Assistance Contract for the Elderly (PACE) program, 1987).

NOTES: PACE is the Program of All-inclusive Care for the Elderly. ADLs are activities of daily living. IADLs are instrumental ADLs. TIA is transient ischemic attack.

SOURCE: Kane, R.L., and Blewett, L.A., University of Minnesota, 1993.

Table 1 outlines the general strategies to be used, but much more detail is needed about the specific criteria and standards to be used. For example, what is meant by hypertension or control of blood sugar levels? In some cases, rather than a specific level, a more complex algorithm may be needed. For example, instead of simply saying that a female should have a pap smear every 3 years, one might propose that those with two sequential negative smears can go for much longer intervals without a test. Or rather than saying that urinary catheters should not be used, one might allow them after other efforts have failed. Standards represent the expected rate of achieving the criteria. One can establish arbitrary standards, but it seems much better to use actual performance as a basis for comparison. Ideally, one would like to know the standards achieved in the FFS sector, but such information is hard to come by. Some may become available from the evaluation of the PACE program, but likely many areas will not be covered by that evaluation. One can then either compare the performance of one PACE site with the rest, or establish the levels arbitrarily. Because at least one purpose of this exercise is to establish a system that can be used for regulation at the State level, some degree of fixed standards that can be applied to PACE projects and other ventures seems desirable.

It thus appears that at least some degree of normative performance levels will have to be established for both criteria and standards. This step is best taken in conjunction with representatives from the PACE sites. Some combination of mailed feedback and a meeting to resolve particularly knotty problems where there is immediate consensus seems the best course.

Another distinction in a quality assessment program is to determine whether implicit or explicit review criteria are used. Implicit review is based on physicians' (or other health professionals') professional judgment, and is usually conducted on a retrospective case-by-case basis. Explicit review lays out the specific tasks or technical components of the process of care that a physician should follow, given patients with certain clinical conditions. Although much of the pioneer work in quality assessment is focused on laying out explicit criteria to direct and monitor the technical process of care, most clinicians find it difficult to specify just what should be done under which circumstances. Physicians are more comfortable reviewing a case and judging whether its management made good clinical sense as opposed to specifying a priori the appropriate course of applied medicine. In this context, medicine is commonly viewed as more of an art than a science. Thus, most quality assessment programs have in the past relied on implicit professional judgment (perhaps protected by a screen based on explicit criteria). A common protocol finds nurse reviewers screening cases, using more or less explicit criteria for good care. Any problems or deviations from the standard criteria are then referred to a physician judge for implicit review. Implicit judgments may be more professionally satisfying because they allow tor the nuances of a case to be represented, but they may not be as reliable as those using explicitly defined criteria. As in most quality assessment programs, it is recommended that the programs developed for PACE use both implicit and explicit criteria. However, explicit criteria will be used whenever possible.

The rate at which the criteria are achieved is called the standard of care. In general there are two sources of information used to establish standards of care: (1) empirical information about the rates in other groups associated with similar types of patients (e.g., control groups), and (2) general expectations about what level of performance seems reasonable, usually called normative standards. Program evaluation represents a special case whereby the treatment group would be compared with the control group, which serves as the baseline against which to make comparisons.

Table 2 specifically addresses areas where some level of preventive action is appropriate for study in the PACE context. Given the small numbers of persons observed, it is highly unlikely that any outcomes which reflected the extent to which preventive actions were followed (with the possible exception of pneumonias) would approach statistical significance in any study period.

Table 2

Proposed PACE Prevention Tracers

Problem	Action	Frequency	Outcome
Influenza	Immunization.	Yearly.	Influenza.Pneumonia.Hospitalization.Death.
Pneumococcal Pneumonia	Immunization.	Every 6 years.	Pneumonia.
Cervical Cancer	Pap smear.	Every 3 years (until 3 negatives).	Early detection.Death from cervical cancer.
Breast Cancer	Breast examination.Mammography.	Yearly.Every 2 years until age 75.	Early detection.Death from breast cancer.
Colon Cancer	Fecal blood.	Yearly for high-risk patients.	Early detection.Death from colon cancer.
Vision	Near and far vision.Visual fields.	Yearly.Yearly.	Corrected vision.Detection and treatment of glaucoma.Prevention of blindness.
Hearing	Audiology examination (need not be a formal full tone testing exam).	Every 2 years.	Corrected hearing.
Exercise	Activity compatible with health status.Moderate walking.Stretching.	Several times a week.	Improved vitality.More positive attitude.More stamina.

NOTE: PACE is the Program of All-inclusive Care for the Elderly.

SOURCE: Kane, R.L., and Blewett, LA., University of Minnesota, 1993.

Although some general health information can be found in the medical record or in DataPACE (the uniform data collection system used by all PACE sites), some of the information addressing general health measures will have to come from special patient surveys. These surveys will be used primarily to assess patients' satisfaction with the care and their perception of access, continuity, and their own health status. The surveys should be done either through a telephone interview or, preferably, through an annual mailed questionnaire. In the former, the interviews should be conducted through a third party, as clients are not likely to give providers honest answers about the care received. Items such as depression, restraint use, and autonomy are more likely to be found in the team assessment reports. More clearly defined items such as hospitalization and death will be indicated in both the medical record and the administrative data.

The proposed design emphasizes process and outcome and ignores structure, although inevitably there will be some demand for the latter, even in an innovative system. The strong preference for outcomes is counterbalanced by some salient concerns about the size of the groups enrolled in the PACE sites. With enrollments of 200 or fewer, even if composed of impaired individuals, the probability of encountering many of the most salient outcomes, once distributed across various problems, is limited. For example, an adverse outcome that occurred in 20 percent of cases per year (an extremely high rate) would only occur 10 times in 50 cases. Finding a difference of 20 percent in rate (a sizable difference) would mean distinguishing between rates of 10 and 12 cases. Allowing for some natural variance, say even 10 percent, this distinction would require more than 1,500 observations. Thus, although we recommend using outcomes as an integral and even emphasized component of the PACE QA program, some process measures will also be needed.

Special Issues

A related concern has to do with the use of advance directives. The PACE projects place strong emphasis on their commitment to respecting patient wishes, and are likely to be more aggressive than standard practice in obtaining and documenting advance directives. Our consultants have suggested that wherever an advance directive indicates a patient preference to forego an indicated procedure or treatment, any pertinent algorithms should be voided for that case. Such a step should greatly enhance a PACE site's process scores. Indeed, one of the proposed process criteria addresses the program's sensitivy to and conscientiousness in obtaining advance directives. However, because it is a prepaid system, albeit nobly motivated, there is an incentive to encourage zelous attention to those advance directives that avoid extensive and expensive care. We have struggled with how to avoid the dilemma of rewarding the PACE sites simply for doing less. We have modified the criteria for decisionmaking, but still see this area as one needing more attention.

Several sites raised concerns about the potential recordkeeping burden such a system would entail. We have tried to minimize the recordkeeping requirements. Nor should the items, especially the sequelae, be things that would not be routinely recorded. Indeed, we do not want to penalize good record keeping by unduly penalizing the mention of adverse events. The elements proposed should be found in minimally adequate medical records, in the administrative files, or in DataPACE. The burden of abstracting relevant information would rest with the contractor.

Implementation Plan

The PACE sites are sufficiently different in their organizational structure that flexibility is necessary in a QA program to permit individual sites to focus on their own specific internal needs. However, the types of information collected for common external review requirements, such as those required for program evaluation and Medicare PRO review, must be collected in a uniform way across sites. Although consistent external review criteria should be used across sites, flexibility must be built into the system so that individual sites can add components to the system to meet additional State-mandated external review requirements or additional internal review criteria unique to that site.

The QA program would be best conducted by a single organization responsible for all PACE sites in order to have the opportunity to pool data and provide inter-site comparisons. The organization would be responsible for all data collection, but the sites would be expected to provide access to charts and patients. The sites would be expected to maintain an information system capable of identifying patients according to designated characteristics. For example, the site should be able to provide the contractor with the names of all patients who exhibited any of the tracers noted in Table 1. (Some tracers may not fit this specification. For example, advance directives apply to all.) Likewise, the site should be able to supply lists of patients who have been hospitalized, who died, who disenrolled, or who received one or more of a list of specific drugs within the past year.

Table 3 summarizes the general categories of measures that are proposed for PACE tracers and for the patient-centered aspects of care, and suggests the types of indicators that might be used for each and the data sources to be tapped. Surveys play a large role. Especially for the patient-centered items, much of the data are usually obtained by this means. They are also needed to verify detection rates. Because surveys are expensive, thought has been given to alternative approaches, such as incorporating important information elements into the information collected as part of the mandated reporting system, DataPACE.

Table 3

Basic Quality Measures and Their Potential Sources

Measure	Indicator	Data Source
Prevention and Detection	Rate of occurrence.Rate of detection.	Survey and record review.
Treating	Adherence to rules and algorithms.	Record review.
Positive Sequelae	—	Survey.DataPACE.
Negative Sequelae	Hospitalizations.Deaths.Complications.	Survey.DataPACE.Hospital records.
Patient Satisfaction	Perceived access.Time spent with practitioners.Convenience.	Survey.
Family Satisfaction	Family stress.Caregiver burden.	Survey.
Functional Status	ADLs and IADLs.Pain and discomfort.	Survey.DataPACE.
General Well-Being	Perceived-health status.	Survey.DataPACE.
Autonomy	Advance directives made and honored.	Survey.Record review.
Psychological Well-Being	Depression.Self-worth.	Survey.DataPACE.
Social Activity	Role performance.	Survey.DataPACE.

NOTES: PACE is the Program of All-inclusive Care for the Elderly. ADLs are activities of daily living. IADLs are instrumental ADLs.

SOURCE: Kane, R.L., and Blewett, L.A., University of Minnesota, 1993.

Record Review

The review organization would then review the charts of the designated patients to look for evidence of either appropriate treatment, appropriate preventive activities, or positive and negative sequelae. In some cases, the contractor may need to question the patient directly to ascertain the presence of specific sequelae. This would be done in conjunction with the survey component of the review.

In some cases, in order to contain costs, the contractor may need to sample cases for the record review. Where there are large numbers of cases meeting one of the identifying conditions, the contractor would sample from within those cases. Cases representing sentinel events would all be reviewed. The contractor would report both the results of the chart review and the frequency of the cases deemed eligible for review in each category. It may be determined that not all triggers would be covered at each review, but that decision would not be announced until all cases were identified. At a minimal, the number of cases eligible for review would be recorded, even if no cases in that trigger were reviewed at that visit.

The record reviews would use criteria established in conjunction with representatives of the PACE sites. Levels of achieving the criteria would be recorded and compared with two types of standards: (1) normative performance standards developed in conjunction with the PACE site representatives, and (2) empirical standards developed by comparing the rates at an individual site with the aggregate rates from all sites, after appropriate corrections for case mix.

Patient and Family Surveys

Certain information needs to be collected directly from patients and their families. A random sample of patients would be selected by the contractor from the general roster of patients provided by the site. The size of this sample would be determined in part by the number of patients needed to be contacted in following up on the chart review information. Each of the patients selected by either means would be interviewed in private by the contractor. The information queried would cover issues of general function and satisfaction. Most of the interviews should occur at the day care site in order to minimize the costs; however, some home visits may be necessary to avoid sampling biases against those who do not attend day care regularly.

The function questions would include patient self-reports of ADLs and instrumental activites of daily living (IADLs), specific questions about levels of pain and discomfort (intensity, frequency, relief), perceived health status, a brief depression scale, and items about level of social participation and meaningful human contact. In those instances where the patients are unable to provide the information themselves, a proxy will be used. The proxy may be a family member who has the opportunity to make pertinent observations or, where such a person is unavailable, a staff member from the site.

In addition, the patients would be asked specific questions about their satisfaction with the medical care they had received, their sense of the thoroughness demonstrated and time spent with them, any experiences in which they felt they did not get care they thought they needed, access to care, attitudes and behaviors of the staff, specific satisfaction with program elements (e.g., day care), and general satisfaction with the program. Specific questions would address the ways in which advance directives had been established and the patients' satisfaction with the process and understanding of what had been decided. Patients unable to answer these questions would not have proxies respond in their stead.

Family members of the patients sampled (including those who may not be able to respond to the satisfaction questions) can be interviewed by telephone to ascertain their satisfaction with the care being delivered. These interviews will address their perceptions of the adequacy of the care, the level of burden they feel for providing care, instances where they believe that adequate care was not provided, or problems they have had in accessing care, attitudes of staff, and convenience of the services (e.g., hours, locations). Family members too will be asked about the way advance directives were established and their role in that process.

The survey data can be used in several ways. For example, undetected depression (on the basis of the screening questions) would constitute a detection problem. (It is feasible but not recommended for the initial stages that similar screening be done for cognitive dysfunction because there is less to be done about it.) ADL and IADL scores would be compared with recorded scores in the record of the DataPACE files to check for accuracy. Satisfaction scores would be subjected to both the normative and empirical standards previously described.

Responses to Deficiencies

Ideally, one would like to track the overall function of all patients. To the extent that accurate data are maintained in DataPACE on ADLs and IADLs, and perhaps depression and cognition, it is possible to compare the progress of adjusted groups of patients across the sites. There is, at present, no data base on which to base normative standards for expected progress. This tracking of outcomes is probably best considered exploratory for the moment, and would be undertaken only if the quality of the data warrants it. We do not recommend that survey data on all patients be collected for this purpose, because the cost would not be justified.

The information from these reviews would be shared with each site, which would be expected to develop appropriate interventions to respond to deficiencies. Those areas of deficiency would be specifically examined in additional modules at the next review. That is, the sites would undergo an expanded review to cover those areas and the regular review in order not to unwittingly reward the sites for poor performance. We have not tackled the question of what levels of poor performance would justify stronger interventions.

The other area of concern, expressed by both quality assessors and clinicians, has to do with the infrastructure for assessing quality. The demand for more detail comes as no surprise, but it is premature to talk about how many people should be on a committee, how many records should be reviewed, or how often the reviews should be done. Nonetheless, the concerns about how much training the reviewers will have (a code phrase for concerns about their ability to integrate multiple clinical elements) and the record-keeping burden that may be imposed are worth attending even now. Ideally, much of this data could be obtained unobtrusively. If the items are accepted as good practice, many can be incorporated into record systems, preferably computerized systems, even a next version of Data-PACE. Some data, such as the rates of missed problems and consumer perceptions, will have to come from special studies, no matter how good the records are.

Plans for QA in PACE should be designed to use as much of the QARI methodology as possible, but it will be important to recognize that PACE's concentration on frail older persons will require at least some special measures and perhaps some special techniques as well. At present many of the areas of emphasis in the recommendations for the PACE QA program are not found in QARI.

As noted earlier, the PACE projects are unique in addressing the acute and chronic problems of frail older persons in an integrated fashion. However, there are other programs that target the same groups at varying levels of integration, and more efforts like these will undoubtedly follow in response to demographic pressures. The approach to QA described in this article has applications to these other efforts to integrate the care of frail older persons. The combination of acute and long-term care is, in effect, intended to produce results that are greater than the sum of the component parts. It is thus inappropriate to rely on discipline-specific strategies to assess the impact of such care. Nor are traditional process measures of quality likely to identify the larger effects. The heart of this approach is a combination of patient-centered and problem-focused techniques that are sensitive to the effects of integration, and permit various combinations of services to be delivered in innovative and flexible ways.

Capitated programs of care like PACE present special opportunities and special challenges. It is not enough to look at the performance of care under this approach and compare it with more conventional FFS medicine (Bernstein et al., 1993). Capitated care offers the opportunity to ask more basic questions about whether access to care is sufficient. Because one can identify a denominator, it is possible to look at the rates at which problems are addressed and the effects on the health status of entire groups.

As the focus of care moves increasingly out of the hospital setting, one of the challenges to be faced is to develop ways to assess episodes of care. Most of the current work in assessing the outcomes of care begins with an event, usually a treatment. It would be better to begin with a diagnosis, and even better to trace the care back to a problem. The ability to follow the course of care from the onset of a clinical problem, to see how well it was diagnosed, what treatment was rendered, and what outcomes resulted will eventually provide the data base needed to understand better just what kinds of care work for what sort of problems. It will also permit pinpointing where in the course of care things went wrong. Such a comprehensive episode-based approach will not come quickly or easily, but it is a goal worth striving to attain (Bernstein et al., 1993).

As more attention is directed to finding efficient ways to address this growing segment of society, QA strategies like those proposed will become more important. The emphasis on outcomes permits more degrees of freedom in designing interventions and encourages creativity, but the outcome focus means that problems uncovered will not be immediately translatable into solutions. Implementing remedial programs will require an interdisciplinary spirit, which looks at the patient first and foremost. Whether this level of flexibility will satisfy the current demand for regulatory specificity remains to be determined.