Quality of care in teaching nursing homes: findings and implications.

This article explores policy implications and selected methodological topics relating to long-term care (LTC) quality. We first discuss the Teaching Nursing Home Program (TNHP), in which quality of care in teaching nursing homes (TNHs) was found to be superior to the quality of care in comparison nursing homes (CNHs). A combination of outcome and process/structural measures was used to evaluate the effects of care and underlying reasons for superior TNH outcomes. Second, we explore policy and analytic ramifications. Conceptual, methodological, and applied issues in measuring and improving the quality of LTC are discussed in the context of TNH research and related research in home care.

Introduction

In evaluating LTC to assess or improve quality, it can be useful to examine general outcomes that pertain to all or most patients to determine whether care has produced overall benefits. By conducting statistical and clinical analyses of more specific outcomes as well as structure and processes of care, it is possible to shed some light on how and why the general outcomes were or were not attained. In addition, the more specific outcome findings (as well as the process and structural findings) can provide useful information in their own right, particularly if the measures were selected so that particular patterns of results using a combination of measures support or refute prespecified assumptions or hypotheses. The first section of this article provides an example of this approach to LTC quality evaluation through a discussion of the methodological approaches and findings of an evaluation of the TNH demonstration. The second section examines the policy implications and selected methodologic implications of the evaluation of the TNH demonstration, also drawing from subsequent work on the quality of LTC provided by home health agencies.

TNH Evaluation Findings

Teaching Nursing Home Program

The approximately $7 million RWJF TNHP was a demonstration project to assess the feasibility of improving nursing home care by establishing affiliations between 11 schools of nursing and nursing homes in 8 States and the District of Columbia. The demonstration was administered by the University of Pennsylvania through RWJF grants to the participating schools of nursing from 1982-88. TNHP data pertaining to 1981-88 were collected and processed from 1984-92. Details on the TNHP are available elsewhere (Aiken et al., 1985; Anderson and DeVore, 1985; Mezey, Lynaugh, and Cartier, 1988, 1989; Mezey and Lynaugh, 1991). The program was designed as a feasibility study to ascertain whether it is possible to improve nursing home care through academic nursing affiliations.

Each participating nursing school selected a nursing home with which they established an affiliation. Affiliation agreements took various forms, resulting in targeted patient-care programs, broad educational programs, staffing changes, and nursing home research. The contents of the targeted patient-care programs resulting from the affiliation varied across sites, but all focused on similar outcomes: reducing hospitalization and increasing discharge to the community. Many also focused on specific problems, such as preventing falls, improving treatment of bladder and bowel incontinence, reducing the incidence of decubitus ulcers, monitoring drug usage, and enhancing exercise and movement programs to improve functioning, self-care, and interaction with others. Educational programs, such as team nursing and training geriatric nurse clinicians (master's- or doctoral-prepared clinical nurses with an emphasis in geriatrics/gerontology), were often focused on the targeted patient-care programs, emphasizing comprehensive assessment of patient problems, improved care planning, and patient monitoring. Staffing changes resulted from involvement of nurse clinicians with specialized training in geriatrics and geropsychiatry and from participation of students in patient-care activities. Nurses' aides and licensed nursing home staff received clinical teaching and in-service training, especially in patient assessment and care planning.

The evaluation reported here was targeted exclusively at effects on nursing home care, not effects related to schools of nursing. The fundamental purpose of the evaluation was to assess the effectiveness of the TNH approach as a means of improving the quality of nursing home care. Six of the TNH sites were examined in detail. Although all six were non-proprietary, the nature of the findings bears promise for LTC patients in a wider array of nursing home and LTC settings. The quality-of-care issues studied in the evaluation of the TNHP included hospitalization, health status outcomes, psychotropic medication use, restraint use, and maintenance care—several of which were emphasized in the Nursing Home Reform Act enacted in the Omnibus Budget Reconciliation Act of 1987 as well as in recent studies of nursing home quality (Garrard et al., 1991; Tinetti et al., 1991; Colling et al., 1992; Avorn et al., 1989; Beers et al., 1988; Werner et al., 1989; Schnelle et al., 1989, Hu et al., 1989; Ray et al., 1993; Harrington et al., 1992).

Methods and Data

Evaluation Sites

Six matched CNHs were selected for the evaluation to augment TNH before-after findings with study-comparison findings, since retrospectively obtained baseline data were judged insufficient to draw conclusive inferences. A random sample of CNHs was not employed because of the likelihood of substantial differences between study TNHs and randomly selected CNHs. Instead, six comparison facilities were chosen so that their predemonstration (1981) profile on selected characteristics was as similar as possible to the TNHs before the TNHP was implemented. Several hundred nursing homes passed the initial screen as candidates for CNHs. The final six CNHs were chosen from the same States as the TNHs using a statistical algorithm designed to simultaneously minimize differences (by minimizing the k-dimensional distance) between the TNH and CNH profiles on several matching variables that included number of beds, occupancy rate, length of stay, percentage of Medicare days, percentage of Medicaid days, hospital-based versus freestanding status, and ownership (government, nonprofit/non-government, and proprietary).

Study Design

Random assignment of patients to TNHs (versus CNHs) was not possible, necessitating use of comparative analyses. The study was designed so that (before-after) changes in hospitalization and community discharge rates for TNH patients could be compared with changes in these rates for CNH patients to assess whether similar trends existed for both populations. Analyses were also conducted to assess differences in more specific outcome and process measures between TNHs and CNHs during the intervention period (i.e., the TNHP) in order to determine whether specific outcome or service-related findings might explain or refute initial inferences following the before-after comparisons. Thus, the outcome and process quality analyses were designed to provide additional information on how hospitalization rates may have been lowered for TNH patients by examining those processes of care deemed on clinical grounds to be important in affecting patient outcomes.

The intent in the comparative analyses was to seek patterns of findings that would be mutually supportive and consistent across various domains of measures and various types of comparisons. Since randomization was not feasible, we analyzed measures using statistical methods designed to adjust for pre-intervention differences and before-after trends that may have occurred independently of program effects.

The unit of observation for the analyses reported in this study is the individual patient. Comparisons are made between TNH patients as a group and CNH patients as a group. The limited number of nursing homes included raised logical and statistical concerns about the analysis of patient-level data. However, the TNHP could not be regarded as a single, uniform treatment. While a few selected characteristics were common to all or most sites (nurse clinicians in particular), the number of different TNHP treatments were almost as numerous as the number of patients, owing to the variations in targeted programs, faculty involvement, staff responsibilities, types of in-services, student rotations, team compositions, clinical approaches, and staff training. Within-site variations were as substantial as among-site variations in many instances. Hence, the patient was used as the unit of analysis. For comparative purposes, therefore, it was appropriate to evaluate TNH patient outcomes relative to CNH patient outcomes. Further, we examined results on a facility-by-facility basis (in addition to the overall patient-level results) to determine whether the rankings of facilities in terms of patient-level means paralleled the patient results. When both types of analyses yielded similar patterns of results, the inferences that could be drawn were on firmer footing because both patient- and facility-level trends were in the same direction (this is further addressed in the later section on analysis methods and variables).

Data were collected on patient-specific risk factors, selected patient status changes, and services provided. All patient-level analyses were restricted to individuals 60 years of age or over. The results for selected performance measures not included in this article are reported elsewhere (Shaughnessy and Kramer, 1989; Shaughnessy, Kramer, and Hittle, 1991). Those reported here are the most consequential in terms of methodological and policy implications.

Samples and Data Collection

Data from cross-sectional and admission samples were analyzed separately because cross-sectional samples of nursing home patients typically contain higher proportions of long-stay patients who require chronic care. Conversely, admission samples are characterized by higher proportions of short-stay patients with greater rehabilitation potential than chronic-care patients.

Pre-intervention data were collected respectively from patient medical records maintained by the nursing homes on patient status, discharge, and hospitalization for: (1) a random admission sample (n = 325) and (2) a cross-sectional random sample (n = 342) of patients in each TNH and CNH. Per facility, there were approximately 25-30 patients in each of the retrospective admission and retrospective cross-sectional samples. Data pertained to the interval from the point of initial data collection to 1 year later or discharge, whichever occurred first.

After or intervention-period data were collected prospectively from 1984-85 to the end of the demonstration in 1988 by nurses trained onsite using the data-collection protocol implemented for the evaluation. Intervention-period data comprised a richer data base of functional, physiologic, diagnostic, and demographic measures than was available from the retrospectively collected preintervention data. Prospective data included longitudinal information for admission (n = 2,649) and cross-sectional (n = 896) random samples of approximately 220 and 75 patients, respectively, per facility. Data were collected at admission or cross-sectional sample entry for each patient, at 6-month intervals until the demonstration was completed or the patient was discharged, and at time of discharge for all discharged patients. Discharge status (died, hospitalized, transferred, or discharged home) was recorded at time of discharge.

In addition to the retrospective (preintervention) and prospective (intervention) admission and cross-sectional samples, a fifth longitudinal primary data sample was used to analyze process measures of quality. Termed the prospective service sample, this sample was designed to yield information on the impact of the TNHP on service provision. Longitudinal followup for patients was monthly for 3 months. Specific quotas of certain types of patients already enrolled in one of our prospective samples (e.g., incontinent patients, confused patients, and patients on psychotropic medications) were used for this sample, since it was funded late in the study and time was insufficient to obtain data on newly admitted patients who could be followed for 3 months. The numbers of TNH and CNH patients in the prospective service sample were 450 and 319, respectively.

In all, primary data were collected longitudinally on more than 5,000 patients, including pilot and reliability testing samples. Updates and changes were made to the data base as a result of ongoing outlier and consistency analyses, as well as reviews of individual patient forms and supplementary information obtained from nursing homes and field data collectors. Results presented here are based predominantly on the primary data obtained on the admission samples. Results from cross-sectional samples are referenced, since findings from these samples generally confirm those from the admission samples. In addition to the primary data collected on patients, data on facility-level costs and facility characteristics, including community hospitals, were obtained from Medicaid Cost Reports, program information, surveys of providers/nursing homes, and several secondary data sources, such as American Hospital Association data tapes and Medicare Provider-of-Service files.

Analysis Methods and Variables

Definitions of specific variables and data items are discussed as necessary in the results section. In addition to the comparative and matching approaches noted, statistical methods were used to risk-adjust for covariates or mitigating variables hypothesized to have a potential association with the outcomes. Prior to conducting the final analyses for the hospitalization and discharge-to-independent-living outcome variables, correlations between the outcome variables and patient-status measures, risk factors, demographic characteristics, payer-source indicators, and facility- or community-level characteristics were assessed. The final risk-factor- or case-mix-adjustment analyses included variables that correlated significantly (p < .15) with the outcome variables. Multivariate methods used to adjust for covariates employed logistic regression (dichotomous dependent variables) or survival analytic methods with covariates (for time until a given event, such as time until hospitalization).

For (unadjusted) two-group comparisons, t-tests were used for continuous or near-continuous variables that were approximately normal according to nonparametric goodness-of-fit tests (e.g., Kolmogorov-Smirnov tests). The Wilcoxon two-sample test was used for continuous or ordinal variables not normally distributed, and Fisher's exact test (or its chi-square approximation) was used in comparing proportions for dichotomous variables. To ensure that important results were not simply based on a few dominant facilities, we further investigated the key findings by nursing home. For example, when risk-adjusted hospitalization rates were found to differ between TNH and CNH patients from all facilities combined, we ranked TNHs and CNHs according to individual (risk-adjusted) facility hospitalization rates to assess the extent to which the finding might be due to a few extreme facilities or represented a consistent pattern among all TNHs relative to CNHs.

Demographic, case-mix, and facility-level characteristics were examined to assess potential differences between TNH and CNH patients in terms of important covariates—prior to conducting outcome (and cost) analyses. In general, we found a moderate increase in sub-acute case mix in TNHs relative to CNHs as reflected by a greater increase in TNH patients with cardiac problems (myocardial infarction, dysrhythmia, and congestive heart failure), certain types of mobility impairment (such as restriction of the upper limbs), and fractures (Shaughnessy, Kramer, and Hittle, 1991). While differences in case mix over time and between TNHs and CNHs rendered case-mix adjustment important, the patient populations were judged sufficiently similar in demographics, functioning, and physiologic indicators of health to allow for valid comparisons.

Results

Before-After Changes

Hospitalization rates within 3 months of admission declined by 7 percentage points after the TNHP was implemented in the study nursing homes (Table 1). This statistically significant decline contrasts with a non-significant increase of 4.9 percentage points in CNHs. After using logistic regression to adjust for case-mix differences between TNHs and CNHs (i.e., differences in risk factors for hospitalization) the net difference of 11.9 percentage points in the change in before-after TNH and CNH hospitalization rates increased to 17.2 percentage points, due partly to an increase in sub-acute case-mix intensity resulting from the TNH program.

Table 1

Before/After Changes in Hospitalization Rates, Hospital Days, and Community Discharge Rates for TNH and CNH Patients

	TNH Patients			CNH Patients			Before/After Change
							TNH Change	CNH Change	Mean Difference	Significance3	Case-Mix Adjusted4
	Before Mean	After Mean	Significance2	Before Mean	After Mean	Significance2
											Difference	Significance
Hospitalization Rates(Admission Sample Patients)
Percent Hospitalized in:	Percent			Percent
3 Months	19.3	12.3	.015	13.2	18.1	.130	-7.0	4.9	-11.9	.007	-17.2	.005
6 Months	20.6	14.1	.010	21.5	21.9	.943	-6.5	0.4	-6.9	.109	-9.9	.073
12 Months	25.3	16.0	.003	28.0	25.1	.408	-9.3	-2.9	-6.4	.128	-10.9	.061
Live Discharges
Percent Discharged to Hospital	41.5	27.0	<.001	42.2	42.8	.765	-14.5	0.6	-15.1	.016	-12.5	.055
Hospital Days(Admission Sample Patients)
Hospitals Days in:	Days			Days
3 Months	2.3	0.7	.009	0.7	1.2	.053	-1.6	0.5	-2.1	<.001	-2.3	<.001
6 Months	3.0	1.0	.006	1.5	1.7	.098	-2.0	0.3	-2.3	.001	-2.2	.002
12 Months	3.4	1.3	.003	2.5	2.3	.609	-2.1	-0.2	-1.9	.030	-2.2	.013
Community Discharge Rates(Admission Sample Patients)
Percent Discharged to Community in:	Percent			Percent
3 Months	34.0	45.7	.001	25.2	29.2	.246	11.7	4.7	7.0	.355	0.7	.882
6 Months	38.5	48.1	.002	26.0	34.1	.047	9.6	8.1	1.5	.982	-7.6	.229
12 Months	39.4	48.3	.003	28.8	35.3	.078	8.9	6.5	2.4	.820	-4.7	.461

Retrospective and prospective admission sample sizes are presented in the article. The results for live discharges are based on an overall sample size of 2,437 discharges.

Exact significance level for comparisons of hospitalization rates and community discharge rates are based on Fisher's exact test or the chi-square test approximation to Fisher's test. For comparisons of hospital days, significance levels are based on t-tests or Wilcoxon tests, depending on the underlying probability distribution.

For hospitalization rate and community discharge rate comparisons, the exact significance level for unadjusted mean differences based on significance of the coefficient of an interaction dichotomy (TNH × intervention [post] period) in a logistic regression model that also included dichotomies for pre/post and TNH/CNH. For comparisons involving hospital days, an ordinary least squares regression model was used.

Significance of the adjusted mean difference is the significance of the coefficient of the (TNH × post) interaction in a logistic (ordinary, for hospital days) regression model with case-mix covariates/risk factors and the pre/post and TNH/CNH dichotomies present. The adjusted mean difference was estimated from this logistic (ordinary, for hospital days) regression model.

NOTES: “Before” includes 1981-82. “After” includes 1985-87. TNH is teaching nursing home. CNH is comparison nursing home.

SOURCE: Primary data collected by nursing home and research staff on Teaching Nursing Home Program study patients (from TNHs and CNHs).

Analogous results pertain for 6-month and 12-month hospitalization rates, although TNH/CNH differences are of borderline significance (p = .073 and p = .061, respectively). For all three time periods, before-after decreases in hospitalization rates were significant for TNH patients, while before-after increases were insignificant for CNH patients; differences in the before-after trend for TNH relative to CNH patients, after adjusting for other hospitalization risk factors, were either significant or nearly significant at p = .05. The differences in 6-month and 12-month hospitalization rates were entirely a result of few TNH hospitalizations during the first 90 days of care. Hospitalizations after 90 days were less frequent in both CNHs and TNHs (which is typical for nursing home patients), and did not differ significantly before and after intervention or between TNHs and CNHs. Case-mix adjustment further increased the hospitalization rate differences between TNHs and CNHs for all three rates, reflecting the tendency of TNHs to reduce hospitalization rates even though they admitted higher proportions of patients at risk of hospitalization.

Table 1 also contains hospitalization rates for analyses restricted to patients discharged alive. Once again, the decrease in hospitalization rates for live discharges from TNHs is greater (p = .055) than the (approximately) unchanged rates for CNH patients. Logistic regression analyses were also conducted excluding patients who died or were discharged to the community (i.e., restricting analyses to chronic-care patients). The results of such analyses were basically the same as those that included patients with these attributes incorporated as covariates. (The Technical Notes contain an illustration of the logistic regression approach used to adjust for case mix or risk factors. This approach was used to compute the adjusted odds ratio [which can be algebraically transformed into a mean difference] reflecting the association between the TNH intervention and hospitalization.)

Analyses of hospital days (Table 1) took into consideration readmissions to hospitals (for patients returning to the nursing home), as well as length of stay for each hospitalization. Ordinary regression was used as the case-mix-adjustment method for hospital days, including community hospital occupancy and payer, along with case-mix variables. The results of the hospital days analyses paralleled the hospitalization rate analyses with all findings significant (p ≤ .01).

Although not shown, when before-after hospitalization analyses were stratified according to payer, it was apparent that the TNH effect (i.e., reduced hospitalization) was attained chiefly through lowering hospitalization rates for Medicare patients, who tend to be shorter stay patients in nursing homes. Nonetheless, a trend toward decreased hospitalization in TNHs relative to CNHs was also evident for Medicaid and private-pay patients in the admission samples. Hospitalization analyses conducted using the cross-sectional sample further substantiated this trend, although the TNHP effect of reduced hospitalization was less pronounced when analyses were restricted to longer stay patients in the cross-sectional samples.

Survival analyses were conducted to ascertain whether the before-after change in time until hospitalization, as well as the cumulative proportion of patients hospitalized within 3 months of admission, differed for TNH and CNH patients. Case-mix-adjusted survival curves are presented in Figure 1. The hospitalization “crossover” previously discussed is evident in that the cumulative hospitalization curve in the before period is higher than the analogous curve in the after period for TNH patients, while the reverse pattern is evident for CNH patients. A pooled 3-month survival analysis was conducted to ascertain the statistical significance of the survival curve crossover pattern. This analysis indicated that the before-after difference in TNH and CNH survival curves was significant both before and after case-mix adjustment (p < .025). Case-mix-adjusted results for 6-month and 12-month survival analyses were similar.

Figure 1

Cumulative Percentage Distributions of TNH and CNH Patients Hospitalized Within 3 Months of Nursing Home Admission: Before and After Implementation of the TNHP1

1 Retrospective and prospective admission sample sizes are presented in the article. The number of valid cases for each variable differs somewhat due to the exclusion of cases with incomplete or missing data for covariates.

2 The before/after differences, both unadjusted (not shown) and adjusted, between the TNH and CNH pairs of survival curves are significant (p < .05), indicating that the reversal or crossover pattern of hospitalization rate changes for TNHs and CNHs is substantiated by survival analysis. Eight case-mix variables were significant at p < .15: (1) a case-mix index reflecting expected need (time required) for nursing and aide services; (2) admitted to nursing home from hospital; (3) urinary incontinence or catheter; (4) disability in feeding; (5) shortness of breath requiring oxygen; (6) number of routine medications; (7) diagnostic category for mental disorders; and (8) diagnostic category for fractures.

NOTES: “Before” includes 1981-82. “After” includes 1985-87. TNH is teaching nursing home. CNH is comparison nursing home. Distribution curves have been case-mix adjusted.

SOURCE: Primary data collected by nursing home and research staff on Teaching Nursing Home Program study patients (from TNHs and CNHs).

Hospitalization rates declined in five of the six TNHs, while the rate for the sixth TNH remained virtually unchanged. Hospitalization rates increased in five of the six CNHs, with the remaining CNH experiencing a decrease. Thus, the overall findings reflect a reasonably uniform pattern of facility-level differences between TNHs and CNHs.

The results in Table 1 show no significant before-after differences between TNHs and CNHs in terms of changes in community discharge rates. Noteworthy, however, are the greater and generally more significant increases in the percentage of patients discharged to the community from TNHs relative to CNHs. TNHs had higher community discharge rates in both the before and after periods for the 3-, 6-, and 12-month indicators. Nonetheless, although the intervention-period discharge rates were greater for TNHs, these must be attributed largely to TNH-CNH differences in discharge rates in the baseline period. The TNH-CNH comparison of changes in discharge rates yielded non-significant results both adjusted and unadjusted for case-mix differences.

Analyses of the costs of nursing home and hospital care were also conducted. The results are not presented in detail here, but a summary of the findings is provided in the Technical Notes. Overall, the cost analyses showed that the total cost of care, including both nursing home and hospital costs, was lower for TNH patients. Nursing home costs did not differ, while hospital cost was considerably lower for TNH patients due to reduced hospitalization rates.

Outcome Findings for the Prospective Admission Sample

The finding that the TNHP reduced hospitalization rates prompted analyses to assess reasons for the intervention-period differences between TNHs and CNHs (case-mix-adjusted differences for the intervention period were significant at p < .05). Hence, the richer data base for the intervention period was used to investigate factors that might explain the hospitalization rate differences between TNHs and CNHs, including functional and health status outcomes and selected processes of care.

The terms “stabilized” and “improved” have specific meanings for the outcome measures in this study. Stabilized means the condition under consideration was no worse at followup than it was at admission. Improved means the condition was better at followup than at admission. If the patient remained in the facility for more than 6 months, a “stabilized pattern” means that at each 6-month interval following admission, the patient's condition was no worse than at admission. Analogously, an “improvement pattern” means the condition was no worse than at admission for each succeeding 6-month interval, and that the condition was better than at admission at the final followup point.

Patients who were not able to improve (i.e., were not impaired) or were certain to stabilize (i.e., could not worsen) by virtue of their initial values on the patient status indicators of interest were respectively excluded from the appropriate improvement or stabilization analyses. Several of the patient status outcomes that appear in Table 2 were selected because they were correlated with hospitalization. In particular, the stabilized or improved patient status outcomes for functioning, including transferring, ambulating, bathing, dressing, bowel incontinence, and catheter use were positively associated with lower hospitalization rates in the prospective admission samples.

Table 2

Risk-Factor-Adjusted Patient Status Outcomes in TNHs and CNHs: Prospective Admission Sample

Prospective Admission Sample	TNH Mean	CNH Mean	Unadjusted Mean Difference	Unadjusted Significance2	Case-Mix-Adjusted Mean Difference3	Case-Mix-Adjusted Significance3
	Percent
Discharged to Community
Stabilized in Bathing	45.7	31.9	13.8	<.001	15.7	<.001
Stabilized in Ambulation	42.1	30.1	12.0	<.001	10.5	<.001
Improved in Transferring	30.1	19.5	10.6	<.001	10.1	<.001
Improved in
Decubitus Ulcer Pattern	95.8	75.0	20.8	.033	10.1	.227
Bowel Incontinence	54.8	32.8	22.0	.013	27.5	.004
Stabilized in
Catheter Pattern	91.9	83.1	8.8	.007	11.7	.002
Decubitus Ulcer Pattern	88.0	83.0	5.0	.119	3.9	.231
Patients in Facility 6 Months
Improved in Dressing	28.2	36.6	-8.4	.046	-5.8	.177
Improved in Feeding	43.7	52.4	-8.7	.086	-7.6	.166
Stabilized in Urinary Incontinence	78.2	85.8	-7.6	.077	-5.0	.315

Data correspond to the intervention-period prospective admission sample. Prospective admission sample sizes are presented in the article. The number of valid cases for each variable may differ due to the exclusion of cases with incomplete or missing data.

The unadjusted significance level for dichotomous variables is that of the odds ratio (coefficient) in a logistic regression model using only the 0/1 TNH indicator as an independent variable.

The mean difference for dichotomous variables was adjusted using logistic regression. The significance for the adjusted mean difference is the significance of the odds ratio, i.e., exp (b), where b is the coefficient of the TNH versus CNH dichotomy in a logistic regression model, with case-mix covariates in the model.

NOTES: TNH is teaching nursing home. CNH is comparison nursing home.

SOURCE: Primary data collected by nursing home and research staff on Teaching Nursing Home Program study patients (from TNHs and CNHs).

The first three patient status outcomes in Table 2 reflect stabilization or improvement in functioning at time of discharge to the community. Although some such variables were insignificant, these three significant differences typify a general pattern of more positive outcomes in functioning for TNH patients relative to CNH patients at time of community discharge, after adjusting for risk factors or case mix. Attainment of the indicated outcome required both discharge to the community and stabilization or improvement in the particular functional measure under consideration. Since these three outcomes were also positively correlated with reduced (non-) hospitalization at the patient level, the capacity of the TNHP to enhance or stabilize activity of daily living (ADL) functioning appears to be a mechanism by which TNHs reduced hospitalization rates among the frail elderly.

The second and third groups of outcome variables pertain to improvement or stabilization in certain conditions. In these cases, the patient may have been discharged to the community, with patient status recorded at time of discharge, or remained in the facility, with patient status recorded every 6 months. Again, the findings in Table 2 are representative of the patterns characterizing a somewhat larger set of outcomes reported elsewhere (Shaughnessy, Kramer, and Hittle, 1991), and include all functional and health status measures for which there was a statistically significant difference between TNH and CNH patients. The bowel incontinence and catheter pattern variables indicate that, after case-mix adjustment, a greater proportion of TNH patients than CNH patients attained favorable outcomes in these areas. A patient was “stabilized in catheter pattern” if he/she did not have a urinary catheter inserted after admission to the nursing home. The case-mix-adjusted result for this outcome suggests that patients were less likely to be catheterized while in TNHs. A patient was “improved in decubitus ulcer pattern” if he/she was admitted with decubitus ulcers but had a subsequent permanent reduction in the number of ulcer sites. Analogously, a patient was “stabilized in decubitus ulcers pattern” if he/she was either admitted without decubitus ulcers and remained so throughout the course of institutional followup or until discharge or admitted with decubitus ulcers that did not worsen over this period. The TNH/CNH differences in both the improved and stabilized decubitus ulcer pattern variables were insignificant after case-mix adjustment.

The final set of patient status outcomes for the prospective admission sample in Table 2 pertains to patients who were in the facility for a full 6 months. The greater positive effect of the TNHP exhibited in the preceding outcomes is reversed for these outcomes prior to case-mix adjustment. However, case-mix adjustment renders the outcome differences between TNHs and CNHs insignificant. This appears to be due to the tendency for CNHs to retain (for a longer period of time than TNHs) those patients who are more stable or have better rehabilitation potential, resulting in a higher unadjusted proportion of CNH patients improved or stabilized in functioning in the facility at 6 months. After taking into consideration (through case-mix adjustment) patient characteristics that reflect a more stable condition and/or higher rehabilitation potential, the TNH and CNH means are not significantly different.

Outcome Findings for the Prospective Service Sample

The results in Table 3 are based on data from the prospective service sample (which consists of longer stay patients than the prospective admission sample). The first two functional outcomes in this table pertain to all patients and are outcome variables analogous to those previously discussed. After adjusting for case-mix differences, a higher proportion of TNH patients in the prospective service sample were stabilized in both dressing and transferring, and did not have urinary catheters inserted. In this analysis, the dressing/transferring result is based on a composite index of the dressing and transferring functional scales. For this index to indicate stabilization, the patient had to be stabilized in both dressing and transferring.

Table 3

Risk-Factor-Adjusted Findings on Process Quality Measures in TNHs and CNHs: Prospective Service Sample

Prospective Service Sample	TNH Mean	CNH Mean	Unadjusted Mean Difference	Unadjusted Significance2	Case-Mix-Adjusted Mean Difference3	Case-Mix-Adjusted Significance3
	Percent
All Patients
Stabilized in Dressing/Transferring Function	54.0	47.3	6.8	.035	8.7	.013
Stabilized in Catheter Pattern	97.9	94.9	3.1	.013	2.4	.042
Care Plan Participant
Nurse Clinician	27.7	1.1	26.6	<.001	–	–
Nurses' Aide	55.2	29.0	26.2	<.001	–	–
Registered Physical Therapist	60.7	67.9	-7.2	.018	–	–
Resident	12.4	17.4	-5.0	.027	–	–
Confused Patients
No Restraints Used	40.6	31.7	8.9	.030	14.0	.005
Restraints Checked Every 30 Minutes	44.0	30.7	13.3	.010	14.2	.007
Psychotropic Medication Patients
Mean Daily Dose of Neuroleptics4	58.5	120.3	61.8	.016	46.6	.072
No Excess Mean Daily Dose of Neuroleptics	100.0	94.6	5.4	.005	3.8	.031
No Long-Acting Benzodiazepines	79.2	60.8	18.4	.008	15.7	.027
Incontinent Patients
Timed Voiding 4 Times Day/Night	50.9	39.7	11.2	.037	8.2	.177
Avoidance of Incontinent Episodes With Timed Voiding	35.9	25.3	10.6	.014	12.4	.010

Data correspond to the intervention-period prospective service sample. Prospective service sample sizes are given in the article. The number of valid cases for each variable may differ due to the exclusion of cases with incomplete or missing data:

The unadjusted significance level for dichotomous variables (continuous variables) is that of the odds ratio (coefficient) in a logistic regression (ordinary regression) model using only the 0/1 TNH indicator as an independent variable.

The mean difference for dichotomous variables (continuous variables) was adjusted using logistic regression (ordinary regression). The significance for the adjusted mean difference is the significance of the odds ratio, i.e., exp (b), where b is the coefficient of the TNH versus CNH dichotomy in a logistic regression model (or of the coefficient of b in an ordinary regression model for continuous variables), with case-mix covariates in the model.

In thorazine milligram equivalents.

NOTES: TNH is teaching nursing home. CNH is comparison nursing home.

SOURCES: Primary data collected by nursing home and research staff on Teaching Nursing Home Program study patients (from TNHs and CNHs).

Structural and Process Quality Findings

The other findings in Table 3 were used to assess structure and process of care in the study nursing homes. The care plan participant variables, considered as structural variables here, were not adjusted for case mix, since they result from unit-level arrangements and decisions that are not substantially associated with case-mix characteristics. The TNHP was characterized by programs to lower hospitalization through greater involvement in care planning of nurse clinicians and nurses' aides. This was an overt objective of the TNHP—reduced hospitalization through better care planning led by and involving nurse clinicians. In view of the focus on hospitalization in the context of such planning programs, it is probable that they were one of the reasons for the lower TNH hospitalization rates. The involvement of nurse clinicians in initial patient assessment, care planning, and teaching of nursing home staff progressively increased over the course of the TNH (Mezey, Lynaugh, and Cartier, 1989; Shaughnessy, Kramer, and Hittle, 1991). These nurse clinicians were more commonly involved in care planning in TNHs for the admission samples of patients than for the longer stay patients in cross-sectional samples, for whom results are presented in Table 3 (Lavizzo-Mourey et al., 1988). The greater involvement in care planning in CNHs by registered physical therapists and residents themselves is less likely to affect hospitalization than nurse clinician/aide involvement because the magnitude of the therapist/resident difference between TNHs and CNHs is considerably less than that of the clinician/aide difference.

The remaining findings pertain to specific subgroups of patients within the prospective service sample: confused patients, patients on psychotropic medications, and incontinent patients. From the perspectives of quality of life and potential for adverse outcomes, use of restraints is an undesirable approach to treating severely disoriented or easily agitated nursing home residents. The rates for restraint use reported in Table 3 are not cross-sectional prevalence rates at a single point in time, but rather restraint use at least intermittently over the previous month among confused nursing home residents. In addition, an indicator of how often restraints are checked when mechanical restraints are used is included. The difference in restraint use and the greater likelihood that restrained TNH patients were checked at least every 30 minutes, after adjusting for case mix, suggest more appropriate care of such patients in TNHs. These rates from the 1980s are higher than would be seen in most nursing homes today because of the increasing Federal emphasis on a restraint-free environment (Tinetti et al., 1991).

Indicators of the use of psychotropic medications suggest that care for patients with psychiatric problems may have been better in TNHs. The mean daily dose of neuroleptics (converted to milligrams of thorazine in Table 3) was significantly lower in TNHs after adjusting for case-mix differences, reflecting a tendency to use lower doses of major tranquilizers in TNHs. The lower prevalence of excessive mean daily doses of neuroleptics, indicating that fewer patients in TNHs were receiving doses that exceeded accepted standards of care, is also consistent with the lower mean daily dose of neuroleptics in TNHs. Less use of long-acting benzodiazepines (i.e., flurazepam or diazepam) further suggests more appropriate prescribing patterns in TNHs, because the shorter acting medications are considered preferable for use in the elderly. Although restraint use and psychotropic medication prescriptions require physician orders, decisions relating to restraint or psychotropic medication use tended to be made through interactions between nursing staff and physicians, and may have resulted from the inability of the staff to manage the patient in another way. Furthermore, the total dose of psychotropics actually administered is often affected by “as needed” use that is controlled by nursing staff.

Successful use of timed voiding to avoid episodes of incontinence, as opposed to catheterization or the management of incontinence with diapers or pads, is preferable for patient self-esteem and can reduce the risk of urinary tract infections or skin breakdown. Two indicators of the use of timed voiding appear in Table 3. The first indicator is the proportion of incontinent patients on a timed voiding regimen, while the second indicator represents successful use of timed voiding to avoid incontinent episodes in the same patients. After case-mix adjustment, TNHs were more likely to avoid incontinent episodes by providing timed voiding for incontinent patients. Conducting timed voiding at least four times per 24-hour period was more common in TNHs before case-mix adjustment, but the difference in rates was insignificant after adjustment for case mix. Thus, TNHs were able to use timed voiding successfully to avoid incontinent episodes among more patients than CNHs. This was not accomplished by regular use of timed voiding for more total patients, but presumably by providing timed voiding more aggressively and effectively for the appropriate patients.

The empirical findings from Tables 2 and 3 paint a picture from which inferences can be drawn with varying degrees of confidence. They suggest rather strongly that the significant TNHP impact of reduced hospitalizations in the first 3 months is due to a composite of several factors rather than just one. Since the enhanced functional outcomes were also positively correlated with reduced hospitalization at the patient level, it is likely that the several TNHP programs which focused on improving functional independence led to reduced hospitalization, although it is not possible to determine the precise nature and extent of this relationship.

Staffing changes and team composition were probably influential; however, with the exception of the involvement of nurse clinicians, this inference is not as clear. The pervasiveness of the nurse clinician effect on hospitalization is almost inextricably tied to the TNHP effect itself. However, based on our knowledge of the program resulting from case studies, site visits, reports submitted by the participating sites, and numerous conversations with the sites and program coordinators/administrators, we are reasonably confident in the conclusion that the nurse clinician involvement in assessment, care planning, and staff training accounted for, directly and indirectly (i.e., via the care behaviors of other staff influenced by nurse clinicians), a substantial portion of the reduced hospitalization rates and improved functional independence of patients. Again, the extent of this effect cannot be quantified precisely, but our familiarity with the program leads us to conclude that the magnitude of the nurse clinician effect was substantial.

The superior process quality results for TNH patients in terms of restraint use, psychotropic medications, and incontinence care suggest greater attentiveness on the part of TNH providers to accepted (and progressive) LTC standards. In fact, these three areas—less restraint use, constrained use of psychotropic medications, and increased independence or control with respect to incontinence problems—can be important components of the quality of life for many LTC patients. In this case, however, perhaps more so than with the superior functional outcomes for TNH patients and the diverse involvement of nurse clinicians in TNHs, a linkage with reduced hospitalizations is less obvious. While arguments can be made that better care in these areas can lead to fewer comorbidities or risks of hospitalization of certain types (e.g., increased skin ulcers and cardiovascular impairment for restrained patients or negative reactions and side effects from overmedication), less restraint use or sedatives for some types of patients can result in increased likelihood of falls and fractures which, in turn, increase hospitalization. On the other hand, more extensive and diligent use of timed voiding can lower the risk of urinary tract infections and impaired skin integrity, both of which can lower the probability of hospitalization. Hence, superior care in these areas suggests a greater attentiveness to quality of life in TNHs and possibly (but not clearly) reasons for reduced hospitalization.

Facility-level rankings were consistent with the patient-level findings. That is, TNHs tended to be ranked higher than CNHs with respect to the patient status outcomes and process quality measures presented here. Policy and other methodological implications of these findings are discussed in the conclusions.

Conclusions

In this section, programmatic inferences and conclusions are drawn from the empirical findings of the TNHP evaluation for purposes of extending the results to the policy level and, equally important, discussing quality-of-care methodologies and applications. Policy or programmatic conclusions are presented first. The final section on methodology extends the comments and principles based on nursing home research (e.g., the TNHP evaluation) by incorporating conceptual and applied principles from more recent work in the home health care field.

TNH Program and Policy Implications

The decrease in hospitalization rates for patients admitted to the six TNHs contrasts not only with the increased trend for CNHs but also with national trends during the same period that showed increased hospitalization rates for nursing home patients in the United States (National Center for Health Statistics and Sekscenski, 1987; Sager, Leventhal, and Easterling, 1987). Although the decrease in hospitalization rates applies to hospitalization within 90 days, this time period is when most hospitalizations occur among nursing home residents. The decline was more pronounced for short-stay and Medicare patients than for long-stay and Medicaid patients. The patient status outcome and process quality results complement the hospitalization and institutional cost results. In particular, the more specific outcome and process quality results provide some evidence of higher quality care in TNHs relative to CNHs, affecting both short-stay and long-stay patients. In TNHs, patients were less likely to experience functional decline and were less likely to be catheterized, restrained, or heavily sedated, all of which suggest greater attentiveness to selected components of quality of life in TNHs. The differences in hospitalization, patient status outcomes, and care patterns occurring in combination indicate that a different care environment existed in TNHs. Some empirical evidence exists for what appears to be an association between enhanced patient status outcomes and lower incidence of hospitalization.

While the frail elderly are often characterized by chronic or degenerative illnesses, it appears that, within reason and the dictates of good care, there is ample room for a stronger philosophy of improving independent physical and cognitive functioning for many nursing home patients. This was an overriding goal in many of the TNH sites, implemented through targeted patient-care programs. The findings suggest that the six sites studied achieved some successes in meeting this goal. Although the programs were multifaceted at the different TNH sites, a characteristic that was present at all sites was a team-oriented approach involving several disciplines that included physicians and therapists, but emphasized the role of nurse clinicians in patient assessment and care planning. It is apparent from the TNHP's impact on specific aspects of patient care that the behavior of nursing staff, especially nurses' aides, who provide the majority of care in nursing homes, was influenced.

The TNHP has some implications for LTC quality improvement. First, nursing home quality improvement through affiliation with schools of nursing is possible and warrants consideration on a more widespread basis. Several of the elemental features of the TNHP, including participation of nurse clinicians, aides' greater awareness of and involvement in care planning, and specific programs targeted at improved functioning, reduced hospitalization, and enhanced quality of life, were integral components of the affiliations between nursing homes and schools of nursing. These types of approaches and programs should be evaluated and tested in more LTC settings.

Second, enhanced care planning through greater involvement of nurse clinicians and aides is a feature of the program which may be transportable, independent of affiliations between nursing homes and schools. Nurses with advanced preparation in clinical nursing are not always available to participate in such planning, however. Of particular concern is whether clinicians such as those in the TNHP would be motivated or able to function in a capacity similar to the way they functioned in the TNHP without strong linkages with schools of nursing. In this regard, other approaches involving participation of individual nurse practitioners in nursing homes also show positive effects on patient care (Kane et al., 1989).

Third, several features of the TNHP, including the involvement of nurse clinicians, a strong approach to maximizing physical and cognitive functioning, enhanced incontinence care, and attentiveness to the proper administration of psychotropic medications, were accompanied by reductions in hospitalizations. While causative associations between these factors and lower hospitalization rates cannot be definitively established, patient-level correlations, clinical reasoning, and an analysis of program goals and behaviors suggest that some of these factors constitute the means by which at least a reasonable proportion of reduction in hospitalizations occur. In fact, since the reduction in hospitalizations results in a savings, or at least is accompanied by enhanced patient outcomes at no additional total cost, the Medicare program especially might consider reimbursement provisions directed at covering the cost of nurse clinicians, education for aides, and selected educational programs targeted at more aggressive enhancement or maintenance of physical and cognitive functioning.

As previously mentioned, the evaluation of the TNHP was a feasibility study. Due initially to budgetary constraints and later to changes that occurred in 2 of the TNHs, the analysis was restricted to patient-level data from 6 of the original 11 TNH sites. Randomization was not possible. Baseline data were retrospectively collected from nursing home records and were not as comprehensive or precise as the prospectively obtained data from the intervention period. Data on the proportion of Medicaid patients initially used to select comparison sites later proved to be somewhat inaccurate for selected sites, slightly weakening the precision of the matching procedures. However, through comparisons with national data, payer-specific analyses at the patient level, programmatic analyses based on both site visits and administrative surveys, analyses of primary data from different patient samples, and reference to the contextual setting of the program in the nursing home field, the TNHP results shed light on directions that might be taken in the future to enhance nursing home care (Shaughnessy, Kramer, and Hittle, 1991). At a time when quality of care and quality of life in nursing homes are of significant concern, the potential value of TNHs warrants consideration as a means of enhancing outcomes of nursing home care.

Methodologic Suggestions and Quality-of-Care Issues

As noted earlier, improvement in outcomes is the object of quality improvement, since beneficial change in health status between two or more time points is the goal of providing health care. When we positively alter the natural progression of disease and disability by providing health services, the extent of benefit reflects the quality of care provided. If outcomes are not optimal, a goal of Continuous Quality Improvement (CQI) is to determine why. Analogously, if outcomes are exemplary, it is appropriate to assess how they were attained in order to reinforce the appropriate care processes. For quality evaluations, linking outcome findings to attributes of the intervention that might explain the effects increases our confidence about the outcome findings and their attribution to the program. Rarely can such linkages be unequivocal, but careful analysis, sound clinical thinking, and assessment of patterns of findings (rather than reliance on one or a few results) can produce useful conclusions on which decisions can be based or provide hypotheses for more in-depth analysis. Thus, a blend of outcome, process, and structural quality measures are beneficial both for program evaluations like the TNHP and in the context of quality improvement.

Nevertheless, for both types of applications, outcomes represent a reasonable starting point. In the context of quality improvement, we can use outcomes to allocate (typically scarce) resources to those areas that require remediation or warrant analysis because of exemplary performance. For example, if outcome findings indicate poorer than expected rates of functional recovery, we can review rehabilitation services and activities for such patients. As another example in the TNHP, enhanced outcomes directly linked to processes, such as lower catheter use rates, can be used to identify specific nursing activities which might be targeted for CQI.

As a result of employing different types of outcomes on the TNHP evaluation and evaluating the quality of home health care in subsequent projects (Shaughnessy, Schlenker, and Hittle, 1995; Shaughnessy et al., 1994; Schlenker, Shaughnessy, and Hittle, in press), we have developed two taxonomies that are useful in conceptualizing quality. The first taxonomy classifies an outcome or an outcome measure according to the directness with which it reflects change in health status that is related to the purpose of care (outcome-type taxonomy). The second classifies an outcome or outcome measure according to the time interval or, more appropriately, the care interval to which the outcome pertains (outcome-interval taxonomy).

Outcome-Type Taxonomy

The outcome-type taxonomy is presented in Table 4. The time interval of interest is assumed to be one during which health care was provided. A “pure outcome” is defined as a change in patient health status between two or more time points, where the change under consideration is directly related to the justification for providing health care or should be monitored as a routine matter of providing care. A more explicit definition and examples are given in Table 4. Such a change in health status can be measured using physiologic, functional, cognitive, behavioral, or affective domains of health. Change can be in any direction: improvement, worsening, or neutral (no) change between baseline and followup points.

Table 4

Outcome-Type Taxonomy

Outcome	Definition	Examples
Pure Outcome1	A change in health status intrinsic to the patient, where the health status attribute used to assess or measure the outcome is (one of) the primary focal points of the care provided, should be monitored as a routine matter of providing care (to prevent or avoid comorbidities or complications), or is a necessary condition for achieving the purpose of the care.	Change in ability to ambulate between start of care and discharge for a home health patient admitted with an orthopedic condition.Change in status of a pressure ulcer over a 4-week period for an immobile nursing home resident receiving repositioning, debridement, and nutritional services.Change in ability to articulate specific words over a 3-month episode of care for a head injury patient who received speech/language pathology services at home since shortly after the head trauma.
Instrumental Outcome1	A change intrinsic to the patient (or the patient's informal caregiver) in knowledge, emotion, or behavior that is or may be instrumental to attaining a pure outcome. Care can be provided for the purpose of enhancing instrumental outcomes which, in turn, facilitate attaining pure outcomes.	Change/improvement in a home care patient's awareness of early signs of infection at a wound site.Extent of a cardiac patient's compliance with a prescribed exercise and nutritional program.Change in patient knowledge of how to conduct a self-examination for a particular type of tumor.
Utilization Outcome	An event that reflects or is a proxy for a pure outcome, often in the form of utilization of health services other than those provided in the care setting or by the individual provider whose outcomes are being assessed.	Hospital admission (for acute malnutrition) of an elderly nursing home patient receiving dietary and nutritional services in the nursing home. Nursing home admission for a moderately cognitively impaired patient receiving home care, who is on several medications simultaneously. Emergent care due to infection at a wound site for a postsurgical patient receiving wound care from a home health provider.

Pure and instrumental outcomes have been respectively termed “end-result” and “intermediate-result” outcomes in some of our prior writings.

SOURCE: Shaughnessy, P.W., Kramer, A.M., Hittle, D.F., and Steiner, J.F., University of Colorado Health Sciences Center, 1995.

Pure outcomes can be measured over short or long periods of time depending on the application. Ideally, however, it is best that the time interval correspond to a particular episode of care (e.g., a home health stay, a nursing home stay, or a particular period of time within such a stay during which the care is expected to meet its objectives). For example, using a well-defined and precisely specified scale for ambulation, it might be appropriate to assess the outcome of change in ability to ambulate for a hip fracture patient who received hip replacement surgery. In this instance, the time period over which change in ambulation might be monitored would be immediately preceding the surgery to several weeks or months after the patient had undergone surgery. The longer the outcome interval, the greater the potential for other factors (other than care provided) to influence outcomes, and the greater the difficulty of attributing outcomes to care provided during the episode. Baseline risk-factor adjustment is important, but it also may be important to adjust for exogenous risk factors that arise, or become evident during the episode of care.

An “instrumental outcome” is a change between two or more time points, typically in patient or caregiver knowledge, emotion, or behavior, that is instrumental in producing a pure outcome. Similar to pure outcomes, instrumental outcomes are intrinsic to patients (or patients' informal caregivers). Unlike pure outcomes, however, instrumental outcomes are not the primary objective of care provided, nor are they routinely monitored in the context of providing care. For example, change in a diabetic patient's awareness of when to report signs and symptoms is an instrumental outcome, because, although important, such outcomes are instrumental to but not the primary objective in treating diabetes (nor is increased awareness of signs and symptoms monitored routinely in providing care to diabetics). Analogously, patient satisfaction with care can influence patient motivation which, in turn, can affect pure outcomes. Satisfaction, however, may not be the primary purpose of care provided. Care is provided primarily to affect health or functional status. However, in cases where satisfaction with support or personal assistance services is considered essential to quality of life, as in some LTC environments (Patrick, 1990; Murtaugh, 1992), satisfaction would be classified as a pure outcome. (Some would argue that satisfaction is a pure outcome because one of the primary purposes of care is to meet the needs perceived by the patient If this premise is accepted—and it is becoming more popular as we move towards the belief that “quality of care is as the consumer defines it”—then satisfaction logically belongs to the category of pure outcomes.) Similarly, patient compliance with a treatment regimen can be regarded as an instrumental outcome, since taking medications at the prescribed times and dosage is instrumental to attaining the pure outcome of reduced hypertension (or, in the longer run, avoidance of stroke).

As illustrated by the patient satisfaction example, outcomes that reflect change in knowledge, emotion, or behavior can be pure or instrumental depending on the purpose of the care provided (or a perspective of the purpose of care). In providing counseling or bereavement services for the terminally ill or their families, changes in behavior, emotion, or knowledge may be the primary objective of care. Intellectual and emotional acceptance of imminent death for the sake of emotional well-being is often the primary purpose of such services. The actual purpose and scope of the type of health care under consideration determines whether an outcome is pure or instrumental, but both types are typically intrinsic to, or occur within, the patient. In contrast, when care is provided to individuals in addition to the patient, such as in home health settings, when services are provided to reduce home or informal caregiver strain, such changes can be an important instrumental outcome for some patients. Changes that are external to the patient, such as a provider-initiated change in the patient's environment that reduces the risk of falling, possibly in the form of handrails or equipment to help the patient ambulate, belong to the category of process or structural indicators of quality, not outcomes.

A “utilization outcome” is an event or indicator of change in patient health status that is not a pure outcome, but reflects the probable occurrence of a pure outcome through the use of health services. Rigorously speaking, a utilization outcome is not an outcome but a process or service. We use the term outcome because the provision of such services under selected conditions signifies that a pure outcome occurred. Thus, inpatient hospitalization for a patient with chronic heart disease would be a utilization outcome, possibly indicating that congestive heart failure or another untoward event resulted from inadequate care. Emergent care for a home care patient due to an infected pressure ulcer would also be a utilization outcome. At times, utilization outcomes reflect good care. For example, some of the hospitalizations examined in the TNHP evaluation were undoubtedly appropriate. In examining utilization outcomes, it is useful to assess reasons for utilization (which is not always possible, as in the TNHP evaluation) and adjust for case mix. If it is possible to analyze pure outcomes and processes of care in conjunction with utilization outcomes, an overall profile of measures then can be used to draw inferences about quality of care.

Outcome-Interval Taxonomy

Outcomes can be measured for a variety of purposes which determine the duration of the outcome interval or the time between the baseline and followup points. It may be appropriate to measure health status at interim time points during the outcome interval to assess the stability of change in health status over time (Shaughnessy et al., 1994). As shown in Table 5, outcomes can be divided according to the extent of services provided during the interval over which the outcome is measured. While it is possible to construct more than four such categories (Daley and Shwartz, 1994), those given in Table 5 can be of practical value in framing applications regardless of whether one is using outcome or process measures of quality.

Table 5

Outcome-Interval Taxonomy

Outcome	Definition	Examples
Service-Specific Outcome	An outcome whose baseline pertains to the time immediately preceding a unit of service and whose followup time point pertains to a time (possibly immediately) following the unit of service.	Increased ability to manage oral medications (dosage and frequency) on the part of a nursing home patient with a cardiac condition after a single LPN teaching session.As a result of the first of several home care visits, increased patient and family awareness of safety hazards for an individual who is impaired in ambulation.
Multi-Service Outcome	An outcome whose baseline pertains to the time immediately preceding several units of service and whose followup time point pertains to a time (possibly immediately) following the several units of service.	Change in a chronic care nursing home resident's ability to transfer (from bed to chair) over a 3-month period during which the resident has been receiving range of motion and resistance training therapy from a physical therapist aide—over the course of an extended nursing home stay.Reduced anxiety after four psychiatric visits for patient undergoing extended psychiatric treatment (including medications) for acute anxiety attacks.
Episode-Specific Outcome	An outcome whose baseline pertains to the time immediately preceding the episode of care and whose followup time point pertains to a time (possibly immediately) following the episode of care.	Change in a stroke patient's ability to bathe herself after a 6-week stay in a skilled nursing facility that immediately followed a 2-day hospital stay precipitated by the patient's stroke.Healed pressure ulcer at discharge from home health care for a patient admitted to home care with a Stage 3 pressure ulcer.
Multi-Episode Outcome	An outcome whose baseline pertains to the time immediately preceding several episodes of care and whose followup time point pertains to a time (possibly immediately) following the multiple episodes.	Number of acute hospitalizations for a congestive heart failure patient over a 2-year period during which the patient was under the care of an HMO receiving primarily physician office-based care and home-and community-based services.Change in ability to speak and communicate interpersonally over a 6-month period for a stroke patient receiving acute care and post-acute speech and rehabilitation services.

NOTES: LPN is licensed practical nurse. HMO is health maintenance organization.

SOURCE: Shaughnessy, P.W., Kramer, A.M., Hittle, D.F., and Steiner, J.F., University of Colorado Health Sciences Center, 1995.

The first domain pertains to an outcome measured over the course of a single unit of service. This type of outcome, termed a “service-specific outcome,” has its baseline point immediately preceding the provision of the service and its followup point immediately following the unit of service. An illustration of a service-specific outcome would be a change in a nursing home resident's awareness of a timed-voiding schedule as a result of a 30-minute training program administered to incontinent patients by a geriatric nurse clinician in a teaching nursing home. Many service-specific outcomes in LTC are instrumental outcomes according to this taxonomy.

Outcomes pertaining to multiple service units that are part of an episode of care are termed “multi-service outcomes.” For example, change in a nursing home resident's range of motion might be assessed between start of care and after several physical therapy visits in order to assess progress or overall effectiveness of the therapy, despite the fact that more physical therapy visits will take place.

If an outcome pertains to the entire period of time between start of care and discharge from (a particular type of) care, it is called an “episode-specific outcome.” Many of the outcomes used in the TNHP evaluation (Tables 1 and 2) were either multi-service or episode-specific outcomes. Another illustration of an episode-specific outcome (that is also a pure outcome) would be the change in status of a surgical wound between admission and discharge for a post-surgical patient admitted to home health care. An episode-specific utilization outcome that could be assessed over the same period would be an indication of whether the patient was hospitalized during the home health stay while receiving treatment for wound care.

An outcome whose baseline and followup time points span several different (types of) episodes of care is termed a “multi-episode outcome.” Often patients receive care from multiple providers at the same time and/or care is provided sequentially by several different providers. Under these circumstances, it is difficult to separate the effects of one provider from another, but it can be useful to examine outcomes from the perspective of the totality of care provided, i.e., assess multi-episode outcomes as the composite effects of care provided. To assess quality of care for specific providers in these cases may require process measures that are known to influence outcomes. While inroads have been made in measuring the first three outcome categories in Table 5, we are far from measuring and comprehensively assessing multi-episode outcomes. Nonetheless, it is useful to define them, since many purchasers, policy analysts, and researchers are now advocating their use.

Taxonomy Applications

The taxonomies presented in Tables 4 and 5 can serve various purposes. The outcome-interval taxonomy can serve to suggest where the burden of responsibility should lie in terms of monitoring outcomes. For service-specific outcomes, this responsibility should typically be the purview of providers or those conducting research on new treatment approaches or interventions. As one proceeds from multiservice outcomes and episode-specific outcomes to multi-episode outcomes, the responsibility for assessing and monitoring such outcomes should continue to remain with providers, if possible, but it should be increasingly shared by purchasers and government as one proceeds through the four types of outcomes in this taxonomy. This pertains especially to multi-episode outcomes because such outcomes are typically beyond the purview of a single provider of care. Where multiple providers are involved, it should be the concern of the purchaser to monitor outcomes. Government may appropriately take on a standard-setting and coordinative role across providers and purchasers of care. As technology and data systems improve, our ability to monitor multi-episode outcomes of care should increase. The challenge, of course, is to specify useful outcome measures (that can be used in a variety of settings).

The outcome-type taxonomy highlights the interrelationships among different kinds of outcomes. This can be useful in quality-improvement programs and research. In the TNHP evaluation, complementary findings using different types of outcomes as well as process and structural indicators served not only to strengthen and clarify results, but assisted, to some extent, in understanding the reasons behind findings. In addition, complementary findings can suggest how to proceed with either decisionmaking for policy purposes or quality-improvement initiatives. Pure outcomes are useful for purposes of empirically validating both utilization outcomes and process or structural measures of quality. Instrumental outcomes, although often difficult to measure, are important for some types of care, especially home care where, unlike institutional LTC, the provider exercises far less control over the patient's environment and personal circumstances. Awareness that instrumental outcomes are important in such instances can lead to useful quality-improvement initiatives (such as critical pathways that use instrumental outcomes on a per visit basis—for example, a critical pathway for cardiac patients would specify individual services to be provided per visit, one of which might be to increase patient awareness of medication dosage and frequency, with the requirement that patient awareness of dosage and frequency be monitored and recorded).

The outcome-interval taxonomy can assist in clarifying the time-dependent nature of outcomes (and even process measures of quality). We have found this taxonomy useful in designing projects to evaluate quality and have begun to use these categories for quality-improvement applications, especially in home care. It is useful for providers to recognize and understand the different types of outcomes in this taxonomy when reviewing quality of care for purposes of performance improvement over the course of time. Such applications require data be collected to measure outcomes so that they can be compared from 1 year to the next. Episode-specific outcomes are particularly useful in this regard in the home health care field.

In conducting the TNHP evaluation and in subsequent work in the home care field, we have experimented with different ways to measure outcomes. Broken down into small increments, a pure outcome can be considered as a series of transitions in health status over multiple time points. In actuality, as providers monitor and react to a patient's changes in health status at multiple times over an episode of care, they often seek to incrementally bring about changes or transitions in health status from one time point to the next (e.g., surgical wound healing or recovery of quadriceps strength in a multiple fracture patient). In this regard, it is possible to define patterns of change over multiple time points (e.g., steady improvement or stabilization/non-worsening) as particular types of outcomes and to construct measures to reflect such patterns (Shaughnessy et al., 1994).

Upon completion of our analyses and subsequent application of selected measures for purposes of outcome-based quality improvement, we concluded that the simple dichotomous measures indicating whether a patient had improved or worsened between start of care and discharge were the most useful for clinical purposes (Shaughnessy et al., 1994). Such measures are understandable for clinicians and others not trained in measurement theory and methodology. Dichotomous measures have the redeeming attribute that, aggregated across pulmonary patients, for example, they yield the percentage of patients improved or worsened in dyspnea.

This criterion of using measures that are understandable or “intuitively attractive” also pertains to process and structural measures of quality. In terms of process quality especially, several measures are often examined simultaneously (that pertain to different services or different attributes of the same service). This naturally leads to the desirability of distilling results from multiple measures down to a single overall measure or more parsimonious set of findings. While there is nothing wrong with attempting to do this, it should be done in a reasonable way, using intuitively attractive measures or summarization methods that are no more complicated than absolutely necessary. Also, summary measures may not be as useful as more detailed indicators in identifying specific problems or in identifying specific changes to make in care processes.

Issues in Quality Measurement

When analyzing outcomes (and, at times, processes of care), comparisons invariably require risk adjustment to compensate for the possibility that outcomes in the two (or more) groups being compared are influenced not only by health care, but by comorbidity, disease severity, and other patient characteristics. Risk adjustment can be done through stratification, statistical methods, or both. Since a substantial literature exists on risk-adjustment techniques, methodologies involved in risk adjustment are not covered here (Iezonni, 1994; Gonnella, Hornbrook, and Louis, 1984; Geraci et al., 1993; Harrell, Lee, and Pollack, 1988; Iezzoni, Moskowitz, and Ash, 1988; Knaus et al., 1991; MediQual Systems, Inc., 1993). Risk adjustment is an inexact science—not because statistical methods are egregiously inadequate, but largely because we cannot typically collect data on all risk factors that should be taken into consideration. Often we are constrained either because of inadequate data collection resources or because we do not have adequate knowledge about risk factors. Since case-mix or risk-factor profiles for a given provider or provider type might not change substantially from 1 year to the next, risk adjustment may not be as imperative in comparing outcomes on a before/after basis or from 1 year to the next for the same provider. In implementing outcome-based quality improvement in home health agencies, we often have found this to be the case (when case-mix changes occur because of a contract with a new payer, changes in the marketplace, etc., an analysis of case-mix profiles for the years under consideration can typically reflect changes that in turn suggest the need for risk adjustment). In the home care field, this is encouraging for individual providers interested in implementing programs to continually improve care provided. Even under such circumstances, however, it is appropriate for providers to monitor potential changes in case mix over time.

In those instances where one provider's outcomes are being compared with those from another group of providers, however, risk adjustment is nearly always appropriate. Owing to the inexactness of risk adjustment, common sense and good judgment are required in interpreting risk-adjusted outcome comparisons. Developmental work is needed on ways to introduce and exercise sound judgment in interpreting risk-adjusted findings in the quality-improvement field. Although methodological research and improved data sets are also needed, we must work diligently at developing ways that knowledgeable clinicians and methodologists can be brought together to make reasonable decisions so that risk-adjusted findings are properly interpreted and utilized for CQI purposes.

The results reported in this article indicate that an innovative program such as the TNHP can have an impact on general or global outcomes (such as hospitalization rates). The findings also suggest that the reduction in hospitalization occurred, at least in part, through focused programs that influence particular types of outcomes (such as specific changes in functioning) by altering processes of care (e.g., approaches to rehabilitation and restoration) and structure (e.g., composition of an interdisciplinary team). Equally important, independent of the reduction in hospitalization rates, the findings imply greater attentiveness to (several dimensions of) the quality of patient life. In assessing quality for purposes of program evaluation (such as the TNHP evaluation) or for purposes of quality assurance (such as CQI within a provider setting or system), proceeding along these lines—investigating general outcomes and also investigating reasons for the general findings in terms of specific outcomes, processes, and structural factors—can be useful. Causality (between global and specific outcomes, between outcomes and process, or between outcomes and structure) can rarely be proven definitively in such analyses (i.e., using data and statistical methods alone). However, a combination of statistical analyses, based on precise data, well-specified measures, and a rigourous framework; knowledge of program operations; clinical reasoning; and objective evaluation of patterns of findings, can produce useful conclusions. In the context of such applications, we would benefit from seeking meaningful patterns of findings across a profile of outcome measures, rather than focusing on a single outcome. At a minimum, we should concentrate on more extreme findings, such as highly exemplary outcomes or egregious problems, rather than on marginal differences between patient groups being compared. When attention is focused on larger differences, the likelihood is less that somewhat imprecise risk adjustment is the reason for such differences. In all, by specifying appropriate outcomes, using frameworks or taxonomies such as those introduced earlier, and collecting information on the specified outcomes and related processes and structure, it is possible to evaluate effectiveness and draw inferences about the reasons for effectiveness of health care.

Technical Notes

Illustrative Logistic Regression Model

An illustrative logistic regression model used to adjust TNH-CNH hospitalization rate differences for case mix is presented in Table 6. The adjusted odds ratio in this model is used to compute the (TNH/CNH) difference in hospitalization rates. The top portion of the table provides information on unadjusted hospitalization rates (see also the first row of Table 1). The lower portion of Table 6 presents logistic regression coefficients for risk factors and treatment variables. The three treatment variables consist of dichotomies corresponding to whether the patient was admitted to a TNH, admitted during the post- or intervention period, and an interaction corresponding to whether the patient was both a TNH patient and admitted during the intervention period. The coefficient of the interaction term was transformed into the ratio of adjusted TNH and CNH after-before odds ratios (using e, where b is the coefficient of the interaction term in the logistic regression). It is this coefficient that reflects the contribution of the TNHP to hospitalization rates, taking into consideration: (1) the risk factors in the logistic regression model; (2) the potential influences of extant hospitalization trends over time (the pre/post dichotomy); and (3) overall differences in TNHs relative to CNHs (i.e., the CNH/TNH dichotomy). The coefficients for the CNH/TNH and pre/post dichotomies are both positive, indicating that TNHs had a higher overall hospitalization rate than CNHs and that pooled hospitalization rates in general were increasing between baseline and the intervention period, taking hospitalization risk factors into consideration. However, the significant negative coefficient for the interaction term indicates that the TNHP had a significant negative impact on hospitalization (i.e., reduced hospitalization), taking into consideration hospital risk factors and the extant trend in hospitalization as well as the (hospitalization) differences between TNHs and CNHs.

Table 6

Patient-Level Logistic Regression Results for Hospitalization Within 3 Months for TNH and CNH Patients, Before and After the TNHP Pooled

Unadjusted Hospitalization Within 3 Months1		TNH		CNH
Hospitalization Rate in Before Period (Percent)		19.3		13.2
Hospitalization Rate in After Period (Percent)		12.3		18.1
After/Before Odds Ratio Unadjusted		0.59		1.45
Significance of Unadjusted Odds Ratio		.015		.130
Significance of Difference in Unadjusted Odds Ratio			.007
Logistic Regression R2: .1452
Significance: <.0012
Percent of Cases Correctly Classified: 85.2
Independent Variables	Coefficients		Odds Ratios		Significance3
TNH (Versus CNH)4	0.728		2.07		.028
Post (Versus Pre)4	0.564		1.76		.029
TNH × Post-Interaction4	-0.981		0.37		.005
Discharged to Community Within 3 Months	-2.307		0.10		<.001
Died Within 3 Months	-1.973		0.14		<.001
Number of Routine Medications	0.081		1.08		<.001
Female	-0.393		0.68		.004
Urinary Incontinence or Catheter	-0.319		0.73		.018
Walking Disability	0.396		1.49		.027
Skilled Care Case-Mix Index5	0.115		1.12		.062
Requires Oxygen	0.331		1.39		.092
Blood Disorder	0.947		2.58		.131
Hypertension	0.185		1.20		.131
Married	0.220		1.25		.111
Intercept	-2.402		0.09
Significance of Difference in Adjusted Odds Ratios:	.0056

Outcome variable = 1 if the patient was hospitalized within 3 months of nursing home admission, and 0 otherwise. The unadjusted odds ratios are based on separate logistic regression models (for TNH and CNH patients) with only the before/after dichotomy present. If the coefficient for this dichotomy is b, exp (b) is the estimate of the odds ratio. The significance level is the p-value for the chi-square test corresponding to the coefficient. The significance of the difference in unadjusted odds ratios is actually the significance for the ratio of the odds ratios.

The R2 is analogous to the R2 in ordinary least squares regression and is given by (X2 - 2p)/(-2L0), where X2 is the overall chi-square for the model, p is the number of independent variables, and L0 is the log-likelihood with only the intercept in the model. The significance level is the p-value for the overall chi-square.

Significance levels correspond to the chi-square tests for the respective independent variables, as described above.

The three treatment variables are dichotomies that correspond to CNH (0) versus TNH (1), pre (0) versus post (1), and the interaction of these two variables.

An index reflecting expected need (time required) for skilled nursing services.

This is actually the significance of the ratio of the adjusted odds ratios. It is exp (b), where b is the coefficient of the (TNHP×post) interaction variable in the full logistic regression model described in footnote 4 of Table 1. Its significance is that associated with b in the logistic regression equation.

NOTES: TNH is teaching nursing home. CNH is comparison nursing home. Retrospective and prospective admission sample sizes are presented in the article.

SOURCE: Primary data collected by nursing home and research staff on Teaching Nursing Home Program study patients (from TNHs and CNHs).

Risk factors that had a negative association with hospitalization included discharge to community in 3 months, mortality within 3 months, female, and incontinence or urinary catheter. The negative association between hospitalization and incontinence/catheter may occur because these conditions are relatively prevalent in nursing homes and require continual and diligent monitoring of nursing home residents. This may increase the likelihood of early prevention of conditions that might otherwise lead to hospitalization before such conditions reach an exacerbated state that requires hospitalization. Risk factors that are positively associated with hospitalization include number of routine medications prescribed for a patient at time of admission, disability in walking, an index reflecting the intensity of skilled nursing care needs, oxygen therapy, blood disorders, hypertension, and an indication of whether the patient is married. The nature of the positive association between most of these factors and hospitalization is evident, although the relationship between marital status and hospitalization is unclear (it may be that marital status serves as a surrogate for other physiologic factors). Risk factors were retained in multivariate adjustment models if their coefficients were significant at p < .15, since we were interested in compensating for as many such factors as reasonably possible. Despite the fact that the logistic regression model in Table 6 contains a reasonable number of risk factors, the proportion of variance explained in hospitalization rates was not substantial (14.5 percent). This is consistent with other studies that have analyzed risk factors for hospitalization, (Barker et al., 1987; Anderson and Steinberg, 1984; Gooding and Jette, 1985) indicating that the state of our empirically validated knowledge of risk factors for hospitalization of nursing home patients is incomplete.

Cost Differences During the Intervention Period

Hospital and nursing home care costs during the intervention period were estimated for fixed periods of 3 months and 6 months. Analyses included patients in the prospective samples who were either institutionalized for the full 3- or 6-month period after admission, or discharged to the community and followed for the remainder of the time until the 3- or 6-month followup period was completed. (The before-after hospitalization comparisons in Table 1 did not include community followup hospitalizations since community followup data were not available [from retrospectively collected data] for the before period.) Using the medical care component of the consumer price index, all costs were adjusted to the same base year (1986). Hospital costs were estimated by multiplying the number of hospital days by a per day hospital rate for each patient (in the prospective samples). The unadjusted and case-mix-adjusted results in Table 7 indicate TNH patients were less costly in terms of hospital care than CNH patients. Case-mix adjustment increased the TNH/CNH hospital cost differences in the intervention period, reflecting the aforementioned tendency of TNH patients to be at greater risk of hospitalization and longer hospital stays.

Table 7

Cost of Hospital and Nursing Home Care for TNH and CNH Patients, Adjusted for Case Mix

Institutional Cost per Patient for Fixed Time Periods	TNH Mean	CNH Mean	Unadjusted Mean Difference	Unadjusted Significance2	Case-Mix-Adjusted Mean Difference3	Case-Mix-Adjusted Significance3
Hospital Care
3 Months	$1,396	$2,110	-$714	.003	-$759	.002
6 Months	2,502	3,533	-1,031	.020	-1,220	.007
TNH/CNH Care
3 Months	3,512	3,598	-86	.407	229	.010
6 Months	5,341	5,648	-307	.163	460	.015
Other Nursing Home Care
3 Months	169	115	54	.022	31	.212
6 Months	421	360	61	.305	-16	.792
All Hospital and Nursing Home Care
3 Months	5,077	5,823	-746	.002	-533	.031
6 Months	8,264	9,541	-1,277	.006	-850	.067

Institutional cost results are based on samples of 1,103 TNH and 1,027 CNH patients for whom a full 6 months of followup data were available, including community or other-institution followup for discharged patients.

The unadjusted significance level for each cost variable (continuous variables) is the coefficient in an ordinary regression model using only the 0/1 TNH indicator as an independent variable.

The mean difference for the cost variable was adjusted using ordinary regression. The significance for the adjusted mean difference is the significance of the coefficient of the TNH versus CNH dichotomy in an ordinary regression model with case-mix covariates in the model.

NOTES: TNH is teaching nursing home. CNH is comparison nursing home.

SOURCE: Primary data collected by nursing home and research staff on Teaching Nursing Home Program study patients (from TNHs and CNHs).

Nursing home cost for each patient was computed by multiplying the inflation-adjusted per day cost (operating cost was used, exclusive of capital and related costs) for each nursing home patient by the number of days of his/her stay. Although unadjusted 3-month and 6-month differences between TNHs and CNHs were insignificant, the significantly higher case-mix-adjusted costs for TNHs reflect the fact that CNH patients were characterized more frequently by chronic conditions that result in longer nursing home stays. The case-mix-adjusted analyses of cost associated with other nursing home care (other than a TNH or CNH) within 3 and 6 months of admission did not yield significant differences between TNH and CNH patients.

Of the six lowest-cost facilities in terms of hospital care at 6 months, four were TNHs, with the remaining two TNHs ranked ninth and tenth. Of the nine lowest-cost facilities in terms of total institutional cost in 6 months, six were TNHs. Analogous TNH/CNH differences were found for hospitalization rates and hospital days by individual facility. In all, as with the before/after analyses, the results were reasonably pervasive at the facility level.

The findings presented in Table 7 entailed adjusting for payer source as part of the case-mix-adjustment process, but not adjusting for community hospital occupancy rate. When community hospital occupancy was included, the results remained basically the same, except the cost of other nursing home care in 6 months became significantly lower for TNHs relative to CNHs. Additional analyses were conducted by stratifying by payer source. In general, the results in Table 7 persist across different patient types, taking payer source and community hospital occupancy rates into consideration. Although generally in the same direction, the results are less strong for Medicaid or long-stay patients.