Potential organ-donor supply and efficiency of organ procurement organizations.

The authors estimated the supply of organ donors in the U.S. and also according to organ procurement organizations (OPOs). They estimated the number of donors in the U.S. to be 16,796. Estimates of the number of potential donors for each OPO were used to calculate the level of donor efficiency (actual donors as a percent of potential donors). Overall, donor efficiency for OPOs was 35 percent; the majority was between 30- and 40-percent efficient. Although there is room to improve donor efficiency in the U.S., even a substantial improvement will not meet the Nation's demand for organs.

Introduction

Although over 70,000 Americans awaited a solid organ transplant in 2000, only 22,953 transplants were performed (United Network for Organ Sharing, 2003). It is generally believed that this disparity is due to the inability of the organ procurement system to tap more efficiently the potential supply of organs from deceased individuals who are medically suitable for donation. Prior estimates of the donor supply range between 6,900 and 15,000 potential donors. However, these estimates are based on data from the late 1980s or early 1990s (Garrison, Bentley, and Raque, 1991; Evans, Orians, and Ascher, 1992; Nathan et al., 1991; Ojo et al., 1999), from a limited number of service areas (Nathan et al., 1991; Garrison, Bentley, and Raque, 1991), from extrapolations generated from data on specific organs (Nathan et al., 1991; Ojo et al., 1999), or from death records (Evans, Orians, and Ascher, 1992), which lack sensitivity with respect to identifying medically suitable organ donors. A more current and valid estimate of the potential number of organ donors is needed. Moreover, the lack of a reasonable estimate of donor potential also makes it difficult to evaluate the performance of organizations that are responsible for procuring organs in the United States.

OPOs are not-for-profit entities, assigned to specific geographical service areas, that coordinate the acquisition and distribution of organs (U.S. General Accounting Office, 1998). CMS sets performance standards for OPOs according to five criteria: organ donors, kidneys recovered, kidneys transplanted, extrarenal organs recovered, and extrarenal organs transplanted. For each criterion, performance is expressed as a function of the population within the OPO service area (for example, the number of organ donors per million population (DPMP).

The approach used to assess OPO performance has been criticized because it assessed performance based on the total population in a service area rather than the potential number of donors (Shafer, Kappel, and Heinrichs, 1997; U.S. General Accounting Office, 1998; Christiansen et al., 1998; Luskin et al., 1999; Ojo et al., 1999). Although a population-based measure is readily available and inexpensive to produce (U.S. General Accounting Office, 1997), it is problematic because donor pools can differ across service areas due to variation in characteristics that define potential organ donors (Ojo et al., 1999). Both OPOs and CMS recognize the limitations of the approach used to assess performance and both parties are interested in identifying viable alternatives (U.S. General Accounting Office, 1997). Alternative performance measures, such as measuring procurement and transplantation as a function of the number of deaths, the number of potential donors based on medical record review, and the number of potential donors based on statistical estimates for each ser vice area have been proposed (U.S. General Accounting Office, 1998).

We estimated the potential supply of organ donors using a previously validated prediction model that estimated the number of potential donors for individual hospitals based on publicly available data (Christiansen, Gortmaker, and Williams, 1998). We aggregated the hospital estimates and determined the overall potential for the U.S. and for each OPO service area. We evaluated the performance of OPOs by comparing their estimates of donor efficiency (actual donors per estimated potential donors) and we compared donor efficiency with the measure, DPMP, previously used by CMS to certify OPOs. Finally, we propose a strategy for certification of OPOs based on donor efficiency.

Methods

Study Population

We used 1998 American Hospital Association (AHA) data to select hospitals for analysis based on previously described criteria (Christiansen, Gortmaker, and Williams, 1998), as shown in Table 1. Eligible hospitals included non-Federal, acute-care facilities with an intensive care unit and at least 50 beds (N = 3,120).

Table 1

Hospital Exclusion Criteria: 1998

Reason for Exclusion	Number of Hospitals
Total Hospitals in American Hospital Association Data Set	6,247
Total Number of Eligible Hospitals	3,120
Type of Hospital1	1,089
Federal Government Hospital	280
Located in Territories Not Assigned to an Organ Procurement Organization Service Area2	4
No Intensive Care Unit	2,094
Number of Staffed Beds < 50	1,899

Excluded the following types: hospital unit of an institution (prison hospital, college infirmary, etc.); hospital unit within an institution for the mentally retarded; psychiatric; tuberculosis and other respiratory diseases; eye, ear, nose, and throat; rehabilitation; orthopedic; other specialty treatment; children's hospital unit of an institution; children's psychiatric; children's tuberculosis and other respiratory disease; children's eye, ear, nose, and throat; children's rehabilitation; children's orthopedic; children's other specialty; institution for mental retardation; and alcoholism and other chemical dependency.

Excluded Marshall Islands, Guam, and American Samoa.

SOURCES: Guadagnoli, E., Harvard Medical School, Christiansen, C.L., Boston University and the Edith Nourse Rogers Memorial Veterans Hospital, and Beasley, C.L., Institute for Healthcare Improvement, 2003.

Estimating Potential Number of Donors

For three OPO service areas, Christiansen and colleagues (1998) used publicly available data on the characteristics of hospitals (bed size, case-mix index, status as a trauma center, and medical school affiliation) and hierarchical Poisson regression (Christiansen and Morris 1997a) to develop a model to estimate the potential number of donors for individual hospitals located in the service areas. They then aggregated the hospital estimates to produce an estimate of the number of potential donors for each OPO. The criterion for their model (number of potential organ donors per hospital) was obtained from a review of hospital records. A potential donor was defined as a medically suitable patient who had conditions consistent with brain death, as defined by the President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research (1981). Patients were excluded as potential donors if they were age 70 or over or exhibited one or more International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) (Centers for Disease Control and Prevention, 2003) codes that are contraindications for organ donation. A list of the ICD-9-CM codes appears in Table 2. These exclusion criteria reflected practice within each of the three service areas studied. Evaluation of the model suggested that it would predict well when used in service areas other than those used in the study (Christiansen, Gortmaker, and Williams, 1998).

Table 2

ICD-9-CM Exclusion Criteria

Code	Condition
010.00-018.9	Tuberculosis
042.0-044.9	Human Immunodeficiency Virus (HIV) Infection with Specified Conditions
46.1	Jakob-Creutzfeldt Disease
54.5	Herpetic Septicemia
070.2-070.3	Hepatitis B Surface Antigen
140.0-190.9	Malignant Neoplasm
192-208.9	Malignant Neoplasm of Other and Unspecified Parts of the Nervous System
284.0-284.9	Aplastic Anemia
286.0-286.4	Hemophilia
288	Agranulocytosis
710.0-710.9	Diffuse Diseases of Connective Tissue
765.00-765.03	Extreme Immaturity
765.1	Preterm Infant of Birth Weight 1,000-2,499 Grams
780.6	Pyrexia of Unknown Origin
795.8	Positive Serologic or Viral Culture Findings for HIV

NOTE: ICD-9-CM is International Classification of Diseases, Ninth Revision, Classified Modification.

SOURCES: Guadagnoli, E., Harvard Medical School, Christiansen, C.L., Boston University and the Edith Nourse Rogers Memorial Veterans Hospital, and Beasley, C.L., Institute for Healthcare Improvement, 2003.

We estimated the number of potential donors for an OPO service area comprised of i = 1, …, k hospitals using:

Trauma center missing refers to an indicator equal to 1 if information on the trauma center status of a hospital was missing or equal to 0, otherwise. The total number of potential donors per OPO was calculated as Θ=ΣΘi and the estimate of the standard error associated with the prediction was defined as ω=√(Σ(Θi2/2.58)+Θi) (Christiansen et al., 1998).

We obtained the number of staffed hospital beds from the AHA (1998) data set or, if missing, from the Hospital Blue Book (1998). We obtained hospital case-mix values from a publicly available file (Centers for Medicare & Medicaid Services, 2003). The case-mix index is a measure of the costliness of cases treated by a hospital relative to that of the national average for all inpatients covered by Medicare, using diagnostic-related group weights as a measure of the relative costliness of cases (Christiansen et al., 1998). We were unable to link 98 hospitals to data in the case-mix file. For these hospitals and for six whose case-mix values were typographical errors, we used the mean case-mix value (1.37). The variables, trauma center (whether the hospital was a certified trauma center) and medical school affiliation, were obtained from the AHA data set; both were represented by dichotomous variables coded as 0 (no) or 1 (yes). If trauma center status was missing from the AHA data set, we obtained it from the Hospital Blue Book (1998) for all, but one hospital.

Statistical Analysis

Using the equation previously mentioned, we calculated the potential number of donors per eligible hospital for 1998. Based on the hospital's geographic location, we calculated the potential number of donors and the 95 percent confidence interval (CI) for each OPO service area. We obtained the actual number of donors and the population for each OPO service area for 1998 (United Network for Organ Sharing, 2003). For each OPO, we calculated a measure of donor efficiency (number of actual donors ÷ number of potential donors * 100) and DPMP, the standard of performance used by CMS. We calculated the 95-percent CI for the efficiency measure, using the Delta method (Casella and Berger, 1990). This method can result in a CI that is outside the bounds of the parameter; in this case, the ratio is bounded by 0 and 1. We truncated a CI at 1 for one OPO. We assessed the degree of association between donor efficiency and DPMP by calculating the Spearman rank-order correlation (Marascuilo and McSweeney, 1977).

Finally, we describe how donor efficiency estimates might be used to evaluate performance of OPOs for the purpose of certification. This approach takes into account that donor efficiency is based on an estimate of donor potential, and therefore there is a level of uncertainty associated with the efficiency measure. We used probability models (Christiansen and Morris, 1997b) for the unknown parameter representing the number of potential donors to calculate the likelihood that an OPO meets a prespecified performance criterion. Then we demonstrate how this information might be used to evaluate performance.

Results

Potential Donors and Donor Efficiency

Table 3 lists the estimated number of potential donors, the number of actual donors, donor efficiency, and DPMP according to OPO. For 1998, we estimated that there were 16,796 (95 percent CI = 16,105, 17,481) potential donors in the U.S. Across OPOs, the number of potential donors ranged from a high of 937 (95 percent CI = 754, 1121) to a low of 53 (95 percent CI = 23, 83).

Table 3

Number of Potential Donors (95-Percent Confidence Interval), Actual Number of Donors, and Donor Efficiency (95-Percent Confidence Interval)

OPO	Potential Donors	95-Percent CI Lower	CI Upper	Actual Donors	Donor Efficiency	95-Percent CI Lower	CI Upper	DPMP	Prob (Donor Efficiency < 25 Percent)
Total	16,796	16,105	17,481	5,793	34.5	24.2	44.8	21.3	—
1	53	23	83	43	81.6	34.9	100.0	21.7	0.00
2	55	31	79	36	65.5	36.9	94.0	9.1	0.00
3	107	62	153	70	65.1	37.5	92.8	24.4	0.00
4	234	174	293	152	65.1	48.5	81.6	23.9	0.00
5	60	21	98	31	52.1	18.2	85.9	21.4	0.00
6	216	117	315	112	51.9	28.1	75.6	40.7	0.00
7	111	67	154	56	50.7	30.9	70.4	29.4	0.00
8	389	287	491	190	48.9	36.0	61.7	25.9	0.00
9	113	55	171	52	46.0	22.5	69.6	25.6	0.00
10	201	140	261	92	45.9	32.0	59.7	21.4	0.00
11	389	293	484	178	45.8	34.6	57.1	24.3	0.00
12	69	38	99	31	45.1	25.1	65.2	17.9	0.00
13	664	530	798	298	44.9	35.8	53.9	28.9	0.00
14	388	290	487	174	44.8	33.4	56.2	28.1	0.00
15	175	84	265	78	44.6	21.5	67.7	30.4	0.00
16	443	357	528	197	44.5	35.9	53.1	17.7	0.00
17	282	198	366	123	43.7	30.7	56.7	24.1	0.00
18	299	215	383	124	41.5	29.9	53.1	29.1	0.00
19	111	61	161	45	40.6	22.4	58.8	22.5	0.00
20	192	116	269	77	40.0	24.0	56.0	26.5	0.00
21	197	129	264	78	39.7	26.1	53.3	25.3	0.00
22	224	152	296	86	38.4	26.1	50.8	26.7	0.00
23	197	135	259	74	37.6	25.7	49.4	17.8	0.00
24	331	241	422	124	37.4	27.2	47.7	24.7	0.00
25	140	76	203	52	37.2	20.3	54.1	19.9	0.02
26	442	346	538	162	36.6	28.7	44.6	21.5	0.00
27	251	173	329	92	36.6	25.3	48.0	20.0	0.00
28	247	141	352	87	35.3	20.2	50.3	23.6	0.03
29	316	235	397	109	34.5	25.6	43.3	15.8	0.00
30	212	140	284	72	33.9	22.5	45.4	21.7	0.02
31	186	113	260	63	33.9	20.5	47.2	30.2	0.04
32	354	257	451	117	33.0	24.0	42.1	26.0	0.01
33	303	215	391	100	33.0	23.4	42.7	25.9	0.01
34	77	39	115	25	32.5	16.4	48.7	21.8	0.12
35	250	174	326	80	32.0	22.3	41.7	18.1	0.03
36	634	497	772	200	31.5	24.7	38.4	21.0	0.01
37	326	214	439	101	31.0	20.3	41.7	20.2	0.08
38	475	345	604	146	30.7	22.4	39.1	26.1	0.05
39	350	257	443	106	30.3	22.2	38.3	16.7	0.06
40	212	128	296	64	30.1	18.2	42.1	22.4	0.15
41	240	154	326	71	29.6	18.9	40.2	20.5	0.16
42	163	92	233	47	28.9	16.3	41.4	19.1	0.24
43	236	139	333	68	28.8	17.0	40.6	22.1	0.23
44	106	45	167	30	28.2	12.0	44.5	15.6	0.32
45	303	234	372	85	28.0	21.6	34.4	19.5	0.14
46	201	125	276	56	27.9	17.4	38.4	22.4	0.27
47	98	45	150	27	27.7	12.8	42.6	18.2	0.35
48	740	583	896	199	26.9	21.2	32.6	18.0	0.24
49	154	86	222	41	26.7	14.8	38.5	18.1	0.39
50	937	754	1121	247	26.4	21.2	31.5	20.7	0.29
51	868	718	1017	228	26.3	21.8	30.8	19.7	0.28
52	261	165	357	67	25.7	16.2	35.1	14.6	0.44
53	196	121	271	50	25.6	15.8	35.4	19.0	0.46
54	514	408	620	130	25.3	20.1	30.5	12.7	0.46
55	343	222	465	86	25.0	16.2	33.9	19.8	0.49
56	96	36	156	24	25.0	9.4	40.6	15.8	0.50
57	142	69	215	34	24.0	11.7	36.4	14.7	0.56
58	355	255	455	83	23.4	16.8	30.0	18.0	0.68
59	142	73	210	33	23.3	12.0	34.6	19.1	0.61
60	187	95	279	42	22.5	11.4	33.5	19.5	0.66
61	77	36	118	16	20.8	9.7	31.9	13.5	0.73
62	162	103	222	32	19.7	12.5	26.9	15.1	0.87

Donor efficiency is the number of actual donors divided by the number of potential donors times 100.

NOTES: CI is confidence interval. OPO is organ procurement organization. DPMP is donors per million population.

SOURCES: Guadagnoli, E., Harvard Medical School, Christiansen, C.L., Boston University and the Edith Nourse Rogers Memorial Veterans Hospital, and Beasley, C.L., Institute for Healthcare Improvement, 2003.

Donor efficiency ranged from a high of 81.6 (95 percent CI = 34.9 percent, 100.0 percent) to a low of 19.7 (95 percent CI = 12.5 percent, 26.9 percent). For more than one-half of the OPOs (53 percent), donor efficiency ranged between 30 and 40 percent; for four OPOs, efficiency was nearly two-thirds or greater (Table 3). For three of these four OPOs, the donor efficiency estimates have wide CIs due to the small number of eligible hospitals (15, 17, and 23 hospitals) within each service area, suggesting that there is a high level of uncertainty associated with the estimate of donor efficiency for these OPOs. Six OPOs were less than 25 percent efficient (Table 3).

Figure 1 displays donor efficiency plotted against DPMP. The Spearman rank-order correlation between the measures was moderate (r=0.57).

Figure 1

Donor Efficiency of an OPO, by Donors per Million Population

Evaluation of OPO Performance

Because the method used to derive donor efficiency is based on an estimate of donor potential, there is a level of uncertainty associated with the efficiency measure. However, one can calculate the probability that donor efficiency is greater than or equal to a specific criterion (say, 25 percent or one out of four potential donors actually donates) by estimating distributions for the unknown parameter, the number of potential donors. We assume that the true number of potential donors at an OPO follows an approximate Gaussian distribution (Colton, 1974) with a mean equal to our estimated number of potential donors and an estimated standard deviation previously noted. Based on these distributions, one can calculate the probability that the true donor efficiency rate is less than or more than a specific criterion. For demonstration purposes, we report the probability that the true rate is less than 25 percent. This probability, in conjunction with the measure of donor efficiency, might be used for purposes of evaluating OPO performance.

For example, if the probability is more than 50 percent (we are more certain than uncertain) that the donor efficiency for an OPO is 25 percent or less, then the OPO might be considered a poor performer. If the probability is less than 50 percent (we are more uncertain than certain) that the OPO is a poor performer, then CMS might reserve judgment with respect to imposing sanctions on the OPO due to a lack of certainty regarding its performance. In this case, CMS might decide to review hospital records in order to determine, with more certainty, the actual performance of the OPO. The selection of a probability criterion more than 50 percent is arbitrary. A higher value (for example, 75 percent) might be used if one wished to be more conservative with respect to imposing sanctions on OPOs.

The probability that an OPO's donor efficiency rate is less than 25 percent appears in Table 3 (donor efficiency <25 percent). Again, this is an artificial standard chosen here only to demonstrate how these estimates can be used to assess OPO performance. For example, the probability is 0.61 that OPO 59 had a true donor efficiency rate of less than 25 percent (Table 3). Based on a criterion of 25 percent, we would be more than 50-percent certain that 6 of the 62 OPOs (OPOs 57 to 62) were poor performers. In the case of OPO 56, where the probability (donor efficiency < 25 percent) =0.50), we might reserve judgment on performance, pending a review of hospital records. If we wished to be more conservative regarding the identification of poor performers and used a 75-percent level of certainty, then one of the 62 OPOs (OPO 62) would be labeled a poor performer. Table 4 provides a summary of the number of OPOs considered poor performers, according to various levels of donor efficiency and various levels of certainty.

Table 4

Number of Organ Procurement Organizations Considered Poor Performers According to Various Levels of Donor Efficiency and Various Levels of Certainty

True Donor Efficiency Rate Less Than	Probability That Donor Efficiency Rate Is < True Rate
	0.50	0.75	0.95
0.25	6	1	0
0.50	55	50	34
0.75	61	59	54

SOURCES: Guadagnoli, E., Harvard Medical School, Christiansen, C.L., Boston University and the Edith Nourse Rogers Memorial Veterans Hospital, and Beasley, C.L., Institute for Healthcare Improvement, 2003.

Discussion

Our estimate of the potential donor pool in the U.S., 16,796 (95 percent CI = 16,105; 17,481), is in line with some past estimates derived from a variety of estimation methods. Considering that 5,793 donors provided organs for transplantation in 1998 (35 percent overall donor efficiency), room for improvement exists. If all OPOs that performed with less than 50 percent efficiency increased their performance to 50 percent and all remaining OPOs maintained their level of efficiency, we might expect 8,479 donors. Moreover, if all OPOs were able to attain a 75-percent level of efficiency, the number of donors would increase to 12,598. However, considering that 41,000 registrations were added to the waiting list in 2000 (Organ Procurement and Transplant Network, 2003), and assuming that each donor yields, on average, three organs for transplantation (U.S. General Accounting Office, 1998), the demand for organs would still exceed the potential supply. The addition of living donors to this pool would bring us closer to meeting demand; nevertheless, this calculation ignores the backlog of patients already on the waiting list.

Although the supply of cadaveric donors may never satisfy the demand for organs for transplantation, cadaveric donors will remain an important, if not the most important, supply of organs for the foreseeable future. Therefore, efforts to evaluate and improve the performance of OPOs are essential. The availability of an estimate of donor potential should allow policymakers to contemplate what a reasonable standard of performance is for OPOs. Past work suggests that 69 percent of the public would be likely to donate their own organs (Gallup Organization, 1993); refusal to give consent for donation ranges between 30 and 50 percent (Gortmaker et al., 1996; Ojo et al., 1999; Capron, 2001). Past work also suggests that consent rates of 74 percent are possible when hospital and OPO staff engage in specific processes when requesting organ donation (Cutler et al., 1993; Klieger et al., 1994; Beasley, Capposella, and Brigham, 1997; Gortmaker et al., 1998; Siminoff et al., 2001). To date, significant and lasting increases in rates of consent have not been reported, perhaps related to the difficulty of orchestrating the donation process consistently to include the recommended process steps. Therefore, policymakers should not assume that 100 percent efficiency is possible. Considering the proportion of the public that is willing to donate and the sensitive context in which the donation request takes place, perhaps 50 percent efficiency is the upper limit for the expected performance of OPOs. Only 7 of 62 OPOs met or exceeded this level in 1998.

The availability of a measure of donor efficiency also allows policymakers to identify OPOs that are good performers. Investigation into the practices of these OPOs might identify strategies and processes that could be transferred to those OPOs whose efficiency is low. A recent survey has indicated that there is wide divergence in consent practices among OPOs (Wendler and Dickert, 2001).

The measure of donor efficiency relies on statistical estimates, and therefore, there is some level of uncertainty associated with the donor efficiency rate for each OPO. Given this uncertainty, commonly used statistical approaches to assess performance, such as ranking OPOs based on donor efficiency rates or comparing the donor-efficiency rate for an OPO with the mean rate for all OPOs, are not appropriate. We describe a method that allows policymakers to evaluate the probability that an OPO's performance is actually below a particular standard. The utility of this approach relies on the ability of policymakers and other interested parties to agree on an acceptable standard and on a probability criterion to evaluate the standard. Similar methods for analyzing performance have been used by the Department of Veterans Affairs to report hospital performance with respect to a number of outcomes (Burgess et al., 2000; Burgess, Lourdes, and West, 2000. Nevertheless, the donor efficiency rate could, at a minimum, be a useful tool for internal quality improvement activities within an OPO.

The association between the donor efficiency measure and the current standard used by CMS, DPMP, was moderate. However, the use of the donor efficiency measure described in this article, in combination with an agreed-upon performance criterion and a probability statement that the criterion is met, improves the interpretation of OPO performance and provides a fairer assessment of performance than the approach currently in use. The latter assumes that donor pools do not vary with respect to the characteristics that define donors.

This study has several potential limitations. First, the model used to predict the potential number of donors was developed using data from three OPOs. Validation at the national level using more current data should be performed. The need to review hospital records for validation would make this an expensive undertaking; however, CMS has sponsored the abstraction of data from hospitals throughout the country (Marciniak et al., 1998). Second, although CMS used five criteria to evaluate OPO performance, we examined only one, primarily because the variables used to predict the other criteria are likely to differ from those we used to predict the number of potential donors. For example, rates of recovery for specific organs might vary due to the skill of OPO staff to clinically manage potential donors or to the characteristics of the donor population, such as the proportion of patients with hypertension.

Although the actual number of donors in the U.S. represents about one-third of the potential number of donors, the demand for organs is not likely to be met from cadaveric donors. Nevertheless, this should not stop efforts to increase the efficiency of OPOs. The approach we describe can be used to measure the performance of OPOs and to identify good and bad performance outliers. In addition, we need to pursue methods to increase the number of organs for transplantation. Some of these methods might relate to OPO practices and cadaveric donors (e.g., expanding the scope of legal powers of attorney for health care) (Wendler and Dickert, 2001), to legislative interventions (e.g., allowing presumed consent and mandated consent) (Murray and Youngner, 1994), or to clinical innovations such as expanding the criteria and strategies for transplantation, for example, using older donors, splitting a liver for transplantation between two adults (Gridelli and Remuzzi, 2000), increasing the use of living donors (Matas et al., 2000; Levinsky, 2000; Abecassis et al., 2000), and exploring xenotransplantation and organ engineering (Chapman and Bloom, 2001). The challenge to meeting the demand for organs through all of these methods is related not only to the process of implementation, but also, and more important, to resolving the ethical issues that these approaches provoke.