Comparing the cost-effectiveness of short orthopedic missions in elective and relief situations in developing countries.

BACKGROUND: The earthquake that occurred in Haiti on 12 January 2010 elicited an unprecedented response from the American orthopedic community. Many small organizations, such as Operation Rainbow, were thrust into the unfamiliar environment of relief surgery, whereas they normally provide short elective reconstruction missions in developing countries. MATERIALS: Because of the chaotic nature of relief work, it was assumed that the organization's efforts would be less cost-effective than their usual elective work. To evaluate this conclusion, the present study was designed to compare the cost-effectiveness of the organization's usual elective missions with the emergency relief provided in the wake of the Haiti earthquake. RESULTS AND
CONCLUSIONS: The assumption that emergency costs would be higher was proven wrong, with estimates of $362 per disability-adjusted life-year (DALY) averted in the elective group, and $343 per DALY averted in the relief group.

Introduction

The last couple of decades have seen a significant increase in the number of groups and organizations providing elective surgical care of all sorts to low-income countries [1-4]. Sizes (from individuals to large organizations such as Interplast or Operation Smile) and affiliations (academic, religious, political) may vary, but almost all share basic principles of humanitarianism and volunteerism. Although models may differ from one organization to another, surgical missions are usually short, primarily focused on service provision rather than capacity building, and target people who could not be treated otherwise because of barriers such as costs, lack of local capacity, or lack of patient access [4, 5]. The benefits are many: premature death can be avoided (hernia surgery); quality of life is improved from a functional (cataract surgery, orthopedic surgery for trauma or congenital conditions) or cosmetic (congenital or acquired disfigurements) standpoint; some knowledge transfer can be provided to local health care providers, as can improvements in equipment and supplies; and there is the added benefit of the “feel good effect” for volunteers [5-7]. There are also drawbacks: short-term follow-up is usually in the hands of local providers, who are sometimes ill equipped or ill prepared to manage some complications; and long-term follow-up is often almost nonexistent, so the outcomes of surgical procedures are usually assumed rather than verified. Outcomes and cost-effectiveness of volunteer surgery can be documented in developed countries [8], but with few very rare exceptions, this is not the case in poorer countries [7]. This lack of documentation is unfortunate, considering that the magnitude of the problem, particularly the high burden of injuries, is widely recognized [9].

Operation Rainbow (http://www.operationrainbow.org) is a small non-governmental organization (NGO) without political or religious affiliation. It started in 1979 as a plastic surgery team and in 1989 an orthopedic surgery section was added, which has since been responsible for the bulk of the surgical activity, with more than 10,000 surgical procedures in over 10 countries, with over 50,000 patients seen, evaluated, and treated in the clinic. Operation Rainbow aims at providing free surgical care for the poor and has concentrated its activity mostly in Central and South America. The main focus has always been pediatrics, but because of increasing demand, more and more trauma cases, usually neglected, are treated. All three authors of this article have done numerous elective missions over the years and all were involved in the relief effort in the Dominican Republic and Haiti after the 2010 earthquake. Surgical teams are self-sufficient, providing surgeons, anesthesiologists, nurses, translators, and all surgical equipment and supplies. A typical team runs at least two, often three, operating rooms, and missions last usually 1 week: the first day is spent in clinic evaluating hundreds of patients, most of whom have been pre-screened by our local counterparts. Surgical cases are prioritized and scheduled, and the patient gets anesthesia clearance. The rest of the week is all surgery. Before the surgical teams depart, clear postoperative orders are left with the local surgeons, who do not hesitate to contact the volunteer surgeon if there is any problem. Teams try as much as possible to return each year to the same locations, as relationships have been established that facilitate surgical follow-up, year-long screening, and equipment preparation.

The earthquake in Haiti thrust the organization into an unfamiliar situation where teams are providing relief care to acutely injured patients in far less than ideal conditions, even by Operation Rainbow’s habitual standards. The government of the Dominican Republic, where we have regular elective missions, asked the organization to help with the rapidly increasing number of patients in a makeshift hospital in the border town of Jimani. The first team arrived 1 week after the cataclysm and was followed by two successive teams. Two more teams later worked in chaotic Port-au-Prince, as did so many other teams from other NGOs [10].

If the effectiveness of such relief or elective missions is poorly documented, their cost-effectiveness is even less so. Because of the lack of organization and coordination of the surgical response, it was assumed that the relief missions in Haiti would be less effective and cost-effective than the usual elective missions. The purpose of the present study was to compare these different sets of activities and to determine their relative cost-effectiveness.

Materials

To achieve comparable numbers in terms of total team size and patient workload, the last three consecutive Operation Rainbow elective missions (elective group), two in Nicaragua and one in the Dominican Republic between July 2009 and February 2010, were compared to the five missions in Haiti and the border town of Jimani, Dominican Republic. There was no attempt to obtain IRB approval for the present study. The sex, age, diagnosis, and procedure were available for all 117 patients (123 procedures) in the elective group. This group ran a total of 8 operating rooms, with a total number of volunteers of roughly 80 people. In Haiti, the total five operating rooms were manned by approximately 60 volunteers; data collection tools were not as exhaustive as for the elective group, but a total of 93 patients underwent surgery (130 procedures). Adequate data were available for approximately 80 patients; the rest were estimated on a pro-rata basis. The burden of musculoskeletal disease was calculated according to the Global Burden of Disease study [11], using disability-adjusted life-years (DALYs) as the unit of measurement, and the updated disability weights from the Disease Control priority project [12]. For conditions that do not have an assigned disability weight, a rough trade-off approach was used to determine a reasonable figure. As an example, untreated clubfoot was given the same disability weight as untreated cleft palate: 0.231. Because it was assumed that no procedure was life saving, all DALYs were attributable to years lived with disability (YLDs). The World Health Organization (WHO) guidelines for cost-effectiveness analysis were followed as closely as possible [13]. The effectiveness of interventions was calculated in terms of DALYs averted, as described in greater detail elsewhere [14, 15]. In summary, the burden of an untreated condition is adjusted by two additional factors: the likelihood of death or permanent disability if left untreated, and the likelihood of success of treatment, as illustrated in Table 1.

Table 1

Scoring system

	Weight
Severity of disease
>95% fatal without treatment	1.0
<95% and >50%	0.7
<50% and >5%	0.3
<5%	0
Effectiveness of treatment
>95% chance of cure	1.0
<95% and >50%	0.7
<50% and >5%	0.3
<5%	0

As an example: a 20-year-old female with a femur fracture entails a long-term 9.6 DALYs [35.02 adjusted years of life left at age 20 (YLLs) × 0.272 (disability weight for femur fracture)]. The effectiveness of treatment, measured in DALYs averted, is calculated by multiplying 9.6 DALYs by 0.7 (likelihood of permanent disability between 50 and 95%) and again by 0.7 (likelihood of treatment success between 50 and 95%) for a total of 4.7 DALYs averted. DALYs and DALYs averted were calculated for each patient, and results are summarized in Table 2, by condition. They were calculated only once for patients undergoing multiple procedures. Costing figures included only variable costs, from a provider standpoint. On elective missions, most volunteers pay for travel, lodging, and food while the organization covers some travel expenses, supplies, implants, and logistics. For the three elective missions the organization paid a total of $70,253, and volunteers paid $103,211 for a grand total of $173,464. For the Haitian missions, all costs were covered by the organization, and it was not possible to disaggregate costs by type, so only total costs were considered. The five missions incurred a total cost of $111,681.

Table 2

Burden of conditions and effectiveness of treatment

Elective				Haiti
	*Patients (procedures)	DALYs	DALYs averted	*Patients (procedures)	DALYs	DALYs averted
Stump revision	0			9 (14)	52	36
Wound debridement	0			20 (42)	76	37
UE amputation	0			1	10	10
LE amputation	1	13	9	6	74	74
Humerus #	7	38	26	5	24	15
Elbow #	0			4	26	15
Forearm #	4	31	19	7	18	12
Wrist/hand	1	4	2	1	3	1
Hip #	2	13	6	1	14	7
Femur #	12	141	69	4	56	27
Tibia #	10	82	57	7	60	43
Ankle #	2	13	8	3	20	12
Foot #	0			3	11	6
External fixator adjustment	0			10 (12)	28	6
Hard remov	4	9	2	6	20	9
Burns	0			2 (10)	25	12
Shoulder dislocation	1	5	2
DDH	6	49	24
TEV	32 (36)	396	182
Triple arthrodesis	4	21	15
Soft tissues	25 (27)	104	49
Others	6	21	10	4	10	4
Total	117 (123)	940	480 (51%)	93(130)	527	326 (62%)

DALY disability-adjusted life-year, UE upper extremity, LE lower extremity, DDH developmental dislocation of the hip, TEV talipes equinovarus

Results

The elective missions treated a total of 117 patients with 123 procedures for a variety of conditions, acute and chronic. Sixty of the patients were males and 57 were females, with an average age of 12.5 years (range: 4 months to 72 years). Most procedures were done for congenital malformations such as clubfoot or developmental dysplasia of the hip. Less than 10% of procedures were for internal fixation of acute fractures or chronic complications such as malunions or non-unions. It is estimated that of a potential total of 940 DALYs, 480 (51%) were averted by treatment. This was done at a cost of $173,464, giving a cost-effectiveness of $362 per DALY averted.

In Haiti, we estimate that 93 patients received 130 procedures, all for acute conditions. There were 44 males and 49 females, of mean age 17 years (range: 4 months to 82 years). The majority of procedures were for debridement, amputation, or stump revision. A few patients underwent application of external fixators, and only three had internal fixation, two of them in the later missions in Port-au-Prince. Of the potential burden of 527 DALYs, it is estimated that treatment averted 326 (62%). At a total cost of $111,681, this gives a cost-effectiveness of $343 per DALY averted.

Discussion

The uncertainty around some of the estimates, particularly in the Haiti group, precludes any meaningful statistical analysis. Nevertheless, even if the true cost-effectiveness figures are most certainly not exactly the same for the two groups, it is also unlikely that they are hugely different, as we had intuitively anticipated. We assumed that in a chaotic environment there were going to be more inefficiencies and waste. The pathology encountered, in terms of potential long-term disability, appears slightly higher in the elective group (8.1 DALYs/patient) than in the Haiti group (5.6 DALYs/patient). This likely reflects more cases carrying a low disability weight, such as soft tissue wounds and infections, in Haiti. The burden of injury and disability averted in Haiti is actually somewhat underestimated, as numerous bedside/ward procedures, such as fracture reduction and casting, external fixator adjustments/removal, etc., are not included for lack of documentation. Furthermore, in Haiti 37 additional procedures were performed on 93 patients (40%), whereas only 6 additional procedures were done on the 117 elective patients (5%). Furthermore, it appears that interventions in Haiti were slightly more effective: 62% DALYs averted, versus 51% in the elective group. This apparent difference might not be significant, but we were expecting a big difference, in the opposite direction.

There are numerous barriers to the provision of safe, effective, and cost-effective surgical care in developing countries, particularly in relief situations. The recent efforts in Haiti have been impaired by well-documented lack of collaboration and coordination [10, 16, 17]. A cost-effectiveness analysis is only one tool used to assess the performance of any given activity. By no means do we suggest it should supersede humanitarian, organizational, or even political considerations in the process of resource allocation. It can be helpful to compare performance within an organization over time or between organizations. We were not able to identify comparable studies in the literature. At around $350 per DALY averted for both elective and relief short surgical missions, the present study is in line with the few published reports [14, 15]. It is also in line with other interventions, such as the full childhood immunization program in the Latin America and Caribbean region, estimated at $438 per DALY averted [18].

There are many weaknesses in this study: incomplete data sets, recall bias, misclassifications, and, most important, the subjective nature of many assumptions (disability weights, anticipated outcomes, effectiveness of treatment). Whatever methodological errors exist, they affect both study groups in similar ways. Our initial hypothesis, that short surgical missions by a small NGO would be much less cost-effective in a chaotic relief situation than elective missions conducted in an organized environment, is not supported by our findings. The present study may be useful to inform Operation Rainbow or similar organizations on maximal allocation of resources. More studies are needed to (in)validate our methods and results.