Impact of short-term reconstructive surgical missions: a systematic review.

INTRODUCTION: Short-term missions providing patients in low-income countries with reconstructive surgery are often criticised because evidence of their value is lacking. This study aims to assess the effectiveness of short-term reconstructive surgical missions in low-income and middle-income countries.
METHODS: A systematic review was conducted according to PRISMA guidelines. We searched five medical databases from inception up to 2 July 2018. Original studies of short-term reconstructive surgical missions were included, which reported data on patient safety measurements, health gains of individual patients and sustainability. Data were combined to generate overall outcomes, including overall complication rates.
RESULTS: Of 1662 identified studies, 41 met full inclusion criteria, which included 48 546 patients. The overall study quality according to Oxford CEBM and GRADE was low. Ten studies reported a minimum of 6 months' follow-up, showing a follow-up rate of 56.0% and a complication rate of 22.3%. Twelve studies that did not report on duration or follow-up rate reported a complication rate of 1.2%. Fifteen out of 20 studies (75%) that reported on follow-up also reported on sustainable characteristics.
CONCLUSIONS: Evidence on the patient outcomes of reconstructive surgical missions is scarce and of limited quality. Higher complication rates were reported in studies which explicitly mentioned the duration and rate of follow-up. Studies with a low follow-up quality might be under-reporting complication rates and overestimating the positive impact of missions. This review indicates that missions should develop towards sustainable partnerships. These partnerships should provide quality aftercare, perform outcome research and build the surgical capacity of local healthcare systems. PROSPERO REGISTRATION NUMBER: CRD42018099285.

There is rising concern about the accountability, patient safety and sustainability of short-term reconstructive surgical missions; however, data on these parameters are lacking.

Evidence provided by research on surgical outcomes is limited and of low quality, and the safety of missions is likely to be overestimated by studies in which the quality of follow-up is not reported. Our data suggest that engagement in sustainable development of the local healthcare system and the feasibility of conducting high-quality, long-term follow-up go hand in hand.

We call for implementing longer-term outcome research of future missions.

One approach that could provide a framework to conduct such research, is to implement diagonal development missions. These missions combine the positive impact of the short-term vertical inputs (e.g. providing surgical services) and long-term horizontal investments (e.g. development of sustainable healthcare systems), with the aim of improving access to, and capacity of, the local surgical healthcare systems in the long-term.

Introduction

Conditions that are treatable by reconstructive surgery make up a large part of the global burden of surgical disease. Examples are burns (8.1 million disability-adjusted life years [DALYs]),1 orofacial clefts (0.23 million DALYs),1 complex wounds including trauma-related wounds (unknown DALYs, but estimated to be significant), pressure sores (0.67 million DALYs)1 or noma (a neglected tropical disease, a roughly estimated 1–10 million DALYs).2 Short-term reconstructive surgical missions are a well-established routine method of addressing these conditions and reducing their impact on global health by providing specialised care in underserved populations.3 Such missions are commonly short term, disease specific, focus on service delivery and have a tendency to work outside the local healthcare system. This is also referred as a ‘vertical approach to healthcare development’.4

Despite being a common model, the impact of reconstructive surgical missions is hardly known.5 6 Medical missions in general are commonly debated in the literature.7–15 There is rising concern about the limited accountability of missions, with little data reported back to healthcare authorities due to a lack of outcome measurements.8 9 Quality of care is debated, as missions often have limited capacity to provide ancillary services or follow-up.16 Furthermore, sustainability is questioned in terms of lasting positive impact on the local healthcare system or its cost-effectiveness. The question is whether short-term surgical missions are the most rational allocation of resources to address local healthcare needs.4 7 8 12 15 Ultimately, the ethical implications of surgical volunteerism often ignite debate.17 18

These concerns are discussed in several reviews of medical missions in low-income and middle-income countries (LMICs).7 9 10 12–15 19 For instance, Martiniuk et al and Roche et al argued that global standards are needed for short-term medical missions,7 14 and Sykes shows that only 6% of all published studies on medical missions report on empirical data.10 Only a few studies reviewed surgical missions specifically.9 12–15 19 Shrime et al systematically compare three types of charitable platforms for global surgery (short-term missions, self-contained surgical platforms and specialty surgical hospitals run by non-governmental organisations [NGOs]). Although they conclude that evidence in the literature is scarce, they state that self-contained temporary platforms and specialised surgical centres appear to provide more effective and cost-effective care than short-term surgical missions, except when no other delivery platform exists.12

These reviews provide valuable insights into medical missions in general. However, the diversity of medical and surgical missions is large, which hampers the interpretation of empirical data. This review aims to systematically review evidence on the impact of short-term reconstructive surgical missions specifically, and critically analyses the quality of the available empirical data. In this review, four key aspects are addressed: basic characteristics of missions, patient safety, health gains of individuals and sustainability.

Methods

We conducted a systematic review of the literature following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines.20

Inclusion criteria

All original studies that analysed empirical data of short-term missions pertaining to reconstructive surgical care in LMICs were eligible. Studies lacking analyses of empirical data, reviews, studies of specialty surgical hospitals that provide continuous year-round care, mobile surgical platforms sent from in-country hospitals, studies in conflicts zones, studies not related to LMICs or studies of patients that were transported to high-income countries (HICs) were excluded. The studies were restricted to English and Dutch language. No restrictions were applied regarding publication dates. Duplicates were excluded.

Search strategy and data sources

PubMed, Embase.com, Clarivate Analytics/Web of Science and Open Grey were searched up to 1 July 2018; Proquest up to 1 July 2017 (by CR, TH and JK). The last database was no longer available to us after 2017. The following terms—including synonyms—were used as index terms or free-text words: ‘plastic surgery’, ‘reconstructive surgical procedures’, ‘cleft lip’, ‘post-burn contractures’ or ‘noma’ combined with ‘medical missions’, ‘humanitarian’ or ‘charity’. More studies were identified by reviewing the bibliographies of retrieved studies. The full search strategies for all databases can be found in the online supplementary file 1.

Study selection

Studies were screened for eligibility by two independent investigators (TH and CR): in case of disparity, a third author was involved (MB). Two investigators independently extracted the data to create tables and figures (TH and CR).

Data on mission characteristics and individual patient-level data were extracted and analysed across four key features:

Basic characteristics of the missions, including mission length, number of patients who received surgery, and age and gender distribution.

Patient safety by means of complication registration. Data were collected on three indicators: follow-up length, follow-up rate and complication rate. The follow-up rate was calculated by the number of patients who completed follow-up divided by the total number of patients who were included for follow-up. The complication rate was calculated as follows: the number of patients with complications divided by the total number of patients who completed follow-up.

Health gains: data on surgical outcomes were collected, for example, improvement of range of motion, patient-reported outcome measures (PROMs) or DALYs averted per patient. All reported PROMs were recorded, for example on surgical outcomes, complications or the quality of care provided. All types of formats, questionnaires or any other tool describing these outcomes were included.

DALYs are used to define the overall disease burden over a population and are calculated by ‘adding the number of years of life lost due to premature mortality to the number of years of healthy life lost related to disability’.21 This means that 1 DALY can be defined as one lost year of healthy life.21 Many limitations of this approach are described in the literature. The biggest challenge is that it is not based on health data from countries, but on complex estimation techniques. DALYs are therefore estimations and many concerns exist about their reliability and uncertainty.22 23 Despite these challenges, DALY metrics are commonly applied in global surgery studies.1 21 24

Sustainable characteristics of missions: studies were reviewed for data on long-term partnerships, training objectives and involvement of local staff. Also, data on the cost-effectiveness of missions were collected. Missions were categorised either as stand-alone or consecutive missions to the same hospital, region or country.

Data analysis and synthesis

After a pilot, data were extracted independently and in duplicate using a data extraction sheet (TH and CR). Authors were contacted when data on complication registration were missing. Quantitative data synthesis consisted of compiling total number of patients (eg, total number of patients who were included, total number of patients with complications) to generate overall outcomes. Table 1 and the online supplementary appendix file 1 provide details of the data extracted from each reference. Due to the heterogeneity of studies in types of surgery, local healthcare settings or available resources, statistical analyses were not feasible. Study quality assessment was performed independently by authors TH and CR according to the Oxford CEBM Level of Evidence classification25 and the GRADE (Grades of Recommendation, Assessment, Development and Evaluation) system.26

Table 1

Study characteristics

Authors (year)	Affiliated organisations	Country of mission	Year of mission	Number of patients treateda	Length of follow-upb	Follow-up rate	Complication ratec	Health gains
Clefts
Aziz et al (2009)27	NA	Bangladesh	2006–2008	146	≤10 days	NA	8/146 (5.5%)	NA
Bello et al (2018)28	CFDF	Nigeria	2011–2017	448	2 months	155/448 (34.6%)	35/155 (34.6%)	NA
Bermudez and Lizarraga (2009)69	Operation Smile	40 countries	2007	4086	1 year	812/4086 (19.9%)	NA	NA
Calis et al (2016)33	Interplast Turkey	Uzbekistan	2009–2014	529	NA	NA	1/529 (0.2%)	NA
Daniels et al (2016)34	ReSurge Int.	China	2005–2009	201	1–5 yearsb	116/201 (57.7%)*	34/96 (35.4%)*	NA
Fayyaz et al (2015)36	Cleft Lip and Palate Association Pakistan	Pakistan	2014	312	3 months	NA	18/312 (5.8%)	NA
Guneren et al (2015)40	Turkish international development agencya	Asia, Middle East, Africa	2007–2014	25	NA	NA	NA	NA
Hackenberg et al (2015)41	Operation Smile	India	2006–2012	3503	NA	NA	NA	Total 21 006 DALYs averted6.0 DALYs averted per patient
Hughes et al (2016)42	Hands Across the World	Ecuador	2015	27	NA	NA	NA	NA
Hughes et al (2012)43	Hands Across the World	Ecuador	1996–2011	1142	7 days	1089/1142 (97.1%)*	40/1122 (3.6%)*	Total 396–1042 DALY averted3.9–10.2 DALY averted per patient
MacIntosh et al, (2013)46	Healing the Children	Colombia	1994–2011	2558	NA	NA	10/2727 (0.4%)d	NA
Madsen et al (2015)47	US military	Dominican Republic	2005–2009	223	30 months	205/223 (91.1%)	13/223 (5.8%)	Speech score improved from 11.4 (6–24) to 5 postoperatively (borderline=6)
Magee et al (2010)48	Operation Smile	Kenya, Russia, Nicaragua, Vietnam	2008	303	NA	NA	NA	Total 3099.52 DALYs averted10.1 DALYs averted per patient
Maine et al (2012)49	ReSurge Int and Rostros Felices	Ecuador	2000–2005	315	>14 days	128/315 (40%)	72/128 (56.3%)	NA
McQueen et al (2007)53	Operation Smile	Jordan, Iraq	2005	71	NA	NA	4/71 (5.6%)	NA
McQueen et al (2009)67	Operation Smile	18 Countries	NA	8151	NA	NA	67/8151 (0.8%)	NA
Moon et al (2012)55	Smile for Children	Vietnam	2007–2010	303	NA	NA	NA	Total 377 to 458 DALYs averted on average mission
Navarro (2015)56	CIRPLAST	Peru	1994–2014	6108	12 days (range 12 days to 9 years)b	5162/6108 (84.5%)	377/5162 (7.3%)	NA
Park et al (2018)39	Operation Smile	India	2010–2011	890	7 days	662/890 (74.4%)	101/662 (15.3%)	NA
Rauso et al (2015)57	Emergenza Sorrisi Onlus	Uganda, Gabon	2012–2014	56	NA	NA	2/56 (3.6%)	NA
Rivera et al (2013)58	Operation Smile	Honduras	2007	45	6 months	22/45 (48.9%)	3/22 (13.6%)	NA
de Buys Roessingh et al (2012)16	SedoGoho hospital, TdH, CHUV Laus.	Benin and Togo	1993–2008	131*	5.6–7.6 years	36/71 (50.7%)*	14/71 (19.7%)*	Speech follow-up: 36 patients. Acceptable 17/36=47.2%. Unacceptable 19/36=52.8%
Rossell-Perry et al (2015)60	ReSurge International and Smile Train	Peru	2002–2012	257	1–5 yearsb	97/353 (27.5%)d	34/257 (13.2%)*	NA
Sharp et al (2008)61	Operation Smile	Philippines	2003	120	6 months	52/99 (52.5%)*	10/50 (20.0%)*	Improved speech 52% Improved eating 25% Improved social benefit 14% Improved appearance 6%
Sieg et al (2004)62	NA	Africa, Asia, Central America	NA	14	≥1 year	10/14 (71.0%)*	1/10 (10%)*	NA
Uemura et al (2015)64	Duang-Kaew Foundation	Thailand, Vietnam, Myanmar, Laos, Cambodia, China, Sri Lanka, Bhutan and India	1988–2008	6832	1 month	5412/6832 (79.2%)	186/5412 (3.4%)	NA
Uetani et al (2006)65	Japanese Cleft Palate Foundation	Vietnam	1993–2003	790	NA	NA	NA	NA
Wes et al (2017)66	Changing Children's Lives Int.	Thailand	2013	56	<1.5 years	30/56 (53.6%)	0/30 (0%)	Self-reported improvement: social interactions 83,3%; confidence 83.3%; school performance 75%
Post-burn contracture
Borghese et al (2005)31	NA	Cambodia, Bangladesh	2002, 2003	200	NA	NA	14/200 (7.0%)	NA
El Ezzi et al (2017)35	Terre des Hommes	Benin and Togo	2002–2011	50	3.6 years	50/50 (100%)	28/50 (56.0%)	NA
Fuzaylov et al (2015)38	Doctors Collaborating to Help Children	Ukraine	2011–2013	39	NA	NA	1/39 (2.6%)	NA
Kim et al (2012)45	Operation ReStore, Operation Smile and Cents of Relief	India	2010	38	NA	NA	9/60 (15.0%)	NA
Sinha et al (2016)63	Operation ReStore	India	2012	31	84 days	31/39 (79.5%)	9/31 (29.0%)	SF-36 QoL : improvement of 5.8 pointsWPI: 13.7% mean improvement
Noma
Bouman et al (2010)32	Facing Africa and Dutch Noma Foundation	Ethiopia, Nigeria	2007, 2008	63	35 days	74/74 (100%)d	47/74 (63.5%)d	Excellent results 36%Satisfactory 23%Mediocre 16%Poor 11%Very poor 14%
Marck et al (2010)50	Facing Africa	Ethiopia	2007, 2008	77	35 days	77/77 (100%)	54/77 (70.1%)	Good results 30.7%Acceptable 34.6%Mediocre 17.9%Poor 7.7%Very poor 9.0%
McGurk and Marck (2010)52	Project Harar	Ethiopia	2007–2009	95	35 days	89/95 (94%)	57/89 (64.0%)	Simple surgery group: good or acceptable results 90%Complex surgery group: good or acceptable results 40%Overall, poor results 6%Overall, very poor results 6%
Rodgers et al (2015)59	Facing Africa and Dutch Noma Foundation	Ethiopia	2008–2014	34	36 days	NA	17/34 (50.0%)	NA
General reconstructive missions
Baran et al (2007)29	Physicians for Peace and Interplast	Multiple countries	1985–2004	4736	NA	NA	NA	NA
Figus et al (2009)37	Interplast Italy	Multiple countries	1988–2008	5235	NA	NA	NA	NA
McClenaghan et al (2013)51	Project Harar	Ethiopia	2012	40	21 days	30/30 (100%)	7/30 (23.3%)	NA
Merrel et al (2007)54	Operation Smile	Vietnam	1990–2004	266	NA	NA	6/266 (2.3%)	NA

Authors were contacted when data were missing for follow-up. Of note, in several studies (indicated with an *), the follow-up rate or complications rate were calculated over different subgroups; therefore, columns may not add up or correlate. aWhen available, this review reports the number of patients who received surgery; when not available, the number of procedures was used; when not available, the number of diagnosis was used. bWhen studies reported a range of follow-up intervals, the shortest length of follow-up was used for calculations. cWhen the total number of patients who completed follow-up was not available, the total number of patients included was used (in line with the cited articles). dThe complication rate cited was calculated over the total number of procedures.

CFDF, Cleft & Facial Deformity Foundation; DALY, disability-adjusted life year; NA, not available; SF-36 QoL, 36-Item Short Form Health Survey on Quality of Life; WPI, Whole Person Impairment questionnaire.

Results

The search identified 1662 unique citations. After screening titles and abstracts 1570 studies were excluded because they did not concern short-term reconstructive surgical missions. Ninety-two studies concerned reconstructive surgical missions and were reviewed full-text. Of the full-text studies, 51 were excluded. Please see figure 1 for the screening and selection process. Studies reviewing specialty hospitals missions were excluded, as this was outside the scope of this review. After full-text analyses, 41 met full inclusion criteria (figure 1 and table 1).27–67

Figure 1

Flow diagram. LMIC, low-income and middle-income country.

The studies included predominantly consisted of case series, with 37 studies graded at Oxford CEBM Level IV. The remaining four economic analyses were graded at level IIB. This resulted in a C grade of recommendations for our review, according to CEBM. The overall GRADE score was 2.7 (low to moderate quality), meaning that our confidence in the effect estimate is limited (table 2).26 68

Table 2

Quality assessment results

Type of missions	Number of studies included	Oxford CEBM Level of Evidence	Average GRADE score
Cleft care mission studies	28 studies	24 Level IV studies (case series)4 Level IIB (economic studies)	2.6 (quality: low–moderate)
Post-burn contractures mission studies	5 studies	5 Level IV studies (case series)	3.4 (quality: moderate)
Noma mission studies	4 studies	4 Level IV studies (case series)	4.3 (quality: high)
General reconstructive surgery mission studies	4 studies	4 Level IV studies (case series)	1.3 (quality: very low–low)
Overall quality and level of recommendation	41 studies	37/41 Level IV studies4/41 Level IIB studiesLevel C recommendations	2,7 (quality: low–moderate)

Information listed per condition. Quality assessment of included studies was performed using the GRADE system68 and Oxford CEBM Level of Evidence.

Twenty-eight of 41 studies included pertained to cleft care (78% of the total study population). The number of patients in all studies totalled 48 546, with a mean age of 13.4 years (SD 8.5) (table 3). The average mission length was 10 days (SD 3.8; range, 6–21 days). Organisations were active in Africa, South-East Asia, Eastern Europe, and South and Central America. A typical mission team consisted of two or three plastic and/or maxillofacial surgeons, one or two anaesthetists, a mission coordinator, theatre nurse and one or two resident doctors, totalling 8 to 10 individuals for one single mission.32 33 37 46 47 50 51 54 58 Some teams were considerably larger, up to 40 individuals.36

Table 3

(A) Overall outcomes on basic characteristics of missions

	Length of mission			Total number of patients			Gender distribution			Age
	Studies (n)	Total length (days)	Average (days)	Studies (n)	Patients (n, %)	Average per study (n)	Studies (n)	Female (n, %)	Male (n, %)	Studies (n)	Mean (years)	Studies (n)	Median (years)
Clefts	18	168	9	28	37 642 (78)	1344	18	12 210 (45.8)	14 435 (54.2)	12	9.22	3	4.5
Post-burn contractures	3	23	8	5	358 (1)	72	4	143 (44.8)	176 (55.2)	2	27.6	1	4.0
Noma	2	28	14	4	269 (1)	67	3	102 (58.6)	72 (41.4)	1	23.9	1	17.0
General reconstructive	4	57	14	4	10 277 (21)	2569	1	143 (44.8)	176 (55.2)	1	24.0	NA	NA
Totals	27	276		41	48 546 (100)		26	12 598 (45.9)	14 859 (54.1)	16		5
Overall mean or median			10 (+/-SD 3.8)			1184 (+/-SD 2134.4)					13.4 (SD+/-8.5)		4.5 (Q25-75 2.9-13)

*The follow-up rate: the number of patients who completed follow-up divided by the total number of patientsi ncluded for follow-up. For clarity reasons only percentages are displayed, patients numbers are omitted.

†The complication rate: the number of patients with complications divided by the total number of patients who completed follow-up. Displayed are the patient numbers, between brackets the complication percentage.

NA, not available.

Regarding patient safety, nine studies (22%) did not report on complications. Twelve studies (29%) only reported a complication rate without reporting on follow-up length or rate. The overall complication rate in these 12 studies was 1.2%. Ten studies (24%) reported a follow-up length shorter than 180 days, with a mean follow-up rate of 81.3% and a complication rate of 7.1%. Ten studies (24%) provided a follow-up length longer than 180 days, reporting a mean follow-up rate of 56.0% and a 22.3% complication rate (table 3). Mortality after cleft surgery was reported in three studies, totalling 3 out of 14 551 patients included in these studies.16 28 64 For general reconstructive surgical missions—not specified for a single disease—one single study reported one death54 and no mortality was reported in contracture and noma missions.

Twelve studies (29%) reported on health gains of the mission, reporting heterogeneous methods and outcomes (table 1). Methods used in cleft studies included photographic assessment of aesthetic outcomes,30 69 speech evaluation16 47 or DALYs averted.41 43 48 55 Three cleft studies evaluated speech functionality postoperatively, either by questionnaires61 or speech tests.16 47 Study methods were clear and showed overall improvements of speech. Four cleft studies reported on DALYs averted by cleft lip and palate repair surgery. DALYs averted per patient were 3.9,43 6.041 and 10.1 per patient.48 In three noma missions, a surgeon-reported outcome scale was used to score aesthetic and functional outcome.32 50 52 Overall findings showed that high-complex surgery is associated with greater risks of unsatisfactory results. Three studies used PROMs. One contracture study reported improvements in quality of life and disability by using validated questionnaires, and reported overall positive outcomes.63 Two cleft studies used self-developed questionnaires to assess PROMs, reporting positive results.61 66 None of the studies reported on patient-reported outcomes on the quality of the care provided.

With regard to the sustainable characteristics of missions, 29 studies reported qualitative data (71%) on sustainability, while none of the studies reported quantitative data. Fifteen out of twenty studies that reported on follow-up and complications also reported on sustainable characteristics such as long-term partnerships or training activities (table 4). Ten organisations (24%) were engaged in longer-term partnerships, and thirteen missions (32%) returned to the same regions or hospitals. Few data were available on the frequency of missions, although several studies reported conducting yearly missions.32 35 43 50 Fifteen studies (35%) described teaching objectives as a goal during their missions. Activities mentioned were lectures,29 37 55 training of local surgeons,16 35 38 54 healthcare workers47 55 60 or fellowships in donor countries.16 54 55 However, none of the studies published empirical data on the effects of training or elaborated on how the training of local healthcare personnel was organised.

Table 4

Sustainable characteristics of short-term missions

Authors (year)	Years	Number and frequency	Long-term relationship				Teaching objective				Advancement of local staff			Quality follow-up and sustainable characteristics†
			Consecutive missions to the same country	Consecutive missions to the same region/city	Consecutive missions with (part of the) same team	Partnership	Training local staff	Lectures/workshops/education	Advice on logistics within healthcare system	Encouraging medical independence of local staff	Participation of local staff in surgical care and pre-op and/or post-op care	Fellowship abroad	Providing medical supplies	Studies that reported both on follow-up details and sustainable characteristics
Aziz et al (2009)27	2006–2008	3 (annual)	✓		✓	✓	✓				✓
Bello et al (2018)28	2011–2017	17	✓	✓			✓				✓			■
Baran et al (2007)29			✓	✓	✓		✓	✓		✓	✓	✓	✓
Bermudez et al (2009)30	2007					✓					✓
Borghese et al (2005)31	2002–2003		✓
Bouman et al (2010)32	2007–2008	4 (biannual)	✓											■
Calis et al (2016)33	2009–2014	6 (annual)	✓	✓	✓						✓		✓
Daniels et al (2016)34			✓		✓									◊
El Ezzi et al (2017)35	2002–2011	9 (biannual)	✓	✓	✓	✓	✓		✓		✓	✓	✓	◊
Fayyaz et al (2015)36	2004–present	130	✓	✓	✓						✓		✓
Figus et al (2009)37	1988–present	47	✓				✓	✓		✓	✓
Fuzaylov et al (2015)38	2011–2013	3 (annual)	✓	✓		✓	✓	✓			✓		✓
Guneren et al (2015)40	2007–2014	27			✓						✓		✓
Hughes et al (2012)43	1996–2011	16 (annual)	✓			✓								■
MacIntosh et al (2013)46	1993–present	(biannual)	✓				✓				✓
Madsen et al (2015)47	2005–2009	(annual)	✓	✓				✓			✓			◊
Maine et al (2012)49	2000–2005					✓								◊
Marck et al (2010)50	2007–2008	2 (annual)	✓	✓	✓									■
McClenaghan et al (2013)51	2012		✓	✓										■
McQueen et al (2007)53	2005							✓			✓
Merrel et al (2007)54	1990–2004	11	✓	✓			✓	✓				✓
Moon et al (2012)55	2007–2010	4 (annual)	✓				✓	✓		✓	✓		✓
Navarro (2015)56	1994–2014	141 (10 annually)	✓											■
Park et al (2018)39	2010–2011	2	✓	✓		✓	✓		✓	✓	✓			■
Rivera et al (2013)58	2007	1									✓			◊
de Buys Roessingh et al (2012)16	1993–2008	(annual)				✓	✓	✓	✓			✓	✓	◊
Rossell-Perry et al (2015)60	2002–2012		✓	✓	✓	✓	✓							◊
Uemura et al (2015)64	1988–2008	458	✓						✓		✓			■
Uetani et al (2006)65	1993–2003		✓	✓			✓				✓		✓
			Reported by 23 studies	Reported by 13 studies	Reported by 9 studies	Reported by 9 studies	Reported by 13 studies	Reported by 8 studies	Reported by 4 studies	Reported by 4 studies	Reported by 18 studies	Reported by 4 studies	Reported by 9 studies	Reported by 15 studies (75%) of all 20 studies that reported on follow-up details

In order to determine whether or not missions were sustainable, the data were collected and allocated to three groups, ie, building long-term relationships, teaching objectives during the mission and the advancement of local staff. Each sustainability group is divided into different subsets. A checkmark (✓) means the study describes the concomitant form of sustainable health care.

*Part of the Operation Smile International (OSI) programme.

†Added to this table were studies that reported both on sustainable27 28 characteristics and on quality of follow-up (including length and rate of follow-up and complication rate). Eight studies with follow-up shorter than 180 days (■ marks), seven studies with follow-up29 30 up longer than 180 days (◊ marks). Five studies reported on quality of follow-up, but did not report on sustainable characteristics.

Four cost-effectiveness studies were available for short-term cleft missions. Three studies reviewed the effectiveness per DALY averted, reporting US$33.94/DALY,48 US$56.0/DALY55 and US$247.42/DALY.41 The variation is explained by the differences in study populations, sample sizes, effectiveness measurements and ‘costing approaches’ used.

Discussion

Several systematic reviews about short-term medical missions are available.7 9 10 12–15 This is the first systematic review that specifically assesses the quality of available data on short-term reconstructive surgical missions.

Although all the studies included in our review reported a positive impact of surgical missions, the level of evidence remains low. It seems that follow-up of treated patients is a challenge. Although a majority of studies provide data on complication rates, the varying quality of this outcome measure makes it difficult to draw any conclusions. The results showed that reported complication rates were considerably higher when the quality and length of follow-up increased. This suggests that without data on quality of follow-up, there is a high risk of reporting bias due to under-reporting of complications. This also means that without comprehensive information on follow-up, the safety of missions is likely to be overestimated.

Furthermore, studies used different control groups to benchmark their respective findings with regard to complication rates. Three of cleft care studies included compared complications between mission patients and patients who underwent similar procedures in a HIC. Results showed substantially higher complication rates in mission patients.34 49 60 One study showed that fistula risk was 15.6 times that for a US cohort.34 Maine et al 49 state that complication rates were 20 times higher in the mission cohort compared with a US cohort, independently of whether the surgery was performed by Ecuadorian or American surgeons. It should be mentioned that comparisons of complication rates between HICs and LMICs cannot be made without taking into consideration that HICs have more resources at their disposal to limit complications. Therefore, we would suggest developing benchmark complication rates of LMICs, which can be used to assess the outcomes of short-term missions.

Some authors argue that longer-term specialty surgical hospitals may be provide more effective care than short-term missions.9 12 39 Specialty hospitals provide continuous care all year round in a LMIC. The cleft care centre of Operation Smile in India,39 or Smile Train’s model are examples of this approach.69–78

Both organisations report lower complication rates than those reported in short-term missions. The centre of Operation Smile reports a short-term complication rate of 4.0% (cleft lip repair) and 15.8% (cleft palate repair), which is lower than the rates of their counterpart short-term missions.39 Smile Train studies report lower rates, between 0.88% and 3%.70 71 73 78 However, they note that there might be a risk of under-reporting or selection bias due to a dependence of Smile Train surgeons on payment-per-patient (risking fewer referrals when higher complication rates are reported) and a limited capacity of surgeons to treat complex cases.70 71 73 78 Furthermore, with only one Smile Train study reporting on follow-up lengths,73 these complication rates should be interpreted with caution. To be able to compare the strengths and weaknesses of different approaches of providing surgical care in a LMIC, there is a need for more high-quality studies.12 39 Apart from registrations of complications, such studies should assess long-term outcome using validated outcome measures and PROMs. Specialty hospitals, which provide services all year round, could provide good conditions for longer-term outcome research.

Several studies in this review consistently report on follow-up, showing that substantial efforts are being made to improve the data output of missions.16 34 35 49–51 61 63 Ten studies reported significant follow-up lengths of more than 6 months and high numbers of patients returning for follow-up were shown.16 34 35 47 49 58 60–62 66 The majority of these missions were engaged in long-term partnerships. This included training of local healthcare personnel, which was likely to improve the feasibility of organising follow-up. Several strategies were implemented to ensure the quality of follow-up. Some missions deployed medical students to assess palate fistulas34 or sent a speech pathologist in-country to review outcomes.61 Others trained local surgeons on follow-up and revision surgery.54 The relatively high number of complications seen in noma missions could be partly explained by a stringent follow-up, done by an independent researcher who consistently reported on follow-up. All the studies mentioned above provide examples of how to ensure patient safety during and after missions.50 79

Although some studies reported on health gains, with several studies reporting positive functional outcomes,16 47 61 63 66 the methods and evidence are heterogeneous and results are too limited to draw conclusions. The role of PROMs are effective in reconstructive surgery to assess the quality and outcomes of healthcare.80 81 Only few of the studies included reported successfully on outcomes using PROMs61 63 66 and none assessed the quality of care experienced by patients. Patient experience of outcomes and quality is important.82 Future studies should include PROMS on surgical outcomes and quality of care. Only a few studies report on the sustainable characteristics of missions. Data on this topic are usually qualitative and highly variable. It is noteworthy that reporting on sustainability and higher quality of patient follow-up often go hand in hand. This suggests that more sustainable missions may be better able to follow their patients for a longer period. However, as empirical evidence on sustainability is still non-existent, there is an urgent need for further studies.12

Limitations

This systematic review has several limitations. Literature on short-term reconstructive missions is scarce and of limited quality, limiting the strength of this review.25 As the majority of studies are cleft studies, the conclusions and recommendations of this review may not be fully applicable to other types of reconstructive surgical missions.

The studies included represent just a small proportion of the many reconstructive surgical missions conducted worldwide. This may introduce a potential bias. It is likely that the small proportion likely does not fully represent the actual effect of all reconstructive surgical missions. In our view, this emphasises the need to incorporate standard monitoring and evaluations into missions.

Furthermore, this review addresses only short-term missions and does not attempt to make a direct comparison with long-term surgical platforms such as specialty hospitals. It is often argued that specialty hospitals are safer and have a more positive effect on local healthcare systems.12 76–78 83 84 Comparative studies of short-term missions and specialty hospitals can identify strengths and weaknesses of each approach. However, a definitive comparison between missions and specialty hospitals seems to be premature at present given the lack of comparative studies.12 39

Concerns regarding the use of DALY metrics are applicable to the studies included in this review. It is argued that surgical conditions are underestimated in the global burden of disease studies.1 Attempts to estimate the surgical burden across all disease conditions have been challenging.85 86 In a recent study, it was argued that the current DALY approach is inadequate to quantify the burden of paediatric surgical conditions.87

Recommendations

There are opportunities for NGOs to develop short-term missions towards more sustainable partnerships. In the past, missions have been a ‘vertical’ approach to healthcare development.4 Such missions have limitations, for example in building local capacity of surgical services. The results of this study indicate that longer-term follow-up is frequently lacking, with complications being potentially missed. To address these shortcomings, the ‘diagonal development’ approach has been proposed.4 It combines the short-term vertical inputs of missions with longer-term horizontal benefits, with the ultimate aim of improving access to, and surgical capacity of, the local healthcare system. Such goals may be achieved through long-term development of surgical infrastructure, continued training of the local surgical workforce or building an academic culture.4

One example of such a diagonal approach is to aim for standardised tracking of longer-term outcomes of missions in strong collaboration with local partners. This might yield several advantages. Besides empowering local researchers and building an academic culture, outcomes can be reported back to patients and healthcare authorities. This will enhance the accountability of NGOs8 9 and allow for evaluations of the quality of care provided.

Another example of long-term investments in the local surgical capacity is strengthening of the training activities of surgical NGOs. Such activities should be integrated into existing national or regional training activities. The training should be adapted to local settings, needs-driven and should focus on bilateral knowledge exchange.4

Conclusion

This review shows that evidence for the effectiveness of short-term reconstructive surgical missions is both of limited substance and quality. Given the overall lack of evidence, there is an urgent need to incorporate outcomes research in future missions. This should include longer-term complication registration and measurements of health gains among individual patients. The effectiveness of training activities should also be evaluated. One approach to achieve this is to develop short-term missions towards diagonal development missions, which aim to build surgical capacity of local healthcare systems through long-term investments.