Accountability in global surgery missions.

Supervisory systems regulating outcomes after global surgical missions are limited.

With a few notable exceptions, most global surgical missions garner low (if any) follow-up rates and thus outcomes reporting is scarce.

Obstacles include logistical difficulties, surgeon and non-governmental organisation non-compliance, journalistic coverage (both positive and negative), and economics.

We suggest a multifaceted solution, including buy-in and direction from global surgical leadership, web-based and tax-incentivised reporting, and a cross-disciplinary cooperation with journalists and social media.

We believe that the implementation of outcomes reporting during the development of surgical systems in low-income and middle-income countries is both an efficient way to promote safe surgery worldwide and vital to our professional ethic.

Introduction

There are limited supervisory systems, if any, regulating poor outcomes after medical and surgical missions in low and middle-income countries (LMICs). Even in the environment of close legal and academic oversight,1 2 the medical community fights the perception that only ‘good’ outcomes are reported in medical journals.3 The nature of global surgery poses significant additional barriers to oversight and outcomes reporting, some of which are more self-evident than others. The void of oversight may encourage untrained teams to embark on missions beyond their abilities4; despite the best of intentions, global surgeons may be doing more harm than good. We owe it to our Oath as medical professionals and to the populations we serve to evaluate the care we are providing, not just in our own countries, but in the countries to which we travel as non-governmental organisations (NGOs) and other humanitarian or global surgical missions. Our aim in this commentary is to (1) survey the current climate of outcomes reporting in global surgery, (2) identify obstacles and (3) propose concrete solutions.

Current climate of outcomes reporting in global surgery

First, it is essential for us to acknowledge the role of outcomes and our reasons for focusing on them in our evaluation of accountability in global health. By definition, healthcare outcomes are intrinsic to quality of care and in general they are relatively free of bias. However, due to the time required from care to outcome, linking care to outcomes can be a challenge. In light of this, all three elements of the Donabedian quality triad—process, structure and outcomes—are essential in building a successful healthcare system framework and should be considered, reported and monitored when working to improve quality of healthcare. Processes and structure can serve as surrogates for outcomes when outcomes have significant lag time or rarity (eg, mortality) that makes it difficult to adequately power studies.5 Without adequate oversight or reporting, none of these elements are appropriately monitored. We will focus primarily on outcomes reporting in global surgical endeavours due to its centrality to quality and concrete nature, with the understanding that process and structure should fall under the purview of a supervising body as well.

We recognise that even in high-income countries (HICs) with established surgical systems, outcomes reporting is not mandatory, or even common. In the USA, the National Surgical Quality Improvement Program (NSQIP) currently has 582 domestic hospitals enrolled, of over 5500 hospitals in the country.6 As outcomes are integral to evaluating care, we do not ignore or minimise the need for improvement in this arena in HICs. Rather, we see an opportunity in the LMIC context to build this critical function into their systems from the outset. Similarly, the danger of the lack of accountability for surgeons overseas, especially in areas with less robust legal oversight, adds urgency to the call for required outcomes reporting for humanitarian or global surgical missions.

Currently, most data published from humanitarian missions are observational reports of care provided, with limited outcomes reporting. Examples include military hospital ships or forward surgical team missions,7–10 short-term trips with a singular focus such as inguinal hernia repair11 12 or cleft lip/palate repair,13 and larger volume reports from entities like Médecins Sans Frontières14 and the Cinterandes Foundation’s mobile surgical unit.15 The outcomes reporting in these studies is largely intraoperative morbidity/mortality, immediate postoperative morbidity (within 2–3 days), and rarely 30-day morbidity, with follow-up rates around 20%–30%. An exception is the work of White et al,16 who reported 7-year follow-up of morbidity, social impact and patient experiences of humanitarian mission surgery (maxillofacial, plastics and orthopaedic) using semistructured interviews and questionnaires. Of 174 patients identified via purposive sampling, 41% participated in the evaluation, and the results included 35% with persistent postoperative pain and 18% who had sought follow-up for a clinical complication. However, 73% of patients felt that they reaped social benefit from surgery and the patient satisfaction rate was 89%.

Recognising that safe surgery is a multidisciplinary effort, the G4 Alliance17 has called for a similar level of outcomes reporting from the four key areas of surgical, trauma, obstetric and anaesthesia care. As part of their work, the G4 Alliance called on the Member States of the WHO to prioritise this issue and ensure mechanisms to collect national surgical and anaesthesia indicators. Several other organisations have also collectively championed ‘safe and effective surgery’ as an essential component of primary health coverage.18 19 The World Federation of Societies of Anesthesiologists adopted the International Standards for a Safe Practice of Anesthesia in 1992, and revisions were ratified in 2008 and in 2010. They are recommended standards for anaesthesia professionals throughout the world, intended to provide guidance and assistance to anaesthesia professionals, their professional societies, hospital and facility administrators, and governments for improving and maintaining the quality and safety of anaesthesia care. As part of the WHO’s effort to improve patient safety, they advocate the ‘Patient Safety Pulse Oximetry’ project, which aims to improve the safety of operating rooms worldwide.20 They include indicators such as caesarean section rate, proportion of operating theatres with pulse oximetry, trauma mortality rate and perioperative mortality rate. A recent consensus recommendation for short-term missions in paediatric general surgery and perioperative care is clearly a step in the right direction to ensure safe, responsible and ethical conduct by HIC providers.21 We agree that these are good places to begin for evaluating and building nascent surgical systems in LMICs.

A few pioneering publications have spearheaded the effort towards more comprehensive outcomes reporting. The Royal Colleges in the UK and Ireland have initiated an advocacy group with the support of the WHO for curriculum development, quality assurance and assessment of young surgeons.22 Orthopaedic surgeons were able to achieve 81.9% follow-up rates of 127 patients over 5 years via coordination with a Peruvian general surgeon—notably, this study began 3 years after the establishment of the programme which has been serving the same city in Peru since 2004 and provides graduated (increasing) financial compensation to patients for returning for follow-up.23 Supervised teaching of surgical techniques to general practitioners in Malawi led to an increase in volume of general surgical cases by 89%, and outcomes of hernia operations done by these practitioners in peripheral hospitals were comparable with those performed in central hospitals.24 Marck et al 25 demonstrated the value of early outcome studies in two consecutive missions to Ethiopia. By comparing outcomes in 2007 with those from 2008 after implementation of quality improvement methods, they show two important results: a decrease in overall complications, but a sustained high rate of complications with complex procedures. Finally, Jindal et al 26 obtained data on 60 corneal transplant recipients in Guyana, demonstrating dramatic improvement in quality of life and aversion of 260 disability-adjusted life years (4.3 per patient). These examples highlight the ability to demonstrate impact and improve quality during humanitarian missions if outcome measures are monitored and reported. Shrime et al 27 completed a systematic review of charitable platforms in global surgery which also demonstrated many good examples of outcomes reporting, such as Venkatesh et al,28 who reported 94% visual acuity improvement after cataract surgery with an 88% 40-day follow-up rate. While this review was able to compare efficacy of platforms in global surgery based on such studies that appropriately published outcomes, Shrime et al noted that they had to exclude platforms such as individual surgeon organised missions and missions that aim to establish training programmes because of a lack of outcomes reporting.

Thus these cases are the vanguard, and not yet the norm. In some cases, outcomes are reported for us by social media and investigative journalism, raising further questions of accuracy, bias, sensationalism and helpfulness. What are the obstacles to outcomes reporting, and what mechanisms can be employed going forward to create a replicable framework for outcomes reporting in global health?

Identifying obstacles

Arguably, the most self-evident obstacle is the logistical difficulty of follow-up. As mentioned above, even in studies that attempted follow-up, rates were around 20% and rarely more than 30 days after surgery. Perhaps the most successful example was a follow-up attempt by Operation Smile, which included 6-month and 12-month postoperative appointments with pictures and independent surgeon reviews; this effort garnered a 37% follow-up rate.12 Obstacles include transportation issues, record-keeping difficulties, and lack of coordination between medical teams themselves and between teams and local healthcare professionals.29 As has previously been stated, ‘without follow-up, the consequences of complications for operated patients could be disastrous’.30

Similarly, by nature, many global health missions take place in conflict or disaster zones. Despite advances in assessment, coordination and delivery of humanitarian aid during the Haiti earthquake (2010), information on surgical outcomes and patient follow-up was poorly documented.31

Even in the absence of logistical difficulties, compliance is an issue, as is demonstrated by the low rates of outcomes reporting even in HICs. As mentioned above, only about 10% of US hospitals currently participate in NSQIP, the American College of Surgeons’ quality initiative. The US-based Anesthesia Quality Institute (AQI) has launched another reporting forum that, while growing, is still voluntary.32 The AQI has identified this same need for safe anaesthesia in LMICs and has created a separate registry specifically for use by NGOs, but to date only one publication has documented its use.33 As safe anaesthesia care is inextricable from safe surgery, compliance with anaesthesia outcomes reporting is an integral part of surgery outcomes reporting.

When social media and journalists fill the information void, results range from effective whistle-blowing to inaccurate sensationalism. It can be difficult to separate one from the other. An older but instructive example is the 1999 New York Times exposé of avoidable deaths of patients undergoing relatively simple procedures for repair of cleft lips and cleft palates.34 Among other accusations reported in the article, a paediatric surgeon in Bolivia branded the organisation’s practices ‘a form of neo-colonialism’; eventually, the reports led to structural changes in the involved charities. More recently, the online news site for the American Anthropological Association published an ethnographic research study on women in Niger presurgery and postsurgery for obstetric fistula.35 After 18 months of interviews of over 100 women, their data make a convincing argument that surgical outcomes statistics were sharply skewed and did not correlate with the results experienced by the patients. While such incongruity may be explicable by the difference between patient-related outcome measures and what is routinely measured in surgeons’ outcomes, their findings are nonetheless compelling and call for this elucidation.

Conversely, while Google, Yahoo, social media sites, ReliefWeb and so on can supply situational awareness to potential volunteers as well as host nation communities, enabling them to organise, coordinate and identify sources of support and assistance, they are not peer-reviewed and may not be factual. Noelle Sullivan has written for Scientific American 36 and Huffington Post 37 regarding ‘medical voluntourism’, reporting examples of untrained volunteers performing surgical procedures. In some of her examples, she makes no mention of whether or not a qualified surgeon was overseeing the procedure (as is the practice in virtually every surgical residency in the USA.) In fact, one of the ‘voluntourists’ she named has written a rebuttal with that very piece of information—that she was assisting trained professionals.38 Thus, cross-disciplinary reporting can sensationalise and undermine good-faith efforts that may actually be well designed and effective.

Finally, the economics of surgical and medical missions impact outcomes reporting. The drive to keep overhead costs low results in a vicious cycle39 that ‘is leaving nonprofits so hungry for decent infrastructure that they can barely function as organizations, let alone serve their beneficiaries. The cycle starts with funders’ unrealistic expectations about how much running a nonprofit costs and results in nonprofits’ misrepresenting their costs while skimping on vital systems – acts that feed funders’ skewed beliefs’.40 Although we cannot fully address economic impact in this article, it is a critical obstacle and others have examined its role.41

Suggested mechanisms for reporting outcomes

Global surgical leadership

We are not the first to recognise this void and suggest a solution. The WHO has taken a lead on the collection of long-term data on the six core surgical indicators as outlined by the Lancet Commission on Global Surgery with case studies of eight countries or regions with differing levels of surgical capabilities.42 Others have followed suit in other parts of the world.43 These six core indicators are widely recognised by the international community as a starting point for evaluating surgical capabilities of a host nation and thus can be a standard for evaluation. However, only one of the key indicators assesses quality (via perioperative mortality rate). As WHO and its partners continue to evaluate needs and build roadmaps for improvement, the next step will be to build on this single quality-related indicator to require more thorough assessment of outcomes themselves (similar to the NSQIP measures in the USA). These large-scale interventions will promote a culture of outcomes assessment as regions’ surgical systems are developing, which will pre-empt a much larger task of having to retroactively put outcomes monitoring systems into place. Industry-recognised standards can provide leverage for host governments to build and enforce policy whenever that is possible in their individual contexts. In this regard, several industry standards currently exist and could be used to greater effect by both NGOs and LMIC governments. These include the WHO Surgical Safety Checklist,44 which has repeatedly been shown to reduce mortality and morbidity after surgery, and the quality assurance and perioperative care guidelines used by cleft surgery NGOs.45–47

While recognising the need for concise and focused goals in such a large-scale project, we suggest that more robust outcomes reporting be added to the mandate of world authorities in surgical care, thereby encouraging smaller entities, NGOs and individual surgeons to include this in their preparation for trips overseas.

An added benefit of having a mandate from global leadership is that the obstacle of follow-up will be somewhat mitigated. Even if individual patients cannot be followed due to cultural, societal and geographical barriers, trends would be much more easily followed via centralised, required reporting. Coordination between teams and local governments and within teams themselves would have a framework. Teams that go to areas that have already been served would have a standardised approach, common goals, and, after some amount of time, outcomes data to guide decision-making.

Web-based, tax-incentivised reporting

A successful pattern has been implemented in the realm of charity supervision by websites such as Charity Navigator48 and CharityWatch. CharityWatch49 has carried out well-publicised exposés of non-profit abuses, of which the most recent was the alleged wasteful spending by the Wounded Warrior Project. For groups involved with practising medicine or surgery, especially in areas of the world where legal systems do not tightly regulate healthcare, the mandate of these watchdog organisations could be enlarged to include outcomes reporting. NGOs and various foundations receiving tax-free status could be required to report outcomes (mortality, morbidity, specific morbidities such as surgical site infections, hospital-acquired infections, return to operating room and so on) and host-nation survey responses in their annual reports. Surveys would be compiled both from patients and from host government and healthcare authorities, adding a necessary component of cultural applicability (ie, which outcomes are important to given cultures or communities). Because NGOs and charities depend largely on donations, these watchdog organisations greatly help with quality improvement by providing an avenue for transparency and accountability that is easily accessible to the public.

Journalism and social media

As mentioned above, journalists and social media platforms play necessary roles and add the value of large audiences and fast dissemination of information. Journalists can be contacted during the planning stages of a global health endeavour, whether small or large scale, functioning as a cross-disciplinary quality check; they can be outfitted with outcomes measures that apply to a given mission; and they can be included in the data gathering. For example, the WHO Disability Assessment Schedule 2.0 evaluates patient-related outcomes measures50 and has been used successfully in multiple settings.51 52 Interested academic groups such as the Society of Environmental Journalists (www.sej.org) and Association of Health Care Journalists (http://www.ahcj.umn.edu/) with strong university ties may be entrusted with the task of reporting outcomes of humanitarian missions. Of course, as stated earlier, since journalists’ coverage may lack a clinical knowledge base, it must be balanced by an accessible and transparent database of surgeons’ own outcomes reporting.

The widespread availability of social media could be used to assess patient satisfaction and what the people on the ground are actually saying. Internet and social media access is not universal, but both are steadily rising in LMICs.53 Several such tools are available, such as the X1 Social Discovery,54 which aggregates social media content (Facebook, Twitter, Instagram, You Tube, Tumblr) and web-based data into a single user interface. The mHealth Programme, an arm of the Groupe Spéciale Mobile Association’s ‘Mobile for Development’ initiative, is operating with a similar mission; they provide a helpful ‘design toolkit’ for building mobile platforms that are user-centric and ‘bring end-users into service development process’.55 It is possible that well-designed mobile services could overcome cultural barriers to accurate reporting. Even though these data would not be given the same weight as formal outcomes reporting, it is likely to give some level of useful data regarding patient satisfaction and improvement in quality of life. The legal implications of using such data are still not defined.

Conclusions

There is currently an exponential growth of humanitarian missions, with no legal or professional requirements to report outcomes data to national or international registries or in annual reports, press releases or websites. While journalists and social media platforms provide a necessary voice, we, as the providers, have the expertise and the tools to evaluate ourselves. Even as we work towards improving outcomes reporting in HICs, we must also proactively promote safe and ethical surgery56 in LMICs as we join them in building their own surgical infrastructure.