Accounting for complexity - Intervention design in the context of studying social accountability for reproductive health.

Background: Social accountability interventions aim to propel change by raising community voices and holding duty bearers accountable for delivering on rights and entitlements. Evidence on the role of such interventions for improving community health outcomes is steadily emerging, including for sexual and reproductive health and rights (SRHR). However, these interventions are complex social processes with numerous actors, multiple components, and a highly influential local context. Unsurprisingly, determining the mechanisms of change and what outcomes may be transferable to other similar settings can be a challenge. We report our methodological considerations to account for complexity in a social accountability intervention exploring contraceptive uptake and use in Ghana and Tanzania. Main Body: The Community and Provider driven Social Accountability Intervention (CaPSAI) study explores the relationship between a health facility-focused social accountability intervention and contraceptive service provision in two countries. This 24-month mixed-method quasi-experimental study, using an interrupted time series with a parallel control group, is being undertaken in 16 sites across Ghana and Tanzania in collaboration with local research and implementation partners. The primary outcomes include changes in contraceptive uptake and use. We also measure outcomes related to current social accountability theories of change and undertake a process evaluation. We present three design components: aspects of co-design, 'conceptual' fidelity, and how we aim to track the intervention as 'intended vs. implemented' to explore how the intervention could be responsive to the embedded routines, local contextual realities, and the processual nature of the social accountability intervention. Conclusions: Through a discussion of these design components and their rationale, we conclude by suggesting approaches to intervention design that may go some way in responding to recent challenges in accounting for social accountability interventions, bearing relevance for evaluating health system interventions. Copyright:

Abbreviations

CaPSAI – Community and Provider driven Social Accountability Intervention

HRP - UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research)

SRHR – Sexual and reproductive health and research

WHO – World Health Organization

Disclaimer

The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated.

Introduction

The importance of taking a complexity approach to evaluating ‘real world’ health interventions has now been well established ( Craig ; Greenhalgh & Papoutsi, 2018; Moore ; Portela ). Social accountability interventions, which have gained recognition as a part of health systems strengthening and raising community voice, are associated with an increasing range of health benefits ( Kruk ; Schaaf ; Van Belle ). As a result, there is growing interest in how to understand, assess, and scale successful results. We explore the design of a social accountability intervention evaluated in a two-country study aiming to improve quality of care in order to increase contraceptive uptake and use and present three design components that responded to and aimed to account for complexity.

Social accountability interventions aim to propel community-driven change and empower citizens and communities to hold duty bearers accountable for promised rights and entitlements ( Joshi, 2017). These interventions aim to be community-owned and led and improve life for local citizens by raising their voices, representing their interests, and increasing their capabilities, ultimately transforming power relations. Accountability in the context of sexual and reproductive health and rights has been described as ‘the appropriate prioritisation of sexual and reproductive health and rights (SRHR) and its implementation throughout the health system and ensuring access to SRHR services, with attention to high-quality and respectful care.’ ( Boydell ). Social accountability is conceptualised as able to bring about change through a series of activities over time. These may include community education and empowerment, increasing the understanding of rights and entitlements, community mobilisation and data collection, a process of evaluation and measurement against standards and priorities, and a process of interfacing between duty bearers and rights-holders, with service users working to hold duty bearers to account. The process of interfacing can be in the form of meetings, public hearings, or other forums where community members can interface with power holders and each other to share concerns, apply pressure, and track change. Social accountability is an ongoing contingent and often political process, and it operates both within and outside of formal structures and processes. It is, amongst other things, evidently complex. However, how social accountability interventions work, whether the theories of change are accurate, and what the key ingredients for success are across contexts and health topics require more empirical insight, particularly as such interventions are taken on by mainstream health actors and implementers across a range of settings.

A recent supplement on complexity approaches, and public health guidance describes the multiple component nature, nonlinear causal pathways, role of local context, and general under examination of outcomes as particular challenges for health interventions with multiple priorities and limited resources ( Portela ). Updated guidance on implementation studies was described by Greenhalgh and Papoutsi as emphasising ‘‘‘non-linearity and iterative local tailoring’ and placed substantially more emphasis on the need for non-experimental, mixed methods and process-based approaches for studying such phenomena’ ( Greenhalgh & Papoutsi, 2018; Moore ). A WHO convened community of practice on measuring and evaluating social accountability interventions for reproductive, maternal, and child health reported similar considerations with specific emphasis on power relations and the political nature of accountability interventions ( Boydell ).

Here we explore the Community and Provider driven Social Accountability Intervention (CaPSAI) ( Steyn ). Considering the social accountability process as a complex intervention, with numerous actors, a highly influential local context, and a number of interacting components, as well as being a process over time, it was considered essential to move beyond simple intervention thinking and call on complexity approaches to design and evaluate the CaPSAI study. This paper explores how the CaPSAI intervention was designed to respond to real-life conditions and how recent thinking in implementation science and complex intervention evaluation were considered in developing the CaPSAI intervention. We consider three design components and how they aimed to account for complexity; intervention fidelity, elements of co-design, and the intervention as ‘intended vs. implemented.’

The Community and Provider driven Social Accountability Intervention (CaPSAI)

Social accountability is complex, and therefore not suited to mainstream notions of interventions that do not allow for multiple and interrelated factors iteratively and simultaneously contributing to a change process ( Dasgupta, 2011; Schaaf & Dasgupta, 2019). Moreover, there are many approaches to social accountability `( Schaaf & Dasgupta, 2019) For some, social accountability is as a form of performance management in which service users are consumers who can use these mechanisms to demand better services ( Ringold ). These are characterized a short-term, bounded tactical intervention, in which tools are used in a particular time and place. At the opposite end of the continuum are approaches that understand social accountability as a process whereby service users, as citizens, demand their legal rights and contribute to social transformation ( Lopez Franco & Shankland, 2018). These tend to be more strategic approaches that deploy multiple tactics to combine citizen voice and public sector responsiveness and actively try to address power asymmetries ( Joshi & Houtzager, 2012). CaPSAI tries to bridge the two ends of the continuum, by codifying the approach of two national groups that have been at the forefront of transparency and accountability for years into a short-term interventions.

The Community and Provider driven Social Accountability Intervention (CaPSAI) study aims to make a robust addition to the literature and evidence base on participatory and social accountability processes for health. It is one of the first studies to evaluate the effectiveness of social accountability interventions on behaviour related to family planning. It aims to build the evidence base on the potential for such interventions to improve SRHR. Literature and evidence on accountability strategies to improve SRHR is steadily emerging ( Boydell ; Gullo ; Van Belle ) though challenges remain in understanding how best to evaluate these programmes and determine best practices for scale-up. The CaPSAI Project has been registered at Australian New Zealand Clinical Trials Registry ( ACTRN12619000378123, 11/03/2019). CaPSAI is a quasi-experimental, mixed methods evaluation implemented across 16 sites in Ghana and in Tanzania. The intervention was delivered and evaluated by local civil society partners and research organisations over a period of 24 months. Specifically, it explores the role of such interventions for aspects of SRHR by evaluating the impact and process of implementation of an eight-step social accountability intervention on contraceptive uptake and use in low resource settings with low modern contraceptive uptake ( Moore ). The CaPSAI study aims to describe and examine how social accountability processes are implemented and operationalised, focusing on behaviours, decision-making processes, and the barriers and facilitators of change. The findings aim to be generalisable to other like settings. It also aims to develop more responsive quantitative measures for social accountability interventions and demonstrate the relationship between social accountability processes and the uptake and use of contraceptives and other family planning behaviours (See Steyn for details on the research protocol).

In the CaPSAI study design, contraceptive uptake is evaluated through an interrupted time series design with a control group (ITS-CG). A cohort of women who are new users of contraception is tracked using standardised interview questions across both intervention and control facilities to measure changes in behaviours around contraceptive use over one year. To capture social accountability intermediate outcomes, such as empowerment of women and health providers and expansion of negotiated spaces, a cross-sectional survey using accountability-related psychometric scales is conducted at pre- and post-intervention phases. The effects of the social accountability intervention and the implementation process are measured through a process evaluation comprising context mapping, qualitative interviews, document review, and implementation tracking. Case studies of change are also collected. A process evaluation was seen as essential due to the complex and processual nature of the social accountability intervention and the challenges in determining causal chains and clearly attributing outcomes to intervention inputs ( Moore ; Palmer ).

Figure 1 outlines the CaPSAI theory of change, and Table 1 describes the eight identified steps in more detail. While these are referred to as ‘steps,’ they may be better conceptualised as phases and may contain a number of activities or ranges of activities within each step. We also acknowledge that social accountability interventions are best considered as a process and not as linear discrete steps or tools that will necessarily combine to create social change; however, these steps present a structure for the intervention.

Figure 1.

CaPSAI Theory of Change (This figure has been reproduced with permission from Steyn ).

Table 1.

Standard steps in the CaPSAI social accountability process.

Step	Description
1.Introduction of the intervention to the community	The implementation partner (usually a civil society organization) meets with local leaders, identifies stakeholders and sets up the infrastructure to deliver the community scorecard.
2.Mobilization of participants for the intervention	The implementing partner will gather community partners, service providers and the user groups of the health facility/ services to describe the intervention. The implementing civil society organisation should also identify a respected community mobilizer who will jointly facilitate the roll out of the intervention and join the implementation team.
3.Health, rights and civic education with community participants	The implementation partner shares information on existing service standards and provides training on rights, good governance and accountability. The group begins to rate existing services against rights-based standards and generate discussion about local priorities.
4.Prioritization meeting with community	The implementation partner distills themes and priorities raised by the community. The community groups then collectively score the issues and indicators and set priority areas for action.
5.Prioritization meeting with duty bearers	The implementation partner distills themes and priorities raised by the service providers. The providers then collectively score the issues and indicators and set priority areas for action.
6.Interface meeting and joint action planning	The implementation partner then holds a joint meeting between the community, the service providers and other duty bearers. Following the presentation of results from the prioritization meetings the community groups and the service providers will aim to reach consensus on the ranking of the priority items and the actions required to address them. An action plan with assigned roles and responsibilities will be developed for the following 6–12 month period.
7.First follow-up meeting with community and duty bearers at three months	Priority areas and action items will be followed up with both the community and service providers. For any unresolved issues these meetings present an opportunity to involve high-level duty bearers of third party pressure (media/ politicians) to increase the pressure to act.
8.Second follow-up meeting with community duty bearers at six months	A second follow up meeting will enable the monitoring of longer range outcomes and on the remedy of unresolved issues raised in the first follow up meeting. The community and service providers will continue to monitor the action plan for changes in relation to agreed priority areas.

Key design features in enabling and accounting for local adaptation

The social accountability process is deeply situated in, and contingent upon, the local context and involves multiple actors and factors that can be difficult to account for, inevitably presenting challenges for evaluation. Each process will be and should be different as it responds to locally determined concerns and power relations. As social accountability interventions are not singular discrete interventions, obtaining mainstream achievements and measurements of fidelity, dose, and reach and ensuring knowledge of the ‘active ingredients that allowed the outcomes to take hold can be particularly challenging ( Boydell ; Craig ; Moore ) Thus, in designing the intervention and its evaluation the team considered emerging research and best practice on complex intervention design and evaluation. Table 2 describes dimensions of complexity present in the CaPSAI intervention. We now present three broad considerations: aspects of co-design, considering ‘conceptual fidelity’ versus standardisation, and accounting for the intervention as intended versus implemented.

Table 2.

Accounting for complexity in the Community and Provider driven Social Accountability Intervention (CaPSAI) study design (This table has been reproduced with permission from Steyn ).

Dimension of complexity	CaPSAI dimension	MRC recommended design features	CaPSAI study design features
A large number of interactions between components within the intervention ( Craig et al., 2008).	CaPSAI intervention requires separate and joint activities with community organizations, health providers, and duty bearers to produce an effective space for collective action and change.	A theoretical understanding of how the intervention causes change ( Craig et al., 2008).	CaPSAI developed a theory of change ( Figure 1)
A number of behaviors required by those delivering or receiving the intervention ( Craig et al., 2008).	Behavior change in varying degrees on the part of community members, health providers, and duty bearers is required for effects to take hold.	A process evaluation design to study the implementation process to address ‘how’ the intervention worked in practice and better understand the ‘active ingredients’ ( Craig et al., 2008; Moore et al., 2015).	Process evaluation is a main component of the study design
A number of groups or organizational levels targeted by the intervention ( Craig et al., 2008).	CaPSAI intervention targets community members, health care providers and duty-bearers at the community and facility level.	A larger sample size and cluster rather than individual level designs ( Craig et al., 2008).	Evaluation and sampling at both the community (individual) and health facility level (cluster).
Numerous and variable outcomes ( Craig et al., 2008).	The primary outcomes include an increase in contraceptive uptake and indicators of contraceptive use alongside intermediary outcomes such as increases in social capital, collective efficacy, and empowerment.	Use of a range and mix of measures and methods to capture complexity and unintended consequences ( Craig et al., 2008; Moore et al., 2015).	A range of methods and instruments aim to capture the primary and intermediary outcomes as well as the implementation process itself.
A degree of flexibility or tailoring of the intervention is permitted ( Craig et al., 2008).	Implementation should maintain conceptual fidelity to the eight standard steps but allows for local adaptation to produce the effects.	Fidelity should be considered ‘functionally rather than Compositionally ( Fox, 2015) to allow interventions to be responsive to context while still being meaningfully evaluated ( Craig et al., 2008; McMullen et al., 2015; Moore et al., 2015).	The process evaluation and combination of research instruments have been designed to capture functional fidelity in the implementation

Aspects of Co-design

Alongside the growth of implementation science has been calls for greater uptake of co-design approaches, particularly in the context of complex interventions ( Craig ; Moore ). Co-design stresses equal participation, particularly of ‘end users’, and recognises that interventions will be more responsive and likely to deliver meaningful results if all those with a stake in their delivery and outcomes are involved in all aspects of intervention design, delivery, and evaluation ( Donetto ; Goodyear Smith ; Slattery ). For the CaPSAI study, local civil society organisations with experience in delivering health-related social accountability interventions were selected as implementing partners. Implementing partners then selected community representatives to become members of the implementation and research teams. Community members were also tasked with the facilitation and implementation of the intervention across the 16 sites. In some cases, these community members had already worked with the civil society organisations; in others, they were newly recruited. In order to recognise the wealth of experience and established routines and practices of implementing partners as well as respond to the local context, aspects of the intervention was co-designed by the study implementation leads and civil society implementing partners from the local community.

Stages in the intervention design

Stage 1:      The first stage in the design of the intervention was a review of existing literature and programmes related to social accountability and health ( Boydell & Keesbury, 2014). Programme descriptions, evidence, and programme reports for health-related social accountability interventions such as community scorecards, report cards, citizen voice, accountability, and citizen hearings were gathered to define the key phases in the social accountability process (See Table 3). These data were brought together with the findings from the formative phase study UPTAKE Project ( Cordero ; Steyn ) and emerging findings from the Evidence Project studies on social accountability in the context of family planning ( Boydell ) to set the groundwork and proof of concept for CaPSAI and were used to develop the overarching theory of change. From a review, a composite of components (‘steps’) were determined that are typical of social accountability interventions and theories of change (see Table 3). The steps identified were: introduction of the project, community mobilization, rights training with the community, separate prioritization meetings with community and health providers, interface meeting and action planning, monitoring and evaluation. This stage of the research was desk based and researcher led. It set up the base material for the team to iterate from. These ‘steps’ structured the second phase of the intervention design process.

Table 3.

Toolkits reviewed to develop the intervention.

Document Title	Author	Date of publication	Source
Information and collective action in community-based monitoring of schools: Field and lab experimental evidence from Uganda	Abigail Barr, F. Mugisha, P. Serneels, A. Zeitlin	Aug.12	https://pdfs.semanticscholar.org/99f5/806ab361f3308d652b9549b390e4f183b672.pdf
Enhancing governance and health system accountability for people centered healthcare: an exploratory study of community scorecards in Afghanistan	Edward Anbrasi, Kojo Osei-Bonsu, Casey Branchini, Temor Shah Targhal, Said Habib, Arwal A.	Jul.15	http://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-015-0946-5
When is community based monitoring effective? Evidence from a randomized experiment in primary health in Uganda	Bjorkman, M. and Svensson, J.	Apr.10	https://academic.oup.com/jeea/article-abstract/8/2-3/571/2296039?login=false
Scorecards and social accountability for improved maternal and newborn health services: A pilot in the Ashanti and Volta regions of Ghana	Blake, C., Annorbah- Sarpei, N., Ismaila, Y., Clark, S.	Sep.16	https://www.sciencedirect.com/science/article/pii/S0020729216304428
Citizen Voice and Action	Jeff Hall	Jul.14	https://www.escr-net.org/node/366893
East African Community Regional Reproductive Maternal Newborn and Child Health Scorecard	Elizabeth Muiruri	Nov.14	https://everyone.savethechildren.net/sites/everyone.savethechildren.net/files/ EAC%20RMNCH%20Scorecard.pdf
Tanzania Community Score Card Transparency and Accountability Project	Sandra Michaelson	Nov.13
Scorecard Toolkit: CARE Malaw	Sarah Gullo	May.13	https://resourcecentre.savethechildren.net/sites/default/files/documents/6800.pdf
Maternal Health Alliance Malawi	Sarah Gullo	Nov.14
Role of Social Accountability in Improving Health Outcomes: Overview and Analysis of Selected INGO Experiences	K.D. Hoffman	Jun.14	https://coregroup.org/resource-library/the-role-of-social-accountability-in-improving- health-outcomes-overview-and-analysis-of-selected-international-ngo-experiences-to-advance-the-field/ to-advance-the-field/
Citizen Voice and Action (as reviewed by "Role of Social Accountabilty" paper	K.D. Hoffman	Jun.14
Partnership Defined Quality Framework	K.D. Hoffman	Jun.14
CARE's Community Scorecard	K.D. Hoffman	Jun.14	https://www.care.org/wp-content/uploads/2020/05/FP-2013-CARE_ CommunityScoreCardToolkit.pdf
White Ribbon Alliance Social Watch Approach	K.D. Hoffman	Jun.14	https://coregroup.org/wp-content/uploads/media-backup/documents/Resources/ Tools/Social_Accountability_Final_online.pdf
White Ribbon Alliance Uganda Participatory Health Facility Assessment	K.D. Hoffman	Jun.14
GOAL Accountability can Transform Health	K.D. Hoffman	Jun.14
The community scorecard process: methodology, use, successes, challenges, and opportunities	Jephter Mwanza and Nina Ghambi	Dec.11	http://pubs.iied.org/pdfs/G03207.pdf
Effects of CARE’s Community Score Card on reproductive health-related outcomes	Sara Gullo, Christine Galavotti, Anne Sebert Kuhlmann, Thumbiko Msiska, Phil Hastings, C. Nathan Marti	Feb.17	http://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0171316&type= printable
Social Accountability Services: Case Study 1: Andhra Pradesh, India: Improving Health Services through Community Score Cards	Vivek Mistra, P. Ramasankar, Lakshmi Durga, Sanjay Agarwal, J.V.R. Murty	Aug.07	https://namati.org/resources/case-study-1-andhra-pradesh-india-improving-health- services-through-community-score-cards/
Governance and Transparency Fund, Environmental Governance, Ghana, the Community Scorecard Approach	Governance and Transparency Programme	2014	https://washmatters.wateraid.org
Evaluation and Design of Social Accountability Component of the Protection of Basic Services Project, Ethiopia	Samuel Taddesse, Biraj Swain, Merga Afeta, Gadissa Bultosa	Jun.10	http://docplayer.net/123994238-Evaluation-report-final-evaluation-and-design-of- social-accountability-component-of-the-protection-of-basic-services-project-ethiopia. html
Transparency for development baseline report, Tanzania	Jean Arkedis, Jessica Creighton, Archon Fung, Stephen Kosack, Dan Levy, Rohit Naimpally, Lindsey Roots and Courtney Tolmie	Sep.16	http://t4d.ash.harvard.edu/sites/default/files/file-uploads/Full%20T4D%20Baseline%2 0Report%20FINAL.pdf
Accountability in Local Service Delivery: The Tuungane Community Scorecard Approach	Guillaume Labrecque and Isatou Batonon	May.15	https://www.rescue.org/sites/default/files/document/660/ accountabilityinlocalservicedeliveryenglishfinal.pdf

Stage 2:      For each of the social accountability ‘steps’ a set of questions was developed that aimed to elicit the routine practices, knowledge, experiences, and concerns of the two national non-governmental organisations implementing the activities. Understanding the existing routines and roles and the ‘normal practice’ of implementation as they are highly contextual and they are how implementers structure and make sense of their roles and worlds ( Greenhalgh, 2008, p1269). The key questions aimed to ascertain the ways in which the implementing partners are already addressing the core aspects of intervention fidelity (as considered functionally ( Hawe ) in their regular practice and existing implementation strategies for existing social accountability programmes. Understanding these practices in advance allows for designing an intervention that better reflects what may take place in ‘actual’ implementation by not suggesting new or changed practices where they are not necessary. Findings from the first and second design stages were synthesised into a guide for design stages three and four.

Stage 3:      In the third stage of the intervention design, multiple meetings were held over a period of months with the implementation teams prior to the start of implementation. Initial meetings introduced the study and the objectives. Subsequent meetings engaged a discussion structured around the findings of the review and the key questions. The tentative theory of change and identified ‘steps’ were used as prompts to discuss and explain previous experiences of the implementing partners in delivering social accountability interventions. The meetings were led by the implementation leads who used the sessions as an adapted form of a ‘focus group workshop’ to consider the different intervention steps, elicit similar experiences and learn what best practice would look like in the context where implementation would take place. This discussion led to an additional step in the proposed process, as implementing partners stressed the importance of more than one round of interfacing to monitor progress in addressing joint action plans in their previous work. Discussions were recorded and loosely transcribed, and notes were fed back to the team and used to ‘build out’ the intervention. Through an iterative process, an intervention manual to use during the study was put together. The manual includes direct feedback from this stage of design and a list of key considerations derived from these discussions for each step of the process. These quotes, key considerations and key questions are all derived from the previous experience of the implementers in the settings where the intervention was to be delivered ( WHO, 2021).

The intervention manual is written in a ‘workbook’ style and does not set out an overly prescriptive or standardised intervention to be delivered. The manual sets out the aims and objectives of the study, along with essential information for study conduct and implementation as part of a research project (which was new for the implementation teams). For each of the eight social accountability ‘steps,’ the manual describes how the step is conceptualised within the theory of change and lists key questions and considerations for implementers with examples that emerged from the co-design process. Workbook pages are included for pre and post-implementation. While the study intervention implementation manual was primarily designed by the implementation team, comprised of WHO team members and civil society partners, the site-specific implementation plans involve the local facilitators and community implementers across the 16 sites. The pre-implementation plan is where implementers set out how their plans adhere to the core tenets of the intervention step and respond to key criteria and concerns, essentially how ‘fidelity’ is composed. This plan is then reviewed by the implementing teams with the implementation leads to discuss how it meets the requirements of the study and achieves fidelity. The post-implementation report allows implementers to account for implementation ‘on the day’ and note any deviations from the plan or to remark on exceptional events.

Stage 4:      In the final stages, feedback on the draft manual was gathered, and further refinements were made, and the ‘workbook’ aspect of the manual was further considered. Finally, the teams worked to finalise the planned intervention and agree on a final manual. Training then took place with the local implementing teams over a period of days.

Tracking the intervention was developed to enable and account for local adaptation. As reflected in the design of the implementation manual, fidelity to the theory of change and core aspects of the intervention is key, and how this fidelity is composed is expected to vary. The (re)consideration of fidelity is one of the intervention design features that aims to account for the complex and contextual nature of social accountability interventions and their success.

Intervention ‘fidelity’

Recent literature acknowledges that adaptive intervention strategies yield more responsive and localised interventions that may respond better to community needs ( Greenhalgh & Papoutsi, 2018). Some thinking in implementation science and complex interventions indicates the value of considering intervention integrity and fidelity based on whether it achieves its purpose (functionality) as opposed as to whether it has the pre-determined components (compositional fidelity) ( Greenhalgh & Papoutsi, 2018; Hawe ; McMullen ; Perez ). In measuring complex interventions alongside quantitative study designs, this conceptualisation of fidelity has been suggested as a potentially more responsive approach to assessing the integrity of interventions. This requires distilling the essential criteria required to have fidelity to the overarching tenets of the intervention but moves away from an overly prescribed and standardised pathway of implementation. Hawe ask what standardisation is in a complex intervention and suggest ‘rather than defining the components of the intervention as standard … what should be defined as standard are the steps in the change process that the elements are purporting to facilitate or the key functions that they are meant to have’ (pg.1561). The ‘workbook’ style of the implementation plan for each step develops a form of ‘mini’ site-specific protocol for each intervention step, that adheres to the core aspects that support the theory of change while accounting for local context and adaptability ( McMullen ). The implementation plans and design also respond to the design of the process evaluation. As described by Palmer , ‘ A key challenge is how to find a balance between the fluidity that complexity and process so obviously warrant and the development of process evaluation aims, questions and procedures in advance (pg 2).’

A complex social intervention such as a social accountability process, implemented by organisations with previous experience and established routines and practices indicates that the intervention as envisaged prior to implementation will differ slightly from actual implementation ‘on the day.’ To account for this reality, a ‘pre-implementation plan and post-implementation report ‘is included as a part of the intervention design, to be completed before and after each step of the social accountability process.

The intervention ‘as intended’ versus ‘as implemented’

The pre-implementation plans and post-implementation reports form a part of the document review for the process evaluation research team, alongside the qualitative interviews, context mapping, and case studies of change. The reports support an understanding of the dose, reach, and fidelity of the intervention while accounting for and enabling adaptability.

This feature allows the intervention ‘as intended’ and the intervention ‘as implemented’ to be tracked and to account for divergences at the reporting stage. This may allow for a better understanding of the ‘active ingredients’ in the implementation process and for a better description of what took place over the course of the study. This will assist evaluators in assessing whether the theory of change was accurate, what the causal pathways for intervention outcomes may be, and what may be essential for scalability and generalisability. Alongside the pre-implementation plans and post-intervention reports, the process evaluation contains a method where small case studies of change are gathered. Through in-depth qualitative interviews and document review, reported instances of change thought to be attributed to the social accountability intervention are explored. Here researchers can trace the instances of change and gather the accounts of local actors as to how these changes took place, gathering relevant documentation and triangulating interviews as needed. Drawing on ethnographic methods, these combined accounts help develop the picture represented by the quantitative findings with rich accounts of the intervention in action and local perceptions and descriptions of impact.

Conclusion

As described by Hawe and Shiell back in 2004, ‘ reducing a complex system to its component parts amounts to an “irretrievable loss of what makes it a system”’ ( Hawe ; p. 1562). ‘Real world’ interventions also experience real-world pressures such as budget constraints, tight timelines, international teams, and so on. Trying to incorporate emerging best practice that challenges the status quo can present challenges. What is described in this paper reflects efforts to incorporate some of the relevant guidance to support a complex and political intervention on what is often a controversial and contested set of rights and health behaviours. This is a reflection on the design process, and the results of the study will be reported, upon completion, elsewhere. We do not suggest that the considerations outlined here can resolve the tension between local and contextual intervention outcomes and their ability to scale and generalize across other settings but do hope that the considerations shared here will go some way to better describing and accounting for the complexity acknowledged in delivering social accountability interventions. Aspects of co-design of the intervention, a conceptualisation of fidelity that is as open and adaptive as possible, and using ethnographic approaches to track the intervention steps as intended versus as implemented as part of a process evaluation have been how we considered complexity in the intervention design for the CaPSAI study.

Data availability

No underlying data are associated with this article.

I have read the authors' response and I am satisfied with their revisions. I have no further comments to make.

Does the article adequately reference differing views and opinions?

Yes

Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes

Is the Open Letter written in accessible language?

Yes

Where applicable, are recommendations and next steps explained clearly for others to follow?

Partly

Is the rationale for the Open Letter provided in sufficient detail?

Yes

Reviewer Expertise:

Design and evaluation of complex health and social interventions in low resource settings; interdisciplinary and mixed methods approaches.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

I appreciate the attention paid to the earlier comments and the added materials and revisions in the text of the paper. Nonetheless, I regret that a point I have made in the earlier review has apparently not been acknowledged.

The revised paper still does not mention how or when the context analysis was undertaken even while it mentions that the process of ‘co-designing’ recognized the ‘wealth of experience’ and tried to ‘respond to the local context’.

Without analysis of the context as a preliminary step in designing any SRHR intervention, there remains a risk that the strategy adopted may not be the optimal one to address the issue.

For instance, it is not mentioned whether the current barriers to low usage of modern contraceptives were analyzed at any stage by the team: were they cultural/gender norms, or supply-side constraints/community scarcity or maybe there were some health system challenges in terms of low accountability to the end-users of SRHR services.

Without analysis of the barriers, or examination of possible alternative strategies to address the issue and its root causes, there is an implicit decision made that social accountability will best serve the purpose towards attaining the intended SRHR outcomes. Yet the paper does not clarify the reasons why this decision was made, or at what stage it was assumed that the social accountability process would be the most effective answer to low usage of modern contraceptives.

This fundamental limitation is not acknowledged anywhere, which leads this paper to promote the rather risky approach that context analysis, understanding the root causes and considering various alternative/complementary strategies are not really needed. The paper may tacitly indicate that merely following the eight steps (with attention to co-design, fidelity and intention-implementation comparisons) is sufficient to ensure outcomes.

Does the article adequately reference differing views and opinions?

Partly

Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes

Is the Open Letter written in accessible language?

Yes

Where applicable, are recommendations and next steps explained clearly for others to follow?

Not applicable

Is the rationale for the Open Letter provided in sufficient detail?

Yes

Reviewer Expertise:

Gender and equity analysis, rights-based approaches, social accountability, maternal health, sexual and reproductive health and rights, choice and adolescent issues, social and political determinants of health, public health systems governance

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Response: Apologies for not responding to your comments in the earlier response to comments. The CaPSAI Project was developed in line with the World Health Organization guideline: “Ensuring human rights in the provision of contraceptive information and services”, which identified nine key health and human rights standards that need to be strengthened: non-discrimination, availability, accessibility, acceptability, Quality of Care (QoC), participation, informed decision-making and choice, privacy/confidentiality and accountability (WHO 2014). The project implemented tested social accountability processes to promote community monitoring and social accountability of contraceptive programs. To do this, we identified national civil society organization using social accountability and were already working or wanting to work on sexual and reproductive health.

We want to clarify that two types of contextual analysis were undertaken as part of the larger study and project.  The first was led by the research team at the start of each project year to better understand the current social accountability and SRHR programmes that were underway in the locations where the project was taking place. The second was undertaken by the national CSOs facilitating the social accountability activities with the community, they undertook an analysis of the SRHR context locally as part of the project preparation work and then this was discussed at the start of the community engagement process.  It was felt that the national partners supporting the social accountability processes were best placed to undertake the analysis of the local and wider barriers to SRHR as the processes have to be responsive to local contexts.  Though many of these barriers are beyond the reach of social accountability processes (which can often be locally bounded if they are not linked up to wider mobilization efforts) and as you point out, this can limit accountability efforts, it was the national partners in conjunction with their community partners who decided what was the most appropriate barriers to tackle based on their analysis.

Thank you for your careful consideration of our manuscript.

This open letter reports the methodological considerations for designing and evaluating CaPSAI, a social accountability intervention exploring contraceptive uptake and use in Ghana and Tanzania. While the letter focuses specifically on a social accountability intervention addressing reproductive health and rights, it makes important contributions to the design of complex interventions more generally, and therefore, will be of interest to readers working in complex health and social interventions research.

The letter addresses an important issue in complex interventions research – how to account for complexity as part of the contingent and emergent links between the intervention, individuals, and society. Given the challenges with representing the intersecting and intertwining processes of designing a complex intervention, I commend the authors for their work on this letter.

Throughout the letter, the authors emphasize the notion of complexity as contingent. What is missing, however, is an exploration of why accounting for complexity in this way is important. The letter would be strengthened with a short explanation of what gets lost or omitted when taking the mainstream notion of intervention as a linear process and a static collection of materials – or alternatively, what is gained when taking a more contingent and context-specific approach to complexity and design.This could be included in the introduction – for example, revising paragraph 4 to provide more explanation of their view/approach to complexity, rather than references that describe acknowledgements of complexity.

The letter would also be strengthened with a short description of the intervention itself. There is a heading with a few paragraphs describing the aims of the CaPSAI and evaluation; however, there is no description of the actual intervention.  What is the social accountability mechanism?  Table 1 describes a community scorecard, but no further explanation is given. It is also unclear what aspects of the intervention were decided/set in advance, and what aspects were co-designed. For example, was the community scorecard decided by the implementation team, then the approach to implementation (i.e. the “intervention manual”) co-designed with the implementing partners?

Additionally, the letter would be strengthened with more methodological detail about the stages in the intervention design process to understand how, and to what extent, the aims of co-design were achieved. At present, it is not clear who was involved in each intervention design stage and what methods were used. For example, who conducted the reviews in Stage 1? What (participatory?) methods were used to ‘elicit the routine practices…’ (page 6) in Stage 2 and who synthesized findings from Stage 1 and 2?  Who facilitated the meetings in Stage 3 and which team ‘built out’ the intervention (page 6)? This information would also help readers looking towards similar approaches for intervention design.

Finally, in the section ‘The intervention ‘as intended’ versus ‘as implemented’, the authors suggest their approach “…may allow for a better understanding of the ‘active ingredients’ in the implementation process…” (page 10) – my emphasis. This is interesting because it is a departure from the mainstream thinking that focuses on distilling intervention components and disregards how interventions are implemented in situ. This distinction between implementation versus intervention is also touched on somewhat in the section on ‘Intervention ‘fidelity’’, but overall, the nuance of this argument gets lost in the text. The letter would be improved with more clearly articulated discussion of this distinction and what a focus on implementation offers before going into describing how this was achieved.

Does the article adequately reference differing views and opinions?

Yes

Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes

Is the Open Letter written in accessible language?

Yes

Where applicable, are recommendations and next steps explained clearly for others to follow?

Partly

Is the rationale for the Open Letter provided in sufficient detail?

Yes

Reviewer Expertise:

Design and evaluation of complex health and social interventions in low resource settings; interdisciplinary and mixed methods approaches.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Thank you for your comments.  We found them helpful for improving our paper and have made a number of changes.  These changes are listed below.

EDIT 10:

Throughout the letter, the authors emphasize the notion of complexity as contingent. What is missing, however, is an exploration of why accounting for complexity in this way is important. The letter would be strengthened with a short explanation of what gets lost or omitted when taking the mainstream notion of intervention as a linear process and a static collection of materials – or alternatively, what is gained when taking a more contingent and context-specific approach to complexity and design.This could be included in the introduction – for example, revising paragraph 4 to provide more explanation of their view/approach to complexity, rather than references that describe acknowledgements of complexity.

Response: We have added a paragraph on the CaPSAI approach to social accountability and why this is important under the section describing the project.

EDIT 11:

The letter would also be strengthened with a short description of the intervention itself. There is a heading with a few paragraphs describing the aims of the CaPSAI and evaluation; however, there is no description of the actual intervention.  What is the social accountability mechanism?  Table 1 describes a community scorecard, but no further explanation is given. It is also unclear what aspects of the intervention were decided/set in advance, and what aspects were co-designed. For example, was the community scorecard decided by the implementation team, then the approach to implementation (i.e. the “intervention manual”) co-designed with the implementing partners?

Response: Thank  you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI.  Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 12:

Additionally, the letter would be strengthened with more methodological detail about the stages in the intervention design process to understand how, and to what extent, the aims of co-design were achieved. At present, it is not clear who was involved in each intervention design stage and what methods were used. For example, who conducted the reviews in Stage 1? What (participatory?) methods were used to ‘elicit the routine practices…’ (page 6) in Stage 2 and who synthesized findings from Stage 1 and 2?  Who facilitated the meetings in Stage 3 and which team ‘built out’ the intervention (page 6)? This information would also help readers looking towards similar approaches for intervention design.

Response: As per our response to reviewer 1, EDIT 4, we now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI.

EDIT 13:

Finally, in the section ‘The intervention ‘as intended’ versus ‘as implemented’, the authors suggest their approach “…may allow for a better understanding of the ‘active ingredients’ in the implementation process…” (page 10) – my emphasis. This is interesting because it is a departure from the mainstream thinking that focuses on distilling intervention components and disregards how interventions are implemented in situ. This distinction between implementation versus intervention is also touched on somewhat in the section on ‘Intervention ‘fidelity’’, but overall, the nuance of this argument gets lost in the text. The letter would be improved with more clearly articulated discussion of this distinction and what a focus on implementation offers before going into describing how this was achieved.

Response: We address this in the paragraph under the description of CaPSAI on our approach to social accountability.

Summary of the article - it describes a way of designing the implementation and evaluation of the CaPSAI intervention that used Social Accountability methods to improve service uptake for contraceptives. The way it was designed quite deliberately uses complex intervention evaluation as well as implementation science. The three key aspects of the designing that are highlighted in the paper include co-designing, intervention fidelity and tracking the 'intended vs implemented' processes throughout the eight steps of the programme. It is indicated that a separate article will bring out the outcomes and findings of the CaPSAI programme.

With regard to my comment about inadequate 'reference to differing views and opinions', my suggestions to improve the paper are as below:

Table 1 sets out the steps of the intervention, based on the various programme interventions studied as per Table 3. However the list as indicated in Table 3 is limited to short-duration interventions by international actors/donors/  researchers who set out to make an intervention in an under-resourced setting in the global South. The objective is to improve contraceptive usage, a 'civilizing objective' based on the assumptions that citizen voice and facilitated negotiations with duty bearers will overcome health system challenges, commodity scarcity and cultural/gender  norms.

Joshi and Houtzager (2012 ) have usefully made the distinction between 'widgets and watchdogs' and define Social Accountability as 'long-term ongoing political engagement of social actors with the state' as opposed to short term project interventions initiated by researchers or donors that seek to bring about linear processes of change based on the formulaic application of 'steps' of social accountability with insufficient analysis of local context.

The paper would have been improved by stating clearly these limitations and referring to this literature that refers to the profoundly political nature of social accountability processes that seek to confront power asymmetries.

The paper would be strengthened by discussing some of the characteristics of the specific local context: for instance by analyzing whether absence of citizen voice and provider accountability are the only barriers to contraceptive use or whether health system challenges, commodity scarcity and cultural/gender  norms may be at the root of low contraceptive uptake in the area.

The paper does not sufficiently refer to literature that acknowledges the non-linear and iterative nature of change processes that work within complex socio-political contexts of public-service provision in most under-resourced health systems of LMICs (refer to Schaaf and Dasgupta, 2019 , Balestra et al., 2018 , Dasgupta, 2011 and so forth).

Does the article adequately reference differing views and opinions?

Partly

Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes

Is the Open Letter written in accessible language?

Yes

Where applicable, are recommendations and next steps explained clearly for others to follow?

Not applicable

Is the rationale for the Open Letter provided in sufficient detail?

Yes

Reviewer Expertise:

Gender and equity analysis, rights-based approaches, social accountability, maternal health, sexual and reproductive health and rights, choice and adolescent issues, social and political determinants of health, public health systems governance

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Thank you for your comments.  We found them helpful for improving our paper and have made a number of changes.  These changes are listed below.

EDIT 6:

Table 1 sets out the steps of the intervention, based on the various programme interventions studied as per Table 3. However the list as indicated in Table 3 is limited to short-duration interventions by international actors/donors/  researchers who set out to make an intervention in an under-resourced setting in the global South. The objective is to improve contraceptive usage, a 'civilizing objective' based on the assumptions that citizen voice and facilitated negotiations with duty bearers will overcome health system challenges, commodity scarcity and cultural/gender  norms.

EDIT 7:

Joshi and Houtzager (2012) have usefully made the distinction between 'widgets and watchdogs' and define Social Accountability as 'long-term ongoing political engagement of social actors with the state' as opposed to short term project interventions initiated by researchers or donors that seek to bring about linear processes of change based on the formulaic application of 'steps' of social accountability with insufficient analysis of local context.

The paper would have been improved by stating clearly these limitations and referring to this literature that refers to the profoundly political nature of social accountability processes that seek to confront power asymmetries.

Response:  As per our response to Review 1, EDIT 5, we have added a section under the description of CaPSAI to describe our approach to social accountability.

EDIT 8:

The paper would be strengthened by discussing some of the characteristics of the specific local context: for instance by analyzing whether absence of citizen voice and provider accountability are the only barriers to contraceptive use or whether health system challenges, commodity scarcity and cultural/gender  norms may be at the root of low contraceptive uptake in the area.

Response: Thank you for this comment. The focus of this paper is describing the methodology of co-desiging the intervention. Other findings from the study, including the context mapping will be reported elsewhere.

EDIT 9:

The paper does not sufficiently refer to literature that acknowledges the non-linear and iterative nature of change processes that work within complex socio-political contexts of public-service provision in most under-resourced health systems of LMICs (refer to Schaaf and Dasgupta, 2019; Balestra et al., 2018, Dasgupta, 2011 and so forth).

Response: Thank you for Reviewers 2 for their suggestions on addressing the issue of   nonlinearity of social accountability interventions and for directing us to literature. We have added the paragraph under the description of the CaPSAI Project to describe our approach to social accountability and how this fits within the literature on social accountability and included additional references.

This open letter introduces the CaPSAI (Community and Provider driven Social Accountability Intervention) study which explores the impact of a social accountability process on contraceptive uptake and use in two countries. More specifically, the letter reflects on the design process, which, recognizing that the intervention is a ‘complex social change process’ required specific methodological considerations.

The letter will be of most interest to readers with closely aligned interests in the design of complex interventions tackling social change processes, and seeking to improve the methodological robustness of such studies. Despite being relatively short it is dense with both conceptual and process-related detail.

Allowing for that target audience, I found the letter to contain some important and interesting considerations, in particular, the way ‘conceptual fidelity’ was operationalized as a design feature within the study. This is the first time I personally have seen/read about this being done (at least in such explicit terms) and it provides an excellent example of a methodological approach that can clearly help account for complexity within a quasi-experimental study design. The potential for applying this approach to other areas of research where ‘controlling for context’ is inappropriate seem substantial.

The (preceding) description of *co-design* and later description of *intended vs actual* process tracking felt less novel. Particularly co-design since this is a long-advocated approach to building meaningful and context-specific research (or other endeavors). As the authors note there is now a substantial literature on ‘co-design’ specifically; and there is a long history of participatory approaches including PAR that have incorporated related approaches to research or interventions. Nonetheless, and bearing in mind the likely readership for this letter is one seeking detailed examples of study design, I do see the value in the descriptions provided.

There are several areas in which the letter could be strengthened. First, careful editing of the front two sections (Intro / CaPSAI section) to reduce the duplicative mentions of the study’s aims and approach would be worthwhile. A single, clear and precise articulation of the study’s aims (potentially sub-titled for ease of reference) in the Introduction or early in the following CaPSAI sub-section would suffice. There are currently a number of slightly different descriptions (e.g. Intro, para 1, final sentence / Intro para 4, first sentence / CaPSAI, para 1, ‘Specifically…’ ) which affect the clarity of writing and comprehension.

With regards to the 3 main ‘features’ – consistency of language around the way they are described would also be helpful. Are co-design, conceptual fidelity and accounting for intended vs implemented activities, ‘features’ or ‘considerations’ or ‘approaches’?

The 4 th paragraph of the Introduction could be much clearer; rather than long introductions to the sources of information, a focus on the nature of the knowledge/methodological developments in the field of complexity research with reference to the source only via citations would help the reader focus on the key information and not get lost in the names of institutions and journals.

The Co-design section: I think this would be elevated by having a tighter summary of the 'stages of design' process, in favour of more explicit reflection on how the co-design process was influenced by, and adapted to manage, power dynamics among the many stakeholders, which is an obvious feature of complexity that co-design is, in part, there to address. This might be prompted by responding to the question: how this co-design process differed from a more straightforward, albeit extensive consultation? Notwithstanding that there is the *intention* to recognise local stakeholders' experience as expertise for example, it wasn't clear to me how the co-design stages mitigated the tendency for external experts to dominate decision points. It would be helpful to illustrate examples of adaptations to the study design (away from the original theory of change for example) arising from the co-design process.

Finally, throughout the letter, there remains a not insubstantial tension between the repeated assertion of the way complexity design must account for context specificity and non-linear pathways on the one hand, and the idea that knowledge arising from these studies will be scalable and generalizable on the other hand. One of the reasons I find the description of ‘conceptual fidelity’ compelling, is precisely because it provides an example of how to plot a methodological path through the epistemological tension created by applying a quasi-experimental study design to a research problem which is contextually contingent. But at a broader level throughout this letter, that tension between describing a study design that produces findings which account for complexity, and the expectation that those findings will be scalable or generalizable does not receive much attention; given the earlier observation that the key readership for this letter is likely to be those with a keen methodological interest, closer reflection on that tension by the authors is recommended.

Does the article adequately reference differing views and opinions?

Yes

Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes

Is the Open Letter written in accessible language?

Yes

Where applicable, are recommendations and next steps explained clearly for others to follow?

Partly

Is the rationale for the Open Letter provided in sufficient detail?

Yes

Reviewer Expertise:

Health systems research with a focus on governance, accountability and trust; qualitative methodologies.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Thank you for your comments.  We found them helpful for improving our paper and have made a number of changes.  These changes are listed below.

EDIT 1:

First, careful editing of the front two sections (Intro / CaPSAI section) to reduce the duplicative mentions of the study’s aims and approach would be worthwhile. A single, clear and precise articulation of the study’s aims (potentially sub-titled for ease of reference) in the Introduction or early in the following CaPSAI sub-section would suffice. There are currently a number of slightly different descriptions (e.g. Intro, para 1, final sentence / Intro para 4, first sentence / CaPSAI, para 1, ‘Specifically…’ ) which affect the clarity of writing and comprehension.

Response: We have removed some of the duplication regarding the aims of the CaPSAI intervention from the introduction.

EDIT 2:

With regards to the 3 main ‘features’ – consistency of language around the way they are described would also be helpful. Are co-design, conceptual fidelity and accounting for intended vs implemented activities, ‘features’ or ‘considerations’ or ‘approaches’?

Response: We have standardised language across the text to refer to the three design components versus features, considerations or approaches.

EDIT 3:

The 4th paragraph of the Introduction could be much clearer; rather than long introductions to the sources of information, a focus on the nature of the knowledge/methodological developments in the field of complexity research with reference to the source only via citations would help the reader focus on the key information and not get lost in the names of institutions and journals.

Response: We have removed references to institutions in the Introduction paragraph as suggested, to improve readability.

EDIT 4:

The Co-design section: I think this would be elevated by having a tighter summary of the 'stages of design' process, in favour of more explicit reflection on how the co-design process was influenced by, and adapted to manage, power dynamics among the many stakeholders, which is an obvious feature of complexity that co-design is, in part, there to address. This might be prompted by responding to the question: how this co-design process differed from a more straightforward, albeit extensive consultation? Notwithstanding that there is the *intention* to recognise local stakeholders' experience as expertise for example, it wasn't clear to me how the co-design stages mitigated the tendency for external experts to dominate decision points. It would be helpful to illustrate examples of adaptations to the study design (away from the original theory of change for example) arising from the co-design process.

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI.  Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the study intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 5:

Finally, throughout the letter, there remains a not insubstantial tension between the repeated assertion of the way complexity design must account for context specificity and non-linear pathways on the one hand, and the idea that knowledge arising from these studies will be scalable and generalizable on the other hand. One of the reasons I find the description of ‘conceptual fidelity’ compelling, is precisely because it provides an example of how to plot a methodological path through the epistemological tension created by applying a quasi-experimental study design to a research problem which is contextually contingent. But at a broader level throughout this letter, that tension between describing a study design that produces findings which account for complexity, and the expectation that those findings will be scalable or generalizable does not receive much attention; given the earlier observation that the key readership for this letter is likely to be those with a keen methodological interest, closer reflection on that tension by the authors is recommended.

Response: Thank you for Reviewers 1 and 2 for their suggestions on addressing the issue of   nonlinearity of social accountability interventions and for directing us to literature.  We have added a paragraph under the CaPSAI study description to describe our approach to social accountability. We acknowledge that there are many approaches to social accountability. On one end, it is viewed as a short-term, bounded tactical intervention and at the opposite end of the continuum are approaches that understand social accountability as a process whereby service users, as citizens, demand their legal rights and contribute to social transformation. CaPSAI aims to bridge the two ends of the continuum by codifying the approach of two national groups that have been at the forefront of transparency and accountability for years into short-term interventions.

We also added a few lines addressing generalizability in the conclusion.