Influencing policy change: the experience of health think tanks in low- and middle-income countries.

In recent years there has been a growth in the number of independent health policy analysis institutes in low- and middle-income countries which has occurred in response to the limitation of government analytical capacity and pressures associated with democratization. This study aimed to: (i) investigate the contribution made by health policy analysis institutes in low- and middle-income countries to health policy agenda setting, formulation, implementation and monitoring and evaluation; and (ii) assess which factors, including organizational form and structure, support the role of health policy analysis institutes in low- and middle-income countries in terms of positively contributing to health policy. Six case studies of health policy analysis institutes in Bangladesh, Ghana, India, South Africa, Uganda and Vietnam were conducted including two NGOs, two university and two government-owned policy analysis institutes. Case studies drew on document review, analysis of financial information, semi-structured interviews with staff and other stakeholders, and iterative feedback of draft findings. Some of the institutes had made major contributions to policy development in their respective countries. All of the institutes were actively engaged in providing policy advice and most undertook policy-relevant research. Relatively few were engaged in conducting policy dialogues, or systematic reviews, or commissioning research. Much of the work undertaken by institutes was driven by requests from government or donors, and the primary outputs for most institutes were research reports, frequently combined with verbal briefings. Several factors were critical in supporting effective policy engagement. These included a supportive policy environment, some degree of independence in governance and financing, and strong links to policy makers that facilitate trust and influence. While the formal relationship of the institute to government was not found to be critical, units within government faced considerable difficulties.

Under the right conditions, health policy analysis institutes can play a positive role in promoting evidence-informed decision making in government.

Factors critical in supporting effective policy engagement include: a supportive policy environment, some degree of independence in governance and financing, and strong links to policy makers that facilitate trust and influence.

Motivation and capacity within government to process and apply policy advice developed by a health policy analysis institute was found to be key to the institute’s ultimate success.

Introduction

Government agencies play a critical role in developing and supporting the implementation of policy ideas. However, there are sometimes problems with ‘in-house’ policy analysis (James 2000; Nathan Associates Inc. 2004). For example, civil servants may lack independence, being heavily swayed by what the Minister wants to hear or they may be short-termist in outlook, focusing more on fighting fires than developing a long-term strategy. The quality of analytical work conducted by civil servants may suffer due to a lack of capacity or lack of incentives for high quality analysis, and civil service structures may become stagnant, resulting in a lack of ‘fresh thinking’. Finally government agencies may be ill-equipped to foster broad public engagement in policy.

In recent years there has been a growth in the number of independent ‘think tanks’ in low- and middle-income countries and particularly in Eastern Europe and the Former Soviet Union (Stone ). These institutes are, in part, an attempt to respond to the challenges associated with in-house policy analysis, described above. This trend has also influenced the health sector, which has seen the development of health policy analysis institutes (HPAIs), learning platforms and observatories. A landscaping exercise conducted for this study found a total of 78 health policy analysis institutes in low- and middle-income countries (of which 38 were in Asia, 21 in Africa, 8 in Latin America, 8 in Europe and the Former Soviet Union and 3 in the Middle East). Given that these institutes were identified solely through searching existing databases, this figure probably underestimates the number of such institutes, particularly in Latin America and the Middle East. Over 80% of the HPAIs identified were established after 1990.

The development of HPAIs has been catalyzed by democratization processes that have both facilitated the development of non-governmental organizations and opened up national policy processes. In addition new information technologies, such as the world wide web, have helped promote transparency and hence greater accountability of government to civil society, and thus have also increased pressure to ensure that policy development takes heed of available evidence (Pina ).

In light of the growth in the number and importance of HPAIs, this study set out to:

investigate the contribution made by HPAIs in low- and middle-income countries to health policy agenda setting, formulation, implementation, and monitoring and evaluation;

assess which factors, including organizational form and structure, support the role of HPAIs in low- and middle-income countries in terms of positively contributing to health policy.

The findings reported here are part of a broader study that also investigated the factors affecting the capacity and sustainability of HPAIs. These other findings have been reported separately (Bennett and Corluka 2010).

Review of relevant literature

There has been virtually nothing previously written about HPAIs, or indeed any form of specialist think tank. Stone and others acknowledge the existence of specialist think tanks, but the literature appears to have very little to say about their relative advantages and disadvantages. James (2000) has argued that specialized think tanks are typically better able to work on the micro details of policy implementation, rather than broader policy issues, and this may be a particular niche for them. Further, Braun argue for the importance of this neglected area.

However, there is a growing body of evidence from the general literature regarding best practices in promoting the use of research evidence in policy (see, for example, Innvaer ; Lavis ; Yaron and Shaxson 2008). Both this evidence and studies of policy analysis institutes in general (Nathan Associates Inc. 2004) concur that there are a few key factors that contribute to success in terms of influencing policy and practice. These include:

The timeliness and relevance of findings;

The production of credible and trustworthy reports;

Close personal contacts with policy makers;

Summaries of findings that present key actionable recommendations.

Autonomy is often held to be a core characteristic of think tanks: it is this element that can enable policy analysis institutes to be critical and to take a long-term perspective. However, it is difficult to pin down exactly what constitutes autonomy. While financial independence may be the most commonly considered form, there are other dimensions such as administrative and intellectual autonomy (James 1993; McGann and Johnson 2005). Osman and El Nolla (2009) identify 10 different factors affecting autonomy ranging from funding modalities, managerial control over issues such as recruitment, the research agenda-setting process, quality assurance mechanisms and the existence of ‘advisory firewalls’ such as technical advisory bodies that can help protect the integrity and independence of research conducted.

Regional differences in the character and institutional affiliations of think tanks (Osman and El Nolla 2009) may substantially affect the nature of their relationship with government. For example, think tanks in the US are typically highly independent non-profit organizations, whereas Europe is inclined to a more mixed model that depends both on public and private financing. In Asia, particularly East Asia, government-sponsored think tanks appear more common. There is no one ‘right’ model for policy analysis institutes; ensuring a good fit between the model and the socio-political context in which it operates is perhaps most critical. In this light, some developing country authors (e.g. Ojagbohunmi 1990; Osman and El Nolla 2009) have suggested that think tanks sponsored by, or incorporated within, government structures may be the most appropriate model for developing countries as they combine reliable long-term financial support with direct opportunities for influencing policy.

Lastly, policy analysis institutes may engage government at different points in the policy cycle. For example, they may seek to influence agenda setting, the selection of particular policy options, policy implementation or to participate in the evaluation of existing policies. These different steps in the cycle have different characteristics and accordingly policy analysis institutes occupying a different niche may require different types of organizational capacity to be effective (Global Development Network 2009). Policy analysis institutes with a high media profile, for example, may be more effective at political agenda setting than lower profile institutions (Abelson 2002).

Definitions and methods

For the purposes of this study a HPAI was understood to:

Have the overall purpose of supporting health policy development and implementation through analysis and research;

Perform at least two of the following functions:

- Conducting policy-relevant research and analysis;

- Providing policy advice and technical assistance in policy formulation and evaluation;

- Conducting policy dialogues at national and international levels, that is bringing together policy makers, civil society and researchers to draw upon evidence and debate key policy questions;

- Training and capacity development for policy makers;

Take any one of multiple organizational forms, but possess some degree of autonomy, and not be profit oriented;

Have health policy makers as its primary clients although also serve secondary clients such as civil society organizations (including service providers and advocacy groups) and senior managers within the health system.

Thus, HPAIs were understood to range from being an almost integral part of a Ministry of Health, to being embedded in a university, or being an entirely separate private, non-profit organization.

A case study approach was used as it provides a structured approach to studying complex causal relationships through the in-depth study of a limited number of cases. It is an appropriate research method where multiple related factors are of interest and the relationship between them is not clear and may evolve over time. Cases were selected using the diverse case technique (Gerring 2007, p. 97): we sought to identify cases that were diverse in terms of their organizational forms, specifically including one NGO, one university and one government-owned policy analysis institute from both Asia and Africa. In addition, institutes selected for inclusion were to (i) have been established for a minimum of 5 years and (ii) have an explicit focus on the health sector. Institutes that met these criteria were identified from a database of HPAIs that was developed by the authors. The final set of selected case studies (Table 1) depended not only upon the criteria identified above, but also the willing participation of the institution itself. Unfortunately, the Centre for Health and Social Services (CHeSS) in Ghana had not been established for 5 years; however, attempts to study another institute in Ghana failed, leading the research team to select CHeSS instead.

Table 1

Overview of case study institutes

Institute & country	Year of foundation	Legal status	Current situation	No. of key informant interviews
Health Strategy and Policy Institute (HSPI), Vietnam	1987; 1998 in its current form	Public entity under jurisdiction of Ministry of Health	Regarded as an effective player in informing policy debates nationally	17
Health Economics Unit (HEU), South Africa	1990	Formally established unit within School of Public Health and Family Medicine, University of Cape Town	Well established and well respected both nationally and internationally	15
Institute for Health Systems (IHS), India	1990	NGO, registered as a society	Has had many changes in fortune, currently re-establishing itself after financial difficulties and about to open major new training programme	17
Health Economics Institute (HEI), Bangladesh	1998	Formally established institute within Department of Economics, University of Dhaka	Now receives minimal funding and health policy analysis functions have dramatically declined. The institute is considering revising its mission and mandate.	13
Health Policy Analysis Unit (HPAU), Uganda	1999	Integral to Ministry of Health	Now receives minimal funding, and its position within the Ministry of Health hierarchy has declined	13
Centre for Health and Social Services (CHeSS), Ghana	2008	Registered NGO	Still in early phases of development	7

Each of the case studies was conducted by researchers from the country or region concerned, who were familiar with the institute but not a member of it. A common detailed protocol and semi-structured questionnaires were developed to guide data collection in each country. Data collection occurred between June 2009 and January 2010, and the main data sources were the following: For each case study a database of evidence was compiled that included data from the various sources identified above, such as electronic versions of reports, transcripts from interviews and a record of the debriefing with institute staff. In most cases interview data were transcribed and analysed by hand according to the central themes of the study. In some instances interviews were not recorded but detailed notes were made of the interviews. Reports on each institute were developed by the respective case study authors (de-Graft Aikins 2009; Doherty 2009; Hussain 2009; Kyabaggu and Namaganda 2009; Tangcharoensathien and Patcharanarumol 2009; Jesani 2010). Both these reports and primary data were used to develop the final synthesis of findings.

Document review including published material from the institute itself (website, research publications, annual reports, published strategies and plans etc.) and from other sources, as well as unpublished material (such as donor agreements);

Financial information from the institute;

Semi-structured key informant interviews with a variety of purposively selected individuals who have different types of engagement with the institute, such as founders of the institute, staff members, funders, members of the institute board and clients of the institute including policy makers and civil society; and

Discussion of the draft report with staff members of the institute.

Findings

Overview of study institutes

In Ghana, India and South Africa the institutes had been established by respected individuals in the field. For the other three cases the institutes were largely established through organizational agreements. For example, Vietnam’s Health Strategy and Policy Institute (HSPI) evolved from a series of past institutions. The institutes in Bangladesh and Uganda were both established by government, with strong support from external funding agencies.

During the course of the research it became apparent that since their establishment, the evolutionary paths of these institutions have diverged (as reflected in the penultimate column of Table 1). The Health Economics Unit (HEU) at the University of Cape Town continues to operate on a relatively small scale, but has weathered substantial volatility in the health policy and funding environment and has become a highly respected research institute, at both national and international levels. The Vietnamese HSPI is far less well known internationally but appears to be an effective and well-respected player domestically. Further, it has managed to establish a broad funding portfolio and relatively large and stable staffing. The fortunes of the Institute for Health Systems (IHS), India have varied over time. The institutes in Bangladesh and Uganda received substantial core budgetary support from donors at start-up, but when these initial grants ended, the institutes found it difficult to find alternative funding sources to replace them. Both institutes have since contracted significantly in terms of staffing, volume of work and budget.

Impacts on health policy

Both HSPI, Vietnam and HEU, South Africa were perceived to have made major contributions to policy development in their respective countries. IHS, India also seems to have contributed at state and national levels. CHeSS, Ghana was too new to have made any such contributions, although informants felt that it has the potential to do so. At the Health Economics Institute (HEI), Bangladesh and the Health Policy Analysis Unit (HPAU), Uganda, the influence that the institutes once had evaporated with diminished budgets. Respondents in Uganda pointed to several instances where opportunities to draw in domestic research evidence were missed, due to the lack of an effective policy analysis institute.

In Vietnam, informants were of the opinion that HSPI had made important contributions to several policy development processes, including the national policy on injury prevention (2002), the national strategy on preventive medicine (ongoing) and the draft law on Health Insurance (2007), as well as the development of a health sector master plan for several provinces and cities. Government respondents in South Africa also cited multiple ways in which HEU contributed to policy; areas frequently identified included health equity, health financing, drug policy, primary health care and district health systems. When asked to give examples of HEU’s impact on policy, one government official said:

“Oh, there are several … I don’t know where to start. The work that they’ve done around the user fees in the public facilities, the work around medicine pricing, the work around costing of tertiary services, perceptions of the public around the public health system. I mean there’s a whole host of research work that they’ve done that’s actually influenced policy.” (Government Official, South Africa)

IHS, India helped draw national policy makers’ attention to cause-of-death statistics and was one of the first stakeholders within India to contribute to the conceptualization of family health insurance policy. In addition, IHS, India contributed at the state level to government’s efforts to improve the health system.

Many of the institutes had conducted analyses around issues relating to health financing (health insurance and user fees); the role of the private sector; development assistance (SWAps and the effectiveness of donor assistance); and hospital autonomy. This surprising degree of commonality in the themes and issues that the different institutes address possibly reflects typical areas where Health Ministries do not have sufficient internal expertise, as well as commonality in policy concerns. It also points to opportunities for engagement between institutes from different countries.

Strategies pursued by institutes to achieve impacts

Table 2 shows the different functions carried out by the case study institutes. Every institution was actively involved in the provision of policy advice, and almost all (with the exception of the Ugandan HPAU) also undertook policy-relevant research. Institutes frequently responded to ad hoc requests from government for policy briefs or specific analyses, although none of the institutes had a clearly defined process for developing research and analytical priorities with government. In terms of providing policy advice, institutes sought to influence government policy not only indirectly through the publications they produced but also directly through formal means (such as participation in government advisory committees, or ministerial meetings) and informal contacts with policy makers and other stakeholders (such as non-governmental organizations) that could influence policy. Strategies employed to influence policy varied both by the position of the institute, and according to the nature of the policy issue under discussion.

Table 2

Strategies carried out by case study institutes

Strategies	HEI, Bangladesh	CHeSS, Ghana	IHS, India	HEU, South Africa	HPAU, Uganda	HSPI, Vietnam
Conducting policy-relevant research and analysis	***	***	***	***	**	***
Providing policy advice and technical assistance in policy formulation and evaluation	***	***	***	***	***	***
Conducting policy dialogues at national levels	**	***	–	**	–	**
Conducting policy dialogues at international levels	–	–	**	**	–	–
Training and capacity development for policy makers	***	*	*	***	–	–
Conduct systematic reviews	–	–	–	**	–	–
Commission research or reviews	–	–	–	–	–	–

Notes:

***Actively engaged.

**Done occasionally.

*Intended but not currently done.

– Not done.

In the case studies, training emerged as a crucial mechanism helping to strengthen the links between the institutes and policy makers. The two university groups and IHS, India were most actively engaged in training and capacity development for policy and decision makers (although given funding constraints HEI, Bangladesh has not been very active in this area recently). While HEU, South Africa originally intended to focus on research, over time it evolved a stronger focus on capacity development activities and training programmes targeted at an audience from across Africa, and this now makes up a core part of its activities. This change in strategy was in recognition of the dearth of health economics capacity in Africa, but also reflected the fact that teaching became an important avenue through which to feed back research findings to health service officials, as well as keep HEU staff well informed about policy makers’ concerns, thus contributing to the relevance of their work.

CHeSS, Ghana was the only institute to be actively engaged in running policy dialogues at the national level. Respondents in Ghana suggested that CHeSS could play a critical convening role, helping to bring together different types of actors who might have something to contribute to health systems strengthening. This convening role was rarely associated with the other institutes studied, possibly reflecting difficulties in sourcing funds for such activities.

None of the case study institutions was actively engaged in commissioning research and HEU, South Africa was the only institute that conducted systematic reviews, albeit on an occasional basis.

Engagement with mass media appeared somewhat limited. Only HEU, South Africa has a communications officer, and this post was only recently filled. Indeed it is only recently that HEU, South Africa has begun to engage with journalists in a more proactive manner. Respondents noted that this was largely sparked by an incident where a report of the African National Congress’s task team on National Health Insurance was leaked to the media, resulting in much misinterpretation which HEU staff attempted to correct through newspaper articles and interviews. Similarly, the experience of IHS, India in engaging with mass media has been somewhat mixed, and occasionally the institute has found itself having to defend work it has done. Further, the institute found that media engagement tended to take up a substantial amount of the time of senior staff. HEU is just beginning to undertake background briefings for journalists on health economics issues in South Africa as a means to try to raise the general level of health literacy in the media. It is noticeable that HSPI Vietnam, while having a large staff does not have a communications officer. Presumably the close and trusted relationship between the institute and the Ministry might actually inhibit broader engagement via other communication channels.

Outputs produced

Much of the work undertaken by the institutes appears to have been driven by requests from government or donors. For example, in 2001–02, HEI, Bangladesh responded to ad hoc policy advice requests from the ministry so as to produce briefing papers on topics such as procurement and effectiveness of donor assistance, user fees, costing of essential (health) services packages (ESP), and health insurance. HSPI, Vietnam responds to ministry requests for health strategy and policy advice and appraises prospective policies for approval as required by the government or the National Assembly, as well as evaluating current policies. Importantly though, HEU, South Africa also conducted self-initiated research which had a longer-term outlook and enabled it to provide advice readily once government was receptive.

The primary outputs of research are often research reports, frequently combined with verbal briefings to government officials. Products from institutes also encompass manuals and actions plans, indicating the very practical work that such institutes often undertake. Both HEU, South Africa and HSPI, Vietnam case studies revealed the primacy of dissemination through personal engagement with policy makers and senior managers, either through project-related processes (such as project meetings and feedback workshops) or through participation in policy-making committees. For example, the participation of the head of HSPI, Vietnam in the weekly meetings chaired by the Minister, and his informal interactions with the Minister, were viewed to be important channels for influence. Such face-to-face contact was found to be particularly important in terms of transferring ideas, keeping an ‘ear to the ground’ and maintaining a high profile.

Only HEU, South Africa and IHS, India publish articles in peer-reviewed journals, or books and book chapters. Key informants at both HEU, South Africa and HSPI, Vietnam highlighted the time and workload constraints to publishing more research, especially in international peer-reviewed journals. In contexts where a key constraint is the availability of skilled human resources, there are clear trade-offs between focusing on informing and influencing government health policy, and getting research findings published.

Factors influencing the nature of policy engagement

A number of factors emerged from the case studies as being critical determinants of the ability of institutes to engage effectively, over time, in policy discussions. These factors include: These are considered in turn.

The broader policy environment;

The ownership and status of the institute;

The governance and financing of the institute;

Institute leadership.

Policy environment

Perhaps the most important single factor influencing successful institute development is a supportive environment, specifically in terms of a demand from government for independent analysis. In Vietnam this has clearly been a positive factor supporting the development of HSPI, and in India policy makers at the state level clearly articulated a demand for evidence to inform the decision-making process:

“Nowadays this is an era of evidence-based decision making. Policy makers need more information or evidence to support their decisions, not just from their thought. It is a new environment which happens not only in health sector but also other sectors or in other words it is for all, throughout Vietnam.” (Government official, Vietnam)

“Now very precious time, precious resources, precious opportunities are lost or forgotten because I have not been given the benefit of advice … And if they are able to tell me this is what happened in Maharashtra or Gujarat or some other country, these are the ways they have improved the health services, here is the evidence for that. I think it becomes much easier for state government particularly to focus on those areas and whenever there are any contrary kind of ideas coming up from the political system, we can juxtapose this and tell them … look this is the evidence we have and that’s why we are doing this …” (Government official, India)

In contrast, in both Uganda and Bangladesh, while the cessation of funding was the most visible factor leading to the decline of the institutes, in fact the underlying factor in both cases was attributed by respondents to lack of government support for the unit. This was clearly the case in Bangladesh, where the incoming government rejected the organizational reforms implemented by its predecessor, but more broadly respondents also questioned the commitment of the government to using evidence in policy. Respondents in Uganda raised similar questions with respect to the decline of HPAU:

“… Do they not see what is happening? Are they not interested? So at the end of the day you cannot blame the person in the unit, you have to blame the people at the top … they have not provided the resources, the leadership to correct this situation. They have shown some level of disinterest, maybe they also do not appreciate the importance of policy analysis …” (External stakeholder, Uganda)

In South Africa, HEU was established 4 years before South Africa’s first democratic elections in 1994, at a time when there were the beginnings of a new climate of openness, and in particular openness to critiques of the apartheid health system. However, over the years there have been periods when government has been more or less receptive to HEU advice, and there have been particular policy issues where HEU has had to play much more of an advocacy role than attempt to influence policy directly. Its ability to shift between these roles bears testimony to its independence and quality of work, but also to a diversified financial base that few of the other case study HPAIs have. The experience of HEU also points to the importance of a wider policy community that includes civil society organizations, and regional and international networks.

Institute ownership

The policy think tank literature stresses the importance of a location outside of government in order to maintain a neutral and potentially critical stance. Two of the case study institutes, HPAU in Uganda and HSPI in Vietnam, were very closely associated with government. For HPAU, the fact that it was embedded within government appeared to have brought largely negative consequences. Even during the period when it was well funded it is clear that its position within the organizational structure created difficulties: while it was meant to provide advice directly to the permanent secretary, the hierarchical culture of decision making at the Ministry made this a difficult arrangement to manage. Further, after the decline of World Bank funding the HPAU became entirely dependent upon government funding, and was not in a position to act entrepreneurially to mobilize resources for itself.

By contrast, the arrangement of HSPI, Vietnam appears to have worked relatively well. While the close relationship between HSPI and the ministry raised some outsider criticism of the independence of HSPI research and advice, the institute appears to have had a considerable degree of influence upon policy. In the relatively closed policy-making environment of Vietnam, it is difficult to imagine an entirely external institute achieving the same degree of influence. One respondent seemed to suggest that part of the reason why HSPI was so much trusted by the Ministry of Health was the very close organizational relationship between the two:

“We trust HSPI as HSPI is a part of MOH. They are very keen in research, especially health system and health policy research. In addition, HSPI will be responsible on whatever the impacts of their recommendations are … As for the [name of external agency], I don’t trust them: they come and go.” (Government official, Vietnam)

It has been suggested that being located in an academic setting might mean that an institute would conduct less policy relevant work (Nathan Associates Inc. 2004). This does not appear to be the case with the South African institute:

“No, I don’t think HEU suffers from that problem. I think they’re very much out there … [T]heir overall objectives and goals are to influence policy and the best way to influence policy is to actually understand what policy makers are looking at and what are their challenges. And they interact with us on a fairly regular basis. They sit on committees that we’re involved with. They aren’t at a distance so they’re in the mix of decision making as such … [In different government programmes] somebody from HEU’s usually involved in some or other way … So they haven’t behaved like what I would call a stakeholder, you know, which has an external plan and is coming to discuss it with us, they’re very much in the mix … We don’t feel lobbied by them because we kind of see them as part of us.” (Government official, South Africa)

All senior HEU, South Africa staff indicated that within the South African situation, being placed in a university environment was preferable to being positioned in government or being an independent NGO. One respondent reflecting on recent Ministerial politics around HIV/AIDS suggested that if the Unit had been positioned in government “we would not have survived. The Unit would have fallen apart”. Foremost amongst the advantages of working at a university is therefore the protection afforded by academic freedom, especially when being critical of government. At the university, “there isn’t really any pressure to apply any particular ideology or politics, as long as we follow scientific principles”. This was particularly important under apartheid but remains true today.

Governance and financing factors

Considerable differences emerged between the policy institutes in how their agenda of work was developed, and the extent to which their governance and financing arrangements made them responsive to government. Some of the study institutes, such as CHeSS, Ghana, appeared largely dependent on shorter term projects funded by development partners for their main revenues. As such, they must be responsive, but the extent to which their work responds to government needs for policy analysis depends very much upon their individual donors. The institutes in Bangladesh, Vietnam and Uganda all have (or used to have) longer-term agreements regarding their financial arrangements, and hence might be thought to have sufficient space to develop a more autonomous programme of work. However, the physical location of the institutes in Uganda and Vietnam, combined with their reliance on the government for funding, means that in practice their agendas have been very strongly driven by government needs. HEI, Bangladesh had the fortunate combination of long-term funding and a degree of distance from government, in the sense that it was located outside of government, and its funding flowed via a third party. However, it was not able to take full advantage of this position. Only HEU, South Africa appears to have combined sufficient long-term financing with a position outside of government, to develop a truly autonomous agenda.

Governance structures, and in particular boards, can be critical in terms of helping to protect independence, while still promoting the policy relevance of work conducted. Three of the six case study institutes—CHeSS in Ghana, IHS in India and HEI in Bangladesh—had their own board. Often the boards facilitated relationships with government; for example, the HIS India board is currently composed of 13 members, and while no board member is formally appointed by government, government officials nonetheless hold a substantial number of places on the board and the IHS constitution allows for formal government representatives on the board. Similarly, for HEI, Bangladesh the board includes 15 members, two positions of which are statutorily held by ministry officials. HSPI, Vietnam does not have any formal board structure, but it does have a Scientific Committee (responsible for maintaining quality standards) and an Advisory Committee with responsibility for overall strategic direction. However, critics suggest that both of these committees are dominated by government officers, and that while HSPI, Vietnam has some independence, it still finds it difficult to criticize government policy.

Institute leadership

Personal links between institute members and policy makers can play a critical role in fostering trust and influence. Respondents in government often referred to the contribution of specific trusted individuals (even if the analytical work had come from a broader team):

“… The policy inroads that X can make are very considerable, really because of her long history and association with the ANC and her ability, and also because of the links that she has. I think this means that she is very readily listened to.” (External stakeholder, South Africa)

“This director has clear vision to influence policies. He is also very close to the Health Minister. He regularly participates in a meeting of all MOH departments every Friday. Frequently, the Minister officially and directly requests him to do some works for MOH. He also has many strategies to meet and talk to the Minister.” (Government official, Vietnam)

In Ghana, while CHeSS was too young to have already influenced policy, government officials were clearly pre-disposed to work with it because key CHeSS staff were well known to them:

“I got to know of CHeSS from Dr X … Recently, we said that with all the experience he has and the people he worked with, they can help us develop our new programme and given the background of the people I know in CHeSS, it is an institution that I personally can work with in the sense that they understand our needs better than I do.” (Government official, Ghana)

Discussion and conclusion

Study limitations

The greatest weakness of the study is that because of the case study approach adopted, detailed information is only available for six HPAIs, and given the great diversity of HPAIs it is difficult to draw generalizable conclusions. Further, the study was dependent on organizations willing to be studied, and this may have led to a bias towards the inclusion of more productive and better organized institutes, though it is notable that two of the study institutes were facing severe difficulties. While a relatively limited number of interviews were done in each case study, we believe that they reached a diverse set of stakeholders. Finally, the case study protocol did not include an objective analysis of the impact that the HPAIs have had on policy, nor was it feasible to make comparisons with similar situations where HPAIs do not exist. Accordingly, while our study casts light on what factors contribute to the effectiveness (in terms of policy influence) and sustainability of HPAIs, it does not draw firm conclusions about how effective they are, compared with other mechanisms.

While the study suffered from the problems outlined above, it also had a number of strengths, specifically it is the first cross-country study that has aimed to draw explicit comparisons between different types of policy analysis institutes in different low- and middle-income settings. Further, while the small number of case studies means that we cannot draw generalizable conclusions, the case study approach has allowed us to investigate the complex linkages between context, institute organization and financing, and policy influence.

Key findings and conclusions

The literature in this field stresses the need for strong in-house government capacity as well as strong external policy analysis capacity (Yaron and Shaxson 2008). As demonstrated by the case studies, particularly those in Vietnam and South Africa, under the right conditions HPAIs can play a positive role in promoting evidence-informed decision making in government. Further, the case studies provide insights as to which factors in terms of the context and organization of the HPAI enable it to play an effective role.

In the case studies, motivation and capacity within government to process and apply policy advice developed by HPAIs was found to be key to the ultimate success of the institute. Further, a strong demand from government for policy advice can potentially translate into a stable and secure source of funding for the institute, although of the institutes studied this had only transpired in Vietnam.

The case study institutes were selected to reflect different types of organizational forms, and accordingly varied relationships with government. While it is generally held that an arms-length relationship between a think tank and government is most appropriate, there is clearly no single optimal, institutional distance between a HPAI and its target audience. The broader policy and political context, forms of funding, organizational and individual characteristics, and the nature of formal and informal relationships are some of the many factors that affect trust, and ultimately influence. In some low- and middle-income settings where democratic and participatory values are not fully developed, HPAIs appear to have prioritized the development of a trusted relationship with government over engagement with a broader network of actors interested in policy (including, for example, media and NGOs). While for a period this strategy can be effective, it leaves an institute vulnerable to political change, and in the longer term it is important for HPAIs to develop a broader set of relationships.

Similarly, the central importance of key individuals to the policy influence capabilities of institutes can be a double-edged sword: if one or two key people leave the institute then influence may wane. One of the strong conclusions emerging from the Vietnamese case study was the need for HSPI to review how to shift from a model of an individual policy champion to a collective institutional capacity to influence policy. This also requires the institutionalization and diversification of relationships with funders, policy makers and other policy actors.

The Vietnam case study also highlighted the need for HSPI, and institutes in a similar position, to strengthen those mechanisms (such as board and advisory committee structures) which can protect neutrality and independence. If there is a close financial or administrative relationship between government and the policy analysis institute, then it is critical to ensure that the institute has appropriate mechanisms in place to prevent conflicts of interest and to ensure independent analysis. An additional mechanism to help maintain scientific credibility and demonstrate strong technical quality is through publishing findings in peer-reviewed journals. However, given constraints on staff time, there are difficult trade-offs to be made between a focus on responsiveness to government policy versus producing journal publications. At least, however, there should be internal processes for capacity-development of staff and collegial review of institute outputs to boost the quality of research and research outputs.

While the literature suggests that think tanks can play an important role in fostering public engagement and bringing fresh new perspectives to policy, the institutions studied had undertaken proactive public engagement to only a very limited degree, and this kind of function was not clearly evident in the mission statements of the case study institutes. This would appear to be an area in which the institutions themselves, and their funders, need to experiment.

In conclusion, strengthening health systems requires investments in basic care infrastructure and health technologies, health human resources training and supply, and appropriate, equitable health financing approaches. However, key to the sustained success of such investments is the availability of organizationally sound, scientifically credible institutions with some measure of autonomy that can provide continuous technical support and guidance to government and other actors involved in policy development. Health policy analysis institutes have a role to play in this regard but remain vulnerable to funding and staff shortages as well as political challenges to their autonomy. Governments and donors should explore ways to strengthen the capacity and sustainability of such institutes.

Funding

This paper was funded by the Rockefeller Foundation, and the study was conducted under the auspices of the Alliance for Health Policy and Systems Research, World Health Organization.

Conflict of interest

None declared.