The terrain of health policy analysis in low and middle income countries: a review of published literature 1994-2007.

This article provides the first ever review of literature analysing the health policy processes of low and middle income countries (LMICs). Based on a systematic search of published literature using two leading international databases, the article maps the terrain of work published between 1994 and 2007, in terms of policy topics, lines of inquiry and geographical base, as well as critically evaluating its strengths and weaknesses. The overall objective of the review is to provide a platform for the further development of this field of work. From an initial set of several thousand articles, only 391 were identified as relevant to the focus of inquiry. Of these, 164 were selected for detailed review because they present empirical analyses of health policy change processes within LMIC settings. Examination of these articles clearly shows that LMIC health policy analysis is still in its infancy. There are only small numbers of such analyses, whilst the diversity of policy areas, topics and analytical issues that have been addressed across a large number of country settings results in a limited depth of coverage within this body of work. In addition, the majority of articles are largely descriptive in nature, limiting understanding of policy change processes within or across countries. Nonetheless, the broad features of experience that can be identified from these articles clearly confirm the importance of integrating concern for politics, process and power into the study of health policy. By generating understanding of the factors influencing the experience and results of policy change, such analysis can inform action to strengthen future policy development and implementation. This article, finally, outlines five key actions needed to strengthen the field of health policy analysis within LMICs, including capacity development and efforts to generate systematic and coherent bodies of work underpinned by both the intent to undertake rigorous analytical work and concern to support policy change.

KEY MESSAGES

Literature on health policy analysis in low and middle income countries (LMICs) clearly demonstrates that politics, process and power must be integrated into the study of health policies and the practice of health system development.

However, the body of published work on health policy processes in LMICs is small, diverse, fragmented and quite descriptive in nature; it is dominated by authors based in Northern organizations.

Deepening and extending health policy analysis work in LMICs will require greater levels of funding to support dedicated capacity development efforts, and efforts to generate systematic and coherent bodies of work that are underpinned by both the intent to undertake rigorous analytical work and concern to support policy change.

Introduction

In the early 1990s several analysts called for a new approach to health policy analysis in low and middle income countries (Walt 1994; Reich 1995; Barker 1996). They noted that, until then, the assessment of health policy had focused largely on technical content and design, neglecting the actors and processes involved in developing and implementing policies, and taking little account of the contexts within which related decisions were made. They argued that this was shortsighted because it did not explain how and why certain policies succeeded and others failed, nor did it assist policy makers and managers to make strategic decisions about future policies and their implementation. Ultimately, all called for new paradigms of thinking to be applied to health policy analysis to enable understanding of the factors influencing the experiences and results of policy change. In particular, these scholars called for the use of analytical paradigms that integrate politics, process and power into the study of health policies.

Study of the processes through which ideas, knowledge, interests, power and institutions influence decision-making is primarily concerned with public policy and pays particular attention to how problems are defined, agendas are set, policy is formulated and re-formulated, implemented and evaluated (Parsons 1995). It is based on the understanding that policy is a product of, and constructed through, political and social processes. The roles of political institutions and public bureaucracies in policy-making are important aspects of this analysis, but it also acknowledges and considers the influence of non-state actors, including private sector and civil society organizations, as well as, in low and middle income countries (LMICs), international agencies. Work conducted within this field is applied and multi-disciplinary, with a broad social science base, is problem focused and seeks, ultimately, to strengthen policy-making (Parsons 1995). It is a fairly well-established field of inquiry in the US and Europe (Parsons 1995; Fischer 2003), and by the early 1990s had also been incorporated into analysis of LMIC public sector reform experiences (Nelson 1990; Grindle and Thomas 1991; Toye 1992; Haggard and Webb 1993).

So, nearly 15 years after the initial calls for such work, what analysis has been undertaken of health policy processes in LMICs, and by whom? These questions form the starting points of this first ever review of health policy analysis literature in such countries. The review was undertaken for the Consortium for Equitable Health Systems and EQUINET (the Regional Network on Equity in Health in Southern and Eastern Africa), both of which are supporting national and cross-national analysis of how to strengthen health policy and system development within LMICs in pursuit of health equity goals. It provides a basis for the development of policy analysis work in such settings by mapping the work already undertaken, drawing out some overarching insights into the processes of policy change and critically evaluating this work in terms of focus and methodological approaches. It does not provide a systematic synthesis of the findings of the articles examined concerning policy change.

Literature review approach

This narrative review of published, English-language health policy analysis literature focused on the period 1994 to July 2007 (final search undertaken August 2007). The 1994 start date was selected because this was the year in which Walt and Gilson published one of the first articles to call for greater application of this form of analysis in health policy work in LMICs. The article also presented a broad analytical framework which is quite widely used in the body of literature examined here. This review itself involved four steps.

First, the databases of PubMed and the International Bibliography of the Social Sciences were searched for the period of focus, using a range of relevant key words (Box 1). Some additional articles were also identified through hand searches of relevant journals.

Box 1 Search terms and inclusion criteria

Search terms:

health AND policy; health AND policy AND implementation; health AND ‘policy analysis’; health AND politics; health AND policy AND agenda setting; health AND policy AND power; health AND policy AND interests; health AND policy AND discourse; health advocacy; health AND construct*

AND Africa OR Asia OR Latin America OR Caribbean OR Pacific OR Middle East OR East Europe OR developing countries OR transitional countries

NOT US OR America OR UK OR Australia OR Canada etc.

Inclusion criteria, steps 2-3:

• Published in English

• Full article accessible

• Health policy focus

• Considers the processes of policy change and/or factors influencing these processes

• Considers experience in low and middle income, including transitional, countries

• Largely acceptable methodology

Inclusion criteria, steps 3-4:

• Primarily empirical study or clear empirical base

• Focuses entirely or mainly on policy change experience within or across country settings (analysis largely undertaken at meso- and/or micro-levels)

• Largely acceptable methodology

Second, listings and abstracts for the initial set of several thousand articles identified were checked and articles not meeting the inclusion criteria (Box 1) were excluded. Most exclusions were linked to geographical focus or a primary focus on policy impacts or design (for example, epidemiological and clinical analysis of the effectiveness of new public health interventions or analysis of the utilization impacts of user fee implementation, without considering how policy processes influenced those impacts). At this initial stage the authors worked independently.

Third, the remaining 391 articles (list available at http://www.crehs.lshtm.ac.uk) were reviewed jointly by the authors to provide a broad mapping of some of their features and to identify the final set of 164 articles to analyse in more detail.

At this stage articles adopting a largely conceptual, theoretical or methodological orientation were excluded, as we were particularly interested in considering the existing empirical work around policy processes within LMIC settings, In addition, although LMIC policy processes are influenced by macro-level forces, such as globalization, our primary concern was for the meso and micro levels of analysis, that is both ‘how policies come to be made, who puts them on the policy agenda and the structure of institutional arrangements in which policy is defined and eventually implemented’ and ‘the impact that particular people … have in designing policy and its final outcome’ (Hudson and Lowe 2004, p.8–9). So although interested in how the meso-level filters the influence of macro-level forces, including that of international agencies, over national policy processes, we were less interested in analyses focused exclusively on global processes and international agencies without considering country experience. Separating levels of analysis was not always easy. We decided, for example, to include an article examining how a research study implemented within an LMIC influenced international agency policy (Philpott et al. 2002), because it was an element within the more complex process of policy transfer between national and international levels that subsequently influenced policy development at national level (Lush et al. 2003). All exclusion decisions were made through discussion between the authors.

Fourth, the final set of 164 articles were analysed by the authors to provide a map of their policy and analytical focus, as well as methodological strengths and weaknesses.

The most important weakness of this review is that only English language literature was considered. In addition, although known to be available, relevant book chapters, doctoral theses and grey literature were excluded from analysis. On both grounds, therefore, the review cannot claim to be a fully comprehensive survey of all available LMIC health policy analysis work conducted between 1994 and 2007. However, it is known that some of the doctoral theses and unpublished material in the field are already represented within the published literature included here. Further, little additional, relevant material was identified through communication with a number of leading analysts—except, for example, books by Grindle (2002) and Nelson (2004). Perhaps most importantly, the aim of the review is to map the terrain of policy analysis work that has been conducted rather than systematically to review all evidence concerning specific policies or aspects of policy change. The absence of some articles from this map is unlikely to undermine the overall analysis, although the focus on English literature does affect assessment of where such work has been conducted, and, perhaps, of approaches to this work. It is recommended that additional reviews of other language literature be conducted.

We also note that the methodological judgements were particularly difficult to make. As discussed below, policy analysis work is very varied and few methodological details are given in many articles. We, nonetheless, sought to make general judgements (of adequacy in approach and/or authoritative presentation) to exclude the weakest articles, and, below, seek to explain what we see as the methodological strengths and weaknesses of this literature.

The broad terrain of health policy analysis in LMICs 1994–2007

The four main characteristics of this body of work are its small size, diversity, fragmentation and domination by authors based in Northern organizations.

On size, the total of 391 articles over nearly 13 years compares with, for example, a total of 612 PubMed hits for HIV/AIDS in Africa for the year 2006 only, or, also for 2006, 333/588 PubMed hits for health financing in Africa/Asia, respectively. An international review of policy implementation work in all fields and sectors, meanwhile, identified 2429 articles for the period 1985–2003 and 153 articles just for the year 2000 (Saetren 2005).

Diversity and fragmentation in the LMIC health policy analysis field is demonstrated by the articles’ subject focus, geographical basis and, partly, journal of publication.

Table 1 demonstrates the wide array of health policy areas addressed. For only six policy areas are there more than 20 articles, and all of these encompass a range of specific policy issues. Although most articles report empirical studies of policy change, only three groups of articles clearly focus on particular dimensions of the policy process: the process through which international policy ideas are transferred to national policy arenas (international-national policy transfer); advocating and lobbying for new policy ideas or options (advocacy); and efforts to use research to influence policy debates and decisions (research-policy). There are also conceptual and methodological pieces. Just over 50 of these articles offer insights of relevance to health policy analysis although not themselves addressing policy change. Most of these discuss how actors’ (such as health workers and patients) attitudes, strategies and knowledge are socially constructed and influence both their behaviour and policy implementation. They reveal the influence of a range of social, cultural and organizational norms over actors. Several articles examine policy discourse, the re-framing of this discourse by specific actors and the hidden operation of power and interests revealed in such discourse; others examine in detail the ways in which values, norms, power and interests are reflected in specific health policies. Finally, a few articles explore the range of socio-political influences shaping health system functioning in specific contexts.

Table 1

Classification of all articles by health policy area

Policy area	No. of articles
Health sector reforms	47
HIV/AIDS	43
Sexual rights and reproductive health	25
Donor coordination	21
International–national policy transfer	21
Advocacy	20
Research–policy	18
Decentralization	17
Role of private sector	14
Family planning and population/fertility	14
Human resources	12
Community participation	12
Primary health care	11
Health care financing general	11
Insurance	11
Methodsa	8
User fees	7
Tobacco	7
Safe motherhood	7
International organizations	6
Malaria treatment policies	6
Abortion	5
Pharmaceutical policies	5
Mental health	5
Environmental health	4
Health information systems	4
Miscellaneous policy focusb	30
Total	391

Notes:

aThe only set of articles not focused on a policy area.

bIncludes more than 20 policy topics, each with three or less articles.

A clear geographical focus can only be identified for 299 (77%) of the articles,1 of which 44% (133) are based on African, 32% (96) on Asian and Middle East, 18% (53) on Latin America and Caribbean, and 6% (17) on Central and Eastern European experience. This geographical mapping at least partly reflects the language focus of the database search. Importantly, 255 articles (65%) each examine experience within one country only; only 7% (28) present experience from multiple countries and 4% (16) focus on a single geographical region's experiences. For only 16 countries are there five or more articles, and for only five countries, 10 or more articles;2 and there is no synthesized analysis across such articles for any single country.

Table 2 shows that there is an unusually high (Saetren 2005) concentration of articles within the core health policy journals and particularly in Health Policy and Planning (73 articles: 34% core; 19% total) and Social Science and Medicine (41 articles: 19% core; 10% total).3 However, almost as many articles were published in a set of non-core journals that encompass a diverse range of disciplinary perspectives. Although two non-core journals account for more than 10 articles each (the Bulletin of the World Health Organization and the Journal of International Development), 92 journals have only 1–2 articles each. Among the non-core group, the largest concentrations of articles are in public health and tropical medicine and development studies journals.

Table 2

LMIC health policy analyses by type of publication, 1994–2007

	No. articles	No. journals	Average no. articles per journal
Core journals (health policy)	212	13	16.3
Non-core journals	179	102	1.8
Total	391	115	3.4
Core as % total	55%	12%
Non-core journals
Public health and tropical medicine	61	28	2.2
Development studies	46	22	2.1
Social science	38	26	1.5
Medical and nursing	21	18	1.2
Geographic studies	13	8	1.6

Finally, using organizational affiliation, two-thirds (265, 68%) of the articles are first-authored by people working in Northern organizations (some of whom are likely to be LMIC citizens), with only 30% (117) of the authors based in Southern organizations. Nine first authors have joint North-South organizational affiliation. Of the 33 people that have first-authored two or more articles, only seven came from Southern organizations; ten had an affiliation with the UK's London School of Hygiene and Tropical Medicine.

One hundred and sixty-four articles were selected for more detailed consideration as they present empirical analyses of LMIC health policy change. Until 1999 less than ten of these articles were published annually, with a total of only 24 in the 1994–98 period. More than 10 articles have been published each year since then, with particular peaks in years when at least one relevant journal special edition was published: 1999 (17 articles), 2000 (19 articles) and 2004 (21 articles).

Policies and issues examined

Table 3 summarizes the policy areas examined within these articles. Almost the same number examine policies addressing broad health system issues (77 + 1 from miscellaneous) as examine experience with specific health programmes or interventions (64 + 8 from miscellaneous); fewer examine experience of seeking to influence policy change (14). Although only the HIV/AIDS group has more than 20 articles, a total of 32 articles address sexual rights and reproductive health issues, including seven policy transfer and seven advocacy articles. A diverse range of policy topics is considered within every set of articles. Within the sexual rights and reproductive health group these topics include abortion, family planning and population/fertility control, comprehensive reproductive health policies, safe motherhood programmes, integration of sexually transmitted diseases and family planning services, cancers and domestic violence. Although the majority of HIV/AIDS articles broadly consider policy development in different contexts (eight considering South African experience), some focus on specific interventions to prevent or treat HIV; and the health care financing articles include coverage of user fees, insurance, and community financing.

Table 3

Empirical analyses of health policy change in LMICs, 1994–2007

		No. of articles by stage of policy process
Policy theme/area	Total articles	POL	POL & IMP	IMP	Equity relevancea
1) System-level policies
General health sector reforms	16	5	7	4	5 (4)
Health financing	15	1	5	9	8 (4)
Donor coordination	12	0	1	11	0
Decentralization	10	0	4	6	0
Community participation	6	0	1	5	6
Human resources	5	0	2	3	1
Pharmaceutical policies	4	3	1	0	4 (2)
Role of private sector	3	1	1	1	1 (1)
Health information systems	3	0	0	3	0
Primary health care	3	0	1	2	2
2) Health programmes/interventions
HIV/AIDS	23	5	4	14	9
SR&RH	18	7	4	7	15 (1)
Policy transfer (7 reproductive health; 6 public health)	15	6	5	3	2
Tobacco	5	1	2	2	0
Malaria treatment policies	3	0	3	0	1
3) Miscellaneous policy focus	9	3	2	4	1
4) Influencing policy
Research–policy	7	2	4	1	1 (1)
Advocacy	7	3	2	2	6 (1)
Total	164	37	49	78	62 (15)
% total		23	30	48	38 (9)

Notes: anumbers in brackets = no. of articles out of total with explicit equity focus.

POL = agenda setting and policy formulation; POL&IMP = policy development and implementation; IMP = policy implementation.

As with the overall set of articles, specific aspects of the policy process are explicitly considered in only three groups of articles (policy transfer, research–policy and advocacy). There is a little more coherence within these groups than within those grouped primarily by policy area. Most clearly, the policy transfer articles all consider the ways in which policies are transferred between international and national arenas, often explicitly using policy transfer theory or concepts. The few other more coherent bodies of work within this set of articles include: articles from different countries on the same topic and/or applying similar approaches, for example special journal editions on donor coordination [Health Policy and Planning, 14(3), 1999], tobacco policy (Tobacco Control, 13, 2004) and advocacy [Reproductive Health Matters, 8(16), 2000; 12(24), 2004]; several articles derived from the same study (e.g. articles analysing family planning programmes in eight countries, and health care financing policies in South Africa and Zambia); and several articles by the same author around a particular policy issue or theme (e.g. Reich on pharmaceutical policies; Shiffman on priority setting for safe motherhood).

However, less than 40% of the articles (59) demonstrate awareness of the wider field of policy analysis by referring to relevant concepts or theories. The articles can, thus, only be categorized crudely by policy stage rather than by specific theoretical theme. Table 3 shows that most articles primarily consider the implementation stage of the policy process (IMP), with fewest exclusively considering the policy development (POL) stage.

Most POL articles present narratives of a particular episode of agenda-setting or policy formulation. The policy transfer articles focus broadly on the role of international organizations in influencing national policy agendas, as well as how international policy agendas are themselves established (e.g. Reich 1995a; Shiffman et al. 2004; Walt et al. 2004). The majority of other POL articles examine the successes and/or failures of attempts to set national policy agendas and formulate a range of policies in relation to, for example, general health sector (e.g. Macrae et al. 1996) and pharmaceutical (e.g. Reich 1994) reform, sexual rights and reproductive health policy (e.g. abortion policy, Guedes 2000) and HIV/AIDS (e.g. Schneider 2002). Only three articles present analyses undertaken to inform policy change, two using Reich and Cooper's (1996) PolicyMaker approach (Aliyu 2002; Glassman et al. 1999). Slightly unusual articles include analysis of why there has been policy inaction on road traffic injuries in Central and Eastern Europe (McKee et al. 2000) and innovative analysis comparing priority-setting for breast and cervical cancer in Ghana (Reichenbach 2002). Those articles focused on system-level policies mostly consider actors based in the bureaucratic sphere (e.g. Ministers of Health, civil servants) whilst those considering sexual rights and reproductive health, HIV/AIDS and experiences of influencing policy include consideration of the wider range of actors located in the public sphere, such as civil society organizations and researchers.4 Only the pharmaceutical and tobacco policy articles consider commercial actors.

Articles categorized as POL&IMP investigate elements of agenda setting, policy formulation and policy implementation, whilst those categorized as IMP focus primarily on the practice and experience of policy implementation. For example, the policy transfer articles in these groups consider what factors of the policy process influence how internationally promoted policies are implemented nationally and locally (e.g. Richey 1999; Cliff et al. 2004; Schneider et al. 2006) and with what results (Lush et al. 2000). On balance, however, most of the POL&IMP articles have a stronger focus on the earlier stages of policy development than on implementation, and the majority present quite broad descriptions of national-level experience, only sometimes laying out a narrative of that experience.

Most IMP articles consider either general implementation experiences (37 articles) or the views and experience of implementing actors (34 articles). Although policy implementation essentially occurs at sub-national levels, 40% (31) of IMP articles exclusively consider experience at the international/national interface or national level. Eleven donor coordination articles, for example, consider national-level implementation experience around mechanisms and processes for managing donor assistance within country settings, including Sector Wide Approaches (e.g. the Health Policy and Planning special edition; Jeppson 2002). Other articles consider general implementation experience in a range of policy areas, including advocacy (e.g. Usdin et al. 2000). The articles focusing on actors, meanwhile, examine:

local and national level actors’ views about the extent of actor participation in health policy processes (e.g. Mosquera et al. 2001; Kapiri et al. 2003);

how the interests, values and beliefs of different actors shape the implementation of policies, including public health care providers and managers (e.g. Mayhew et al. 2000; McIntyre and Klugman 2003), private doctors (Hurtig et al. 2002), and beneficiaries (e.g. Macgregor 2005); and

how, in implementation, health staff resist and reformulate a range of policies (e.g. Duckett 2001; Tolhurst et al. 2004).

The remaining seven IMP articles specifically consider the influence of context features over implementation experiences and actors, including political culture (Atkinson et al. 2000), social and political context (e.g. Allen and Heald 2004), and political and bureaucratic organizational structures and processes (e.g. Zakus 1998; Steytler 2003).

Finally, we were particularly interested in considering what insights the articles offered around developing and implementing health policies seeking equity gains. As Table 3 shows, we judged that only just over one-third of the articles had equity-relevance. Only 15 articles explicitly considered the influence of a policy's equity focus over the policy process, but others were judged as being equity-relevant because they considered policies implying equity goals (e.g. sexual rights and reproductive health, community participation) or the policy-related experience of socially marginalized groups commonly prioritized in equity discussions (e.g. women, the rural poor).

The explicit assessments show the:

complexities of managing equity-oriented policy reform processes, given opposition and resistance from powerful actors whose interests or values are challenged by the policy (Reich 1994; Macrae et al. 1996; Armada et al. 2001; O’Rourke and Hindle 2001; Reichenbach 2002; Gilson et al. 2003);

ways in which policy design interacts with political forces in shaping implementation of equity-oriented financing policies (Gilson et al. 1995; Reich 1995b; Plaza et al. 2001; Tadros 2006);

exclusion of intended beneficiaries from policy-related decision-making processes even at local levels (Foley 2001; Gilson et al. 2001);

the opportunities and difficulties for activists and researchers seeking policy change from inside and outside the state bureaucracy (Weyland 1995; Klugman 2000; Pittman 2006).

The other articles confirm these experiences and also show how wider social processes act to exclude marginal groups from decision-making or consideration in decision-making (e.g. Zakus and Lysack 1998; Palmer et al. 1999; Hill and Ly 2004). Picking up on the issues raised in the IMP actor articles, they also emphasize that, by re-framing equity-relevant policies in implementation, health manager and worker resistance often generates unexpected consequences that include the development of antagonistic, even abusive, relationships with beneficiaries. Only one article presents a positive experience of strengthening these relationships in ways that support and sustain implementation towards equity goals (Tendler and Freedheim 1994).

Methodological and analytical rigour

The varied disciplinary perspectives of the articles, and the associated difference in accepted analytical practice, require sensitivity in assessing the methodological and analytical approaches adopted within them. Ethnographic or historical analyses, for example, are very different from economic analyses or from those based on best methodological practice in the public health field. Box 2 nonetheless provides a list of common criteria used in assessing qualitative studies. However, methodological judgements against these criteria are hard to make because the level of detail provided in articles is often fairly limited. Indeed, as around one-third of articles (56) provide very limited details on their data sources, or data collection and analysis methods, even describing the methods underlying these articles requires an act of judgement.

Box 2 Common criteria for assessing the methodology of qualitative studies

Clarity of research question and appropriateness of design to question

Systematic approach to data collection and analysis

Use of more than convenience sampling and efforts made to obtain data that might contradict/modify analysis

Adequately described context

Analytical approach persuasive, e.g. incorporates all observations; uses triangulation; describes how interpretation reached; explains and develops categories and concepts capable of explaining key processes and observations; includes search for disconfirming cases; includes use of relevant theoretical/conceptual material

Reflexivity shown in presentation, e.g. identifies and explains limits; situates in wider literature

(Drawing on Mason 1996; Mays and Pope 2000)

Although most articles indicate either the article's objectives or research questions/objectives, these are very often framed as descriptive or exploratory aims or endpoints. Even the slightly more tightly defined research questions remain quite broad, such as: how far do technical knowledge and actors shape financing policy change? Who influences the decision-making processes by which issues reach the international policy agenda and are formulated into guidelines? How far is policy informed by evidence and debate? Why are some contexts conducive to certain types of policy? Moreover, as Table 4 shows, an explicit study design is really discernible in only around 35% of the articles.

Table 4

LMIC health policy analysis articles 1994–2007, data sources and study design

		Data sourcesa		Study design (derived from article)
Article group	Total articles	Primary	Secondary	Case study	Ethnography	Historical/archival	Discourse analysis
POL	37	17	20	7	0	1	1
POL&IMP	49	23	25	12	0	3	3
IMP	78	59	18	15	11	3	1
Total	164	99	63	34	11	7	5
% total		60	38	21	7	4	3

Note: aNot possible to judge data sources for two articles.

Sixty per cent of the articles are based on primary data. Data were generated for 82 of these articles mostly through an apparently systematic and mixed set of largely qualitative data collection approaches, such as some combination of in-depth interviews, semi-structured interviews, focus group discussions, document review, media analysis and participant observation. Sampling approaches are often difficult to determine. Only three articles are based on data collected as part of an intervention evaluation; six articles were based primarily on personal experience. A few of the 38% of articles apparently using secondary data present reviews of existing literature, whilst most use documentary material sometimes combined with other data forms. The more authoritative articles clearly triangulate data and present richly contextualized arguments (e.g. Buse 1999; Schneider and Stein 2001; Austria 2004). Five discourse analyses are included as secondary analyses, as well as seven articles based on historical or archival analysis (including five tobacco policy articles).

Only a few articles present or use quantitative data substantively in their analysis, and only one is based on statistical analysis (Bor 2007, considering HIV/AIDS leadership). Analytical approaches used with qualitative data are generally difficult to determine. Many articles appear to apply an inductive approach in generating narratives of experience or identifying themes of experience from their data; and a few generate their own conceptual frameworks (e.g. Collins et al. 1999; Sauerborn et al. 1999; Gladwin et al. 2002). Few articles report deliberate attempts to validate preliminary judgements through, for example, presenting initial analyses for discussion with at least some informants/knowledgeable people (e.g. Amin et al. 2007); and only some articles demonstrate analyst reflexivity (e.g. Alonso and Brugha 2006). As noted, a minority of articles make substantive reference to relevant concepts and ideas. A number of those that adopt a case study design, meanwhile, have analytical weaknesses, such as limited contextualization of the experience they report or inadequate comparison and contrast of cases in analysis (see below). On the whole, therefore, against the criteria of Box 2 many of these articles present only weakly persuasive and authoritative descriptions or arguments.

Analytical approach

Based on a combination of the articles’ stated objectives/questions and the nature and argument of their findings and discussion sections, Table 5 groups the articles into four analytical types. In total, around 20% of articles each are categorized as either simply descriptive or aiming at explanation, with most categorized as either descriptive and analytical (30%), or analytical (27%). As the table shows, this categorization is based on the articles’ use, or not, of four analytical features (that were themselves derived from review of these articles). The two features most commonly adopted across the articles (although each is applied in less than one-third of the articles) are the use of conceptual ideas and frameworks in analysis, and comparison and contrast between several cases (Box 3). Table 6 shows the distribution of these four types of analytical approach across articles grouped by policy stage.

Table 5

Analytical approach of LMIC health policy analysis articles 1994–2007

		Analytical features
Analytical approach	Total no. of articles	Cases used	Concepts or framework applied	Cross- country analysis	Test propositions
Descriptive	38	1	3	1	0
Descriptive and analytical	50	13	9	3	0
Analytical	44	11	16	7	4
Explanatory	32	17	14	9	6
Total	164	42	42	20	10
% total		26	26	12	6

Table 6

LMIC health policy analysis articles 1994–2007, categorized by policy stage and analytical approach

		Analytical approach
Article group	Total	Descriptive	Descriptive and analytical	Analytical	Explanatory
POL	37	7	13	8	9
POL&IMP	49	14	14	12	9
IMP	78	17	23	24	14
Total	164	38	50	44	32
% total		23	30	27	20

Box 3 The use of frameworks and cases in analysis

Descriptive articles either present a narrative of an episode of policy change or simply present their findings around an aspect of such experience, with little attempt even to categorize their findings. A fairly large proportion of the articles examining policy development and implementation experiences (POL&IMP) fall into this category. Some are rich and interesting analyses drawing on primary data or archival analysis (e.g. Amin et al. 2007 on Kenyan malaria drug change; Hoodfar and Assadpour 2000 on population policies in Iran; Mackenzie et al. 2004 on tobacco ingredients’ disclosure in Thailand; Stein et al. 2006 on providers’ responses to anti-retroviral roll-out in South Africa). However, others tell quite thin policy stories that offer limited detail, or cover too many issues or experiences, each in too little depth, without reference to the wider empirical or theoretical context. These articles barely apply any of the identified analytical features and, partly as a result, make little effort to consider the wider relevance of their findings.

Descriptive and analytical articles focus on general experiences of policy change across all policy areas except tobacco, with around one-third of both the POL (agenda setting and policy development) and POL&IMP articles falling into this study type. These articles range from quite thin presentations of experience to much richer stories of particular experiences structured around, or accompanied by, a discussion of the themes of that experience, including reference to relevant, broader literature (e.g. Harrison et al. 2000; Kwon and Reich 2005). Pavignani and Durao (1999) present, unusually, a longitudinal analysis of experience (on donor coordination). The more analytical of these articles include several that are ethnographic studies examining policy and organizational processes (e.g. Hurtig et al. 2002; Heald 2005) or beneficiary experiences (e.g. Macgregor 2005). Indeed, Harper (2005) specifically argues that ethnographic work offers important insights into the mistaken assumptions embedded in policy and allows exploration of unintended policy consequences. Around one-fifth of the articles apply cases or conceptual frameworks in analysis but only three report cross-country analyses.

Analytical articles all have a more specific focus than other groups of articles. A larger number than in other groups use conceptual frameworks to guide analysis, whilst some use case-based and cross-country analysis. The majority of IMP articles (24, 31%), focused primarily on implementation experiences, are of this type, with clusters focusing on financing and HIV/AIDS policies, and donor coordination. They examine particular aspects of policy change [e.g. veto points in implementation, derived from Pressman and Wildavsky (Atkinson 1997); the adoption of a new health information management system, using innovation diffusion theory (Gladwin et al. 2003); priority-setting for reproductive cancers (Reichenbach 2003); the operation and management of policy advisory committees, using Eden's stakeholder management framework (Thomas and Gilson 2004)], or the forces influencing actor behaviour [e.g. a cluster examining street-level bureaucrats, as well as one on doctors and their communication practices (Datye et al. 2006)]. Four articles test propositions in some way drawing on: street-level bureaucracy theory (Walker and Gilson 2004); cultural diffusion theory (Luke and Watkins 2002); an author-constructed model of health policy, implementation and management in small-island developing states (McNaught 2003); and decentralization experience (Mogenson and Ngulube 2001). Based on ethnographic work, three others also examine provider practices and understandings (Aitken 1994; Seidel 2000; Penn-Kekana et al. 2004), whilst Hill (2000) draws on participant observation and organizational analysis in examining Cambodian health planning processes. Finally, all five of the discourse analysis articles are categorized with this group and consider the role of discourse and its re-shaping within policy processes, with implications for how policy is understood (Hunter 1996; Atkinson 1997; Richey 1999; Hill 2002; Hill and Ly 2004).

By definition, the explanatory articles seek to explain policy change, with particular consideration of HIV/AIDS and sexual rights and reproductive health policy experiences. Although the group includes a significant proportion (24%) of the POL (including policy transfer) articles, it contains a higher absolute number of IMP articles. Some articles seek to explain the political feasibility of reform (Reich 1995), or priority setting among policies (Shiffman et al. 2004). Others examine how context influences implementation (e.g. Atkinson et al. 2000), what explains the gap between intended and experienced forms of decentralization (Arajuo 1997), and what factors drive actor behaviour or their policy influence (e.g. Klugman 2000; Kaler and Watkins 2001; Putzel 2004; Robins 2004). A few articles specifically seek to explain policy outcomes, including: differences between countries in relation to HIV/AIDS experiences or responses (Allen and Heald 2004; Gauri and Lieberman 2006), safe motherhood outcomes (Shiffman et al. 2006), family planning programme outcomes (Lee et al. 1998) or the equity impacts of the Bamako Initiative (Gilson et al. 2001). Others, finally, seek to explain research impact on policy (Haaga and Maaru 1996; Trostle et al. 1999; Phillpott et al. 2002) or positive health worker performance (Tendler and Freedheim 1994).

Compared with other groups, higher proportions of articles in the explanatory group applied all four identified analytical features. Drawn from theory, the propositions more or less formally tested include influences over agenda setting for child and adult health (Reich 1995a), influences over priority setting for safe motherhood in Honduras (Shiffman et al. 2002), contextual influences over health reform, examined over several distinct historical periods in Chilean history (de la Jara and Bossert 1995), whether different system structures are conducive to certain types of HIV/AIDS policy, examined across two countries (Parkhurst and Lush 2004), and how organizational culture explains how street-level bureaucrats adapt to new expectations of them (Crook and Ayee 2006). Explanatory authority is sometimes assisted by careful study design. For example, Lush et al. (2000) present analysis around propositions about policy divergence from four pairs of countries, in which countries were deliberately paired to allow matching around social, economic and cultural attributes but variation in policy and programme histories. Similarly, Atkinson et al. (2005) compare health care experience in two purposively selected study sites that were both rural but differed in terms of the preventive and promotive health care activities of focus in their analysis. However, authoritative explanation is also derived from rich and nuanced analysis of specific experience, for example the analyses of Butler (2005) and Robins (2004) around South African HIV/AIDS policy.

A critical evaluation of LMIC health policy analyses, 1994–2007

The small size, diversity and fragmentation of this work, and the quite descriptive approach of many articles, makes systematic knowledge synthesis across the articles a challenging task that is not attempted here.

Yet even the quite broad features of experience that can be noted from these articles clearly add weight to the initial calls for this sort of analysis. First, the articles show that policy and policy change is always contested. Contestation around some policies occurs within the public arena (e.g. abortion), and in some countries, the failure to take policy action may generate fierce public opposition (HIV/AIDS). In other instances, opposition even within the more closed bureaucratic arena can prevent policies from being implemented, as even apparently uncontroversial policies are resisted by implementing actors. Policy actors are not just those officially tasked with policy development; they also include all those with concern for particular policy issues or likely to be affected by policy developments, including commercial interests, civil society organizations and beneficiaries. Second, the articles show that policy decisions (or non-decisions) often result in unintended and unwanted consequences. An ethnographic examination of health worker responses to training interventions, thus, shows how different understandings of the broader health system environment undermine the extent to which training interventions work to develop new skills and improve service delivery (Aitken 1994). Third, and relatedly, the articles show that policy is socially constructed, wrapped up in and influenced by the meanings different actors attribute to policy content or goals (Fischer 2003). As a result, and fourth, bringing about effective policy change does not simply require good technical design or using evidence to generate policy. These articles suggest that it must always involve clear attention to the processes by which change is brought about, including concern for the values and interests of the actors with potential to block or subvert policy development and implementation, and for the discourses surrounding policy change processes.

However, health policy analysis in LMICs clearly remains in its infancy. The relatively coherent body of work on agenda setting and policy formulation is quite small in size, whilst the slightly larger body of implementation work is disparate and scattered, perhaps more strongly rooted in micro- than meso-level analysis. There are also five other main weaknesses in this body of work.

The first is its analytical weakness. The depth of data presented, and perhaps even collected, is often limited, as shown by the weak contextualization of experience in many articles. Cross-sectional descriptive analyses, for example, commonly exclude any assessment of the always important historical influences over experience. At the same time, the articles often do not provide clarity about their analytical approaches, provide little commentary on how they add to the existing empirical evidence base or offer reflections on the interpretations made (such as their basis, or alternative possibilities). In some instances there appear to be failings against basic criteria of rigour, and some articles simply do not persuade the reader of their validity or authority.

Most articles, secondly, lack an explicit explanatory focus. The main question asked is often ‘what happened’ and not ‘what explains what happened’. Only a few articles, thus, focus analysis on explaining why a policy succeeded or failed (for example, but not only, with respect to equity), with even fewer specifically considering successful experiences. They provide, in the main part, therefore, only a weak foundation for informing future policy action. In part this weakness reflects most articles’ focus on experience around one policy in one country at one time, rather than comparing and contrasting experience across countries or over time, between health policies or across sectors within a country, or between implementing units and people, for example. The articles that do present such comparisons, particularly those categorized as explanatory articles, clearly illuminate experience and, sometimes, offer insights of wider relevance. Although many articles present case studies, such comparative case analysis would require more explicit use of formal case study analysis approaches than is common. For example, appropriate case selection criteria must be established, each case must be adequately contextualized, and efforts must be made to deliberately identify and explain unusual experiences and findings (Yin 1994).

Third, little of the existing body of work draws on policy analysis theory to direct and guide analysis, deepen understanding, enable explanation and support generalization. The theories of Kingdon (agenda setting) and Lipsky (street-level bureaucracy) are among those referred to in at least some articles and network analysis is beginning to emerge within some of the policy transfer articles. However, the vast majority of implementation theory available to policy analysts is largely ignored in LMIC literature. From review of high-income country implementation analyses, Hill and Hupe (2002), for example, identified seven independent influences over implementation: policy characteristics; policy formation; layers in the policy transfer process (or vertical public administration); the overall characteristics of implementation agencies/organizations; the behaviour of front-line staff; the impact of responses from those affected by policy; and wider macro-environmental factors. Within the LMIC literature there is some consideration of only three of these topics, of which the slightly larger pool of work considers the behaviour of front-line staff.

A particularly surprising thinness in the overall LMIC body of work, moreover, relates to power, a central element of policy analysis theory and policy change experience. Although broadly discussed in a range of articles, very few present explicit or formal assessments of the practice of power in policy change (e.g. Zakus 1998; Walt et al. 1999; Seidel 2000). From an implementation perspective, there is also very little explicit consideration of the institutions, understood as the rules, laws, norms and customs that clearly shape actor behaviour, such as organizational culture, networks or the ‘assumptive world’ (Parsons 1995; Hudson and Lowe 2004). Managing policy actors, and so policy change, clearly requires better understanding of such influences.

Fourth, the vast majority of analyses can be categorized as analyses of policy rather than for policy. In other words, although most seek to assist future policy-making, only a handful were undertaken as a direct input into policy-making or as part of an implementation evaluation.

Fifth, although a range of articles demonstrate that policy is socially constructed, only a few apply forms of analysis (such as discourse analysis) that consider the role of language, rhetorical argument and stories in framing policy debate. On both grounds, therefore, LMIC health policy researchers could learn from the field of deliberative policy analysis (Hajer and Wagenaar 2003), which seeks to ‘construct an interpretation of present political and social reality that serves not only intellectual goals of explaining or comprehending that reality, but also the practical goal of enabling constructive action to move the community from a flawed present toward an improved future’ (Jennings 1987, p. 127, in Fischer 2003, p. 223). Deliberative policy analysis calls for the more active engagement of analysts in the policy process, rather than examining it from the outside. By providing participants (citizens, analysts, decision-makers) with access to and explanation of relevant data, analysts could, for example, contribute to public policy discussion and so to public learning and political empowerment. In calling for ‘a social science that matters’, Flyvbjerg (2001), for example, describes how his interpretive study of the relationship between rationality and power within an urban planning process ultimately led to the establishment of processes for democratic dialogue and decision-making in urban planning.

Conclusions

Despite its weaknesses, the existing body of health policy analysis work in LMICs demonstrates its contributions to understanding the nature of policy and the often unexpected challenges to bringing about policy change. It confirms that politics, process and power must be integrated into the study of health policies and the practice of health system development in such settings.

Deepening and extending this body of health policy analysis work will, however, require a large-scale effort involving more work undertaken by more people, supported by greater levels of funding. But this effort must go beyond a simple expansion of the existing work. It must incorporate:

dedicated efforts to build policy analysis capacity within LMICs, including awareness of relevant theory and empirical work, and analytical skills;

rigorous synthesis of the existing more coherent bodies of work (e.g. work on agenda setting or policy transfer);

programmes that generate new systematic and coherent bodies of work driven both by a concern to support future policy change and by the intent to undertake rigorous and analytical work, including:

more work on implementation, and specifically, the challenges of implementing equity-oriented policies, as well as more examination of successful policy change experiences;

multi-country studies, whether framed in relation to a specific health policy topic or experience, or as a policy analysis issue investigated through health policies or by comparison of health and other sectoral policies;

sets of studies (either within or across countries) in which the conclusions from one are fed into the next, so deepening understanding;

more rigorous use of case study design;

rich historical analyses of specific country experiences and rich, micro-level analyses of decision-making;

greater reflexivity among analysts;

more studies that deliberately seek to explain policy change, unpacking causality through a combination of careful study design and appropriate use of relevant theory;

more deliberative engagement by health policy analysts in processes of policy and health system change; for example, working with policy-makers, advocacy groups and/or civil society organizations through structured processes of dialogue or action research, to enable public learning and political empowerment.