Dynamics of evidence-informed health policy making in Pakistan.

Incorporating evidence is fundamental to maintaining the general acceptance and efficiency in public policies. In Pakistan, different actors-local and global-strive to facilitate the development of evidence-informed health policies. Effective involvement however, requires knowledge of the country-context, i.e. knowing the intricacies of how policies are formulated in Pakistan. Obtaining this knowledge is one of the key steps to making interventions impactful. We carried out a qualitative study to explore the environment of evidence-informed health policy in Pakistan. The study involved 89 participants and comprised three phases including: (1) literature review followed by a consultative meeting with key informants to explore the broad contours of policy formulation, (2) in-depth interviews with participants belonging to various levels of health system to discuss these contours and (3) a roundtable with experts to share and solidify the findings. Policy development is a slow, non-linear process with variable room for incorporation of evidence. Political actors dominate decisions that impact all aspects of policy, i.e. context, process and content. Research contributions are mostly influenced by the priorities of donor agencies-the usual proponents and sponsors of the generation of evidence. Since the devolution of health system in 2012, Pakistan's provinces continue to follow the same processes as before 2012, with little capacity to generate evidence and incorporate it into health policy. This study highlights the non-systematic, nearly ad hoc way of developing health policy in the country, overly dominated by political actors. Health advocates need to understand the policy process and the actors involved if they are to identify points of impact where their interaction with policy brings the maximum leverage. Moreover, an environment is needed where generation of data gains the importance it deserves and where capacities are enhanced for communicating and understanding evidence, as well as its incorporation into policy.

Key Messages

Policy development is dominated by political actors impacting all aspects of policy i.e. context, process and content.

Public health advocates need to understand these dynamics for their effective engagement with policy.

Environment is needed where evidence generation and its incorporation into policy gains due importance and relevant capacities are available.

Introduction

Policymaking is a process that defines and pursues the right course of action in a given context, at a particular time, for a certain group of people, with a particular allocation of resources (Greenhalgh and Russell 2006). Using a hypothetical example, explains how health policy and its process can set trajectories of health outcomes: If health economists are consulted for HIV policy in a country, prevention will be emphasized because prevention tends to be cost-effective. On the other hand, if only patients or pharmaceutical companies are consulted, the policy will be curative. The direction an HIV epidemic takes in that country will depend on who participated and helped shape the HIV policy (Buse ).

Health policymaking is a complex phenomenon (Bosch-Capblanch ) in which the incorporation of evidence is an intricate step (Gilson and Raphaely 2008; Greenhalgh and Sietse 2011). The notion that policy can or should be based entirely on evidence has been debated widely. Researchers have argued that the very expression ‘evidence-based policymaking’ is flawed as it suggests that there are technical solutions to what are essentially political problems (Greenhalgh and Russell 2006). Policy tackles ‘how or what a society should look like’—questions that evidence alone cannot answer. In reality, policy involves multiple social norms and there can be different evidence bases relevant to each of these norms (Parkhurst 2016).

There is agreement among many in the research and policymaking arenas that policy can be ‘evidence-informed’ rather than ‘evidence-based’, and that the use of evidence in health policymaking does not simply imply the incorporation of knowledge emerging from research but includes information from various sources (Smith and Joyce 2012; Hawkins and Parkhurst 2015). ‘Policy’ is a broad concept and includes laws, regulations, judicial decrees, as well as agency guidelines and budget priorities (Brownson ). The interaction of researchers with all policymaking processes therefore is vital for incorporating evidence into policy (El-Jardali ). Public health advocates can contribute better to policymaking if they understand the processes of policy development and implementation, as well as the many factors that influence policy formulation, including individuals, institutions, interest groups, ideas, power and politics (Walt and Gilson 1994; Buse and Dickinson 2007).

Pakistan is a lower middle-income country (World Bank 2016). Similar to other countries in this category, its policy context differs from that of high-income countries (Walt ). Lower middle-income countries often share characteristics such as weaker regulations and regulatory capacity, lack of purchasing power, patronage in the political system and more reliance on donor funds than high-income countries. As in many other developing countries, Pakistan’s public budget for health is expended on hospitals to treat conditions of the urban elite, while the poor and vulnerable continue to die from diseases that can be prevented for a few cents (Filmer ). The state is both provider as well as purchaser of health services and enters into partnership with the private sector while being responsible for its regulation at the same time (Nishtar ).

Policy development in Pakistan has not been a focus of research. Our review of the literature revealed a dearth of information overall. A few studies exist on policy context (Khan 1996; Collins et al. 2002; Khan and Heuvel 2007) or policy content (Siddiqi ). The lack of research-based information is not unique to Pakistan. Studies from elsewhere have also concluded that in general, health policy research has focused on better health policies (Innvaer ; Gilson and Raphaely 2008) more than on how health policies are formulated and systems evolve. A recent report describes that out of 239 research grants approved by a funding agency in the past 15 years, only 11% concerned health policy analysis, suggesting that such analyses in lower middle-income countries are still in their infancy and need more focus (Ghaffar ).

The present study is one step towards addressing the gaps highlighted above. It aims at documenting the environment and mechanisms of evidence-informed public health policy at the national and provincial levels in Pakistan. Other sectors (e.g. the Ministry of Finance) that also have a bearing on health policy are not included in this study, as we have focused on the policy process in the health sector only, at this stage. Based on a time-tested model (Walt and Gilson 1994), the specific objectives of the study include exploring the context, actors, process and content of health policy at the national and provincial levels, in order to draw recommendations for advocates of public health in Pakistan and countries having a similar context.

Methods

The policy process has been referred to as a black box (Buse ) because of its ambiguity. Frameworks have been developed to address the complexity while carrying out policy analysis (Lavis ; Shiffman and Smith 2007; Smith and Joyce 2012). We found the ‘health policy triangle’ as most applicable to our study because of its relative simplicity. In this triangle, actors (individuals, groups and organizations) have a central place, while three elements: context, process and content form the angles (Walt and Gilson 1994). Actors can be individuals or organizations; however, individuals cannot be separated from the organizations in which they work. Context means systemic factors arising from the political, economic and social environment—nationally and internationally—that may have an effect on health policy. Process is the way the policies are initiated, negotiated, formulated, communicated, implemented and evaluated. Content refers to the specific policy elements that are likely to be effective.

Using the tenets of the health policy triangle as a guide, our qualitative discussions with key informants usually started with policy content. This initial question provided a broad understanding of the respondent’s conceptions about policy while also providing a hook around which we generated further discussion about the policymaking actors, context and process. Where appropriate, we used examples of landmark decisions including policy guidelines (such as the inclusion of a zinc supplement to the treatment of diarrhea), resource allocations (development and recurrent budgets), institution of new programmes [e.g. the Lady Health Worker Programme (LHWP)], and constitutional amendments (such as the devolution of healthcare system under the 18th constitutional amendment) to understand the interplay of the four dimensions outlined in our chosen framework.

The study was conducted in an iterative manner and comprised of three phases. Phase 1 included a literature review followed by a consultative meeting with 28 key informants to explore the broad contours of health policy formulation. The key informants for this phase were purposively selected by the research team and belonged to the categories of researchers, technocrats and bureaucrats from both national and provincial level. Phase 2 consisted of in-depth interviews with 36 participants selected through purposive as well as snowball sampling (Patton 2002; Suri 2011) belonging to national and provincial levels of health policy makers. Phase 3 included a final roundtable with 25 purposively selected public policy experts, including some from the first consultative meeting, to share and solidify the findings. Focus groups were conducted where a holistic exploration was required while individual interviews were employed to allow independent context and perceptions to arise and to avoid inhibited expression that could take place in group settings (Miles and Huberman 1994).

The study team was comprised of researchers having formal training in qualitative methodology and policy research, led by a team member with significant experience in policy development and implementation. To ensure uninhibited expression of opinions, only that member of the research team conducted interviews and discussions, which did not have a role in the current policy processes, either at national or provincial level. Following interviews or discussions all team members met to discuss the proceedings, examined field memos and brainstormed the future course of action (e.g. any change to the guiding questions) for the study.

The study was conducted across Pakistan with key informants representing the federation, all four provinces and the Azad Jammu and Kashmir region (AJK). In addition to officials, we also involved researchers having the experience of working with policy. We included participants from all ‘institutions’ to ensure broad representation and did not include for consideration ‘ideas’ that these participants shared in the first meeting to prevent our findings from being skewed towards a particular group or way of thinking. During the interviews, we also inquired about other potential respondents who could provide useful information on the subject. The criterion we used was that the participant should have played some role in the development and/or implementation of health policy while working inside the health sector.

Those agreeing were visited or interviewed by phone. One member of the study team moderated and recorded these interviews. Verbal consent was obtained, objectives explained and confidentiality discussed before the interview. The interviewer took detailed, verbatim notes of the discussions. Most of these interviews were conducted using a mix of Urdu and English languages and the duration ranged from 30 min to approximately two hours. No incentives or payments were offered to the respondents. Ethical clearance was obtained from the Institutional Review Board (IRB) of the Health Services Academy Islamabad, Pakistan.

Using the four dimensions of our framework, we carried out thematic content analysis on the data (Patton 2002). Two members of the team agreed on initial codes, independently analysed a number of transcripts, and met again to discuss and finalize a unified code sheet. Employing this method, they manually coded all the transcripts to identify the significant statements across individual interviews. Data reduction was achieved by displaying the sub-themes in the form of a matrix in which each category of respondents was placed in rows while sub-themes were placed in columns. This allowed visualizing the recurrent sub-themes across each respondent category. Through this systematic comparison (Patton 2002), the convergent sub-themes were grouped as themes emerging from discussions. While greater weight was given to recurring themes, attention was also paid to the divergent themes—points that were not shared by a majority of respondents but which were found to be significant (Miles and Huberman 1994; Patton 2002). The study was completed over a year and a half from January 2014 till July 2015.

Results

A total of 89 policymakers and health experts participated in the three phases of this study (Table 1).

Table 1

Profile of participants (n = 89)

Category	Number (%)	Profile
Federal	28 (31)	Former and current health secretary and director general health, Managers of various health programmes, Members of Planning Commission, Officials from federal bureau of statistics
Provincial/	36 (41)	Former provincial health secretary, Former and current provincial director general health, Managers of provincial programmes, Former and current district health officers
Regional
Academia/	14 (16)	Researchers and administrators from public and private research organizations
Research
Civil society, including development partners	11 (12)	Members from donor and advocacy organizations, professional bodies and associations

The themes emerging from their interviews and discussions are described below.

Content

At the outset, the participants discussed the notions they had of policy, which over the course of discussion started blending with their views about the other elements of the policy triangle, usually in an unprompted manner.

Various notions of policy

Different participants had different positions, perspectives, understanding of and appreciation for the word ’policy’. For example, the majority of participants mentioned project concept (or PC-I documents) as the ‘policy’. These projects are developed to address a health issue where the lessons learned during their implementation and evaluations are expected to further feed into the policy.Many participants also referred to the development or revision of technical guidelines of various Primary Health Care (PHC) programmes as the policy. The adoption of the Essential Health Services Package into PHC in the province of Sindh and guidelines on Severe Acute Malnutrition in Punjab were mentioned.Many also mentioned the budgetary allocations as the ultimate reflection of policy. They referred to the ‘recurrent budget’, which is the amount allocated on a regular basis and which includes salaries, supplies and the maintenance of infrastructure. The ‘development budget’ on the other hand, is spent on building new programmes. Through the development budget, a programme should prove its effectiveness and become absorbed into the mainstream.‘The person behind’ is the policy

‘For ultimate policy and its implementation, we have submitted the concept of Health Sector Reform Unit through a PC-1 which when approved, will pave the way for optimal working of the health system including the policy’—Health Programme Manager, Balochistan

‘We have finalized the Essential Health Services Package as part of the provincial health policy, however, some of the essential medicines are not available for which we are advocating with various donors’—Health Programme Manager, Sindh

‘The recurrent budget allocated every year is looked after by custodians of health system including the Secretary at the province and Executive District Health Officer at the district. The preventive programmes which are a reflection of an evolving policy, should produce results for incorporation of their interventions into recurrent budget’—Programme Managers, federal and provincial level

Some participants expressed their views in a way that content and actors of the policy converged as if they are two sides of the same coin. To them, even the budgetary allocations may not convey the policy intent, as the budgets are usually revised multiple times and the revised budgets may look different altogether from what was presented and approved in the national or provincial assembly.Some also mentioned how persons and their particular context become part of policy. To them, public policy is a statement of intent from the government; it should comprise a broad set of objectives and should not become an action plan. Such policy decisions are reached when evidence is synthesized and presented to the policymakers along with various policy options—something that seldom occurs. They mentioned a couple of policy examples from the period of President Musharraf who imposed military rule in the country and enforced policies in that context.

‘These revisions are made by individuals and parties based on their assumptions and interests. So, in real world, it is not the policy but the man behind its development or implementation who is more important’.—Health administrator (retired), federal

‘The health policy document of 2001–2002 was not a policy but an action plan written in a few days for a person who wanted to have a new policy that could be presented as another landmark of that government. There was no evidence, discourse or process of weighing options behind that hollow document’—Health administrator, federal

Context

Policy development—not a true priority

The participants discussed the social and political environment in which everything happens, and drew attention to the importance or lack of it, accorded to health, health policy and the incorporation of evidence. In a country where security and livelihood are much bigger issues and where only 0.7% of GDP is allocated to health, having clarity about health policy and giving importance to the incorporation of evidence seemed a utopian idea to them.Many participants expressed that resources for the generation of evidence are limited due to weaker policy emphasis on the generation of evidence. Participants went as far as saying even the technical specialists show ineptitude towards generating, interpreting and absorbing data into policy, a lacuna that further contributes to a lack of evidence and its incorporation into policy.

‘In the current circumstances, health is not a priority for the administrative or the political level. It may be fair to say that health and health policy at times are not a priority even for the technical people who deal with health’—Health administrator, federal

‘Dealing with statistics is considered the dirtiest job—something because of which professionals dealing with this are also considered inferior. In the current circumstances, health is not a priority for the administrative or the political level. It may be fair to say that evidence at times is not a priority even for the technical people who deal with health’—Health administrator (retired), federal

Capacity issues

Not all policymakers use data or have interest or capacity to oversee or facilitate the incorporation of evidence into the policy process. Of equal importance, researchers demonstrate limited capacity to carry out credible research and then communicate it in a language that resonates with the decision-maker.The lack of evaluation capacity was voiced more from smaller provinces like Balochistan, where inadequate capacity is one of the reasons of not having enough bottom-up policy processes. The Health Management Information System (HMIS) data are not reliable, many of the districts do not report on HMIS, and those who report usually submit incomplete data.

‘The policymakers, especially politicians do not have time and capacity to understand research jargon and complex analyses. They need data in a language that make sense to them. Ironically, this capacity is commonly found lacking among the research fraternity’—Health administrator, federal

‘District managers do not consider data collection and management as their job. There are very few research and evaluation persons, hence minimal outputs in terms of data. Most of the district positions are hired based on safarish (favoritism)’—Programme Manager, Balochistan

Agenda setting

Party manifestos are an indicator of how political priorities and agendas shape policies. For the 2013 general elections, the Pakistan Peoples’ Party and Pakistan Tehreek-e-Insaf promised an increase in overall health spending, but this was not a consideration for the Pakistan Muslim League-Nawaz (PML-N), the party that eventually won the elections. Similar factors are identifiable in the party’s practices.In speaking about who shapes the opinion of political parties and the public representatives, participants shared the views that public opinion, media coverage and on-going debates certainly have an effect on political mindset. Issues consistently reported and highlighted by print and electronic media gain the attention of political players.

‘In line with their manifesto, the ruling party allocated only 0.5% of GDP to health in their proposed budget of 2013–2014. However, since PMLN manifesto introduced a new National Health Insurance Scheme for the whole of Pakistan, a bill was passed in May 2015 on this scheme to assuage the vulnerable populations’.—Health administrator, federal

‘Government is conscious about what media are reporting and what public is thinking. The departments routinely monitor news for coverage that may damage their reputation and try to address it promptly. This also includes realizing the importance of these issues and including them to policy agenda as priority’.—Health administrator (retired), federal

Actors

The participants spoke at length about the negotiators who discuss policy decisions. At times these discussions about negotiators or actors merged with other elements, such as context or process in our policy framework.

Decision-maker

The ultimate decision-making lies with the ruling party, or cabinet, or head of the government at respective administrative levels, i.e. federal, provincial or district. How these decisions affect the public image of the leader and their chances of getting another term in the government are the usual motivations.The most paramount considerations are partisan interests. The devolution of ministries such as the Ministry of Health following the 18th constitutional amendment was not done specifically for health or for the actual transfer of authority; it was a political decision taken at the highest level of the legislature and ministries were asked only to comply. The premise may be correct however, not enough consideration was given to what and how things will happen at various ministries after devolution.Polity can promote not only the development of a certain policy; it can also stunt it if the context changes. A 2008 political government promoted the development of bottom-up health policy, involved several stakeholders, and oversaw the process of multiple drafts of this document. Towards its finalization however, the priority changed and political forces became interested more in devolving health to provinces than having a national health policy.Policymaking, however, has not always involved elected public representatives. On three different occasions during 70 years of the country’s history, civilian governments were sacked by a military coup, resulting in decade-long military rule each time. This not only changed the main actors for a significant period of time, it also affected the context of policymaking during those periods, and even afterwards.

‘It is customary for the political parties and their leadership to examine all policy initiatives in terms of leverage they will get. The current social or health insurance scheme being proposed to create a health safety net for the poor is a move that is politically beneficial to the government’.—Health official, federal

‘There were various political benefits to the parties involved in bringing the 18th constitutional amendment. At the time of its approval by the legislature, no one was thinking about how devolution of health and education etc. will materialize to the grassroots nor the authority was actually devolved to the lower most level’.—Health official, federal

‘It was interesting that the same government that promoted a bottom-up approach for a new health policy shelved its final drafts in 2010. This happened because 18th amendment was being considered under which health was to be devolved to provinces’.—Health administrator, federal

‘General Zia (1977–88) shelved the country’s population programme because of his notion of Islam, while ignoring data that supported the existence of such a programme, and nobody could dare ask a question. General Musharraf (1999–2008) ordered a health policy to reflect his vision and a few paged document was developed within days to fulfill his desire. Additionally both made political decisions that impacted security as well as health for many years during and after their rule’.—Health administrator (retired), federal

Administrative and technical groups and the fuzziness of their roles

Technical inputs

At the national level, various actors including academics, research professionals and their organizations, and the technical arms of various health programmes are expected to contribute to health policy through the offices of the Director-General of Health. Notwithstanding, the technical input—no matter how strongly grounded in evidence—may not always get incorporated into policy.At times the technocrats are not sufficiently forthcoming, especially when the process seems to be not favouring them. For example, the technical side at the federal level was not very keen on implementing devolution to the domain of health since it meant a decrease in their authority. There was little input and no interaction with the provinces in order for them to prepare for the devolution.Interestingly, there are times when policymakers ask for data but the technical side is unable to provide them. Decision-makers shared that data are not available in a timely manner and as such cannot be used in a meaningful manner. Locally driven research and evaluation are rare. The few that are available are donor-driven and usually interpreted in a way that builds the argument of those who funded them.

‘At best, the technical input is examined by the administrative and political levels of policy and if found to be fitting to their priorities and agenda, it is incorporated. This however is not always the case as the top level usually is far removed from ground situation’.—Health administrator (retired), federal

‘After the 18th amendment, legislature gave one year for implementation but the technical side and managers of health programmes saw it less favourable for their own office and did not focus on planning this devolution. As a result all the decisions were made in haste and executed by administrative offices’.-Health official, federal

‘Research and evaluation studies are carried out according to work plans that are agreed by those who fund it and those who carry it out. Little is discussed with government because of which appropriate information is not available even if sought by the government’.—Official, Planning Commission

Administrative inputs

The participants also shared that in a country where the policymaking occurs in a non-systematic way, mostly by compulsion at a time when the decisions cannot be delayed further, the administrative offices assume a dominant role in the process.With a supportive individual in office, the administration can play a key role in the incorporation of evidence. For example, the office of the federal health secretary played a significant role in receiving inputs from academics, technical experts, and civil society, while overseeing the development of various drafts of the Presidential Ordinance 2002 on the protection and promotion of breastfeeding.

‘Owing to frequent changes of governments, and transfers of both technical and administrative positions, we lack consistency in the policies. Areas like health gain importance only when the issues become unavoidable and administration is concerned with rapidity more than consensus’.—Health administrator (retired), federal

‘The secretary of health had the capacity to steer the process and the commitment to ensure that the draft remains confidential till its final vetting by the law ministry. This was necessary in a country where the infant formula industry is notorious in pushing the policymakers and bribing the health professionals’.—Civil society representative

Partners

Nearly all of the respondents discussed the role of bilateral and multilateral organizations in the policy process. The WHO, for example, advocated ‘Health for All’ and proposed the strengthening of primary health care (PHC) in the country. The Lady Health Worker Programme was initiated as a result of this process.The international commitments to Millennium Development Goals (MDGs) 4 & 5 and the continued lobbying and grant support of donors such as USAID and DFID led to the formulation of the Maternal, Newborn and Child Health (MNCH) programme in 2005. Similarly, the continued assistance from and obligations to the international community kept the polio eradication programme working.Credible research and improvement in research capacity is another area consistently supported by partners. The Pakistan Demographic and Health Survey (PDHS)—the most widely cited source of health indicators, and the Multiple Indicators Cluster Survey (MICS) are only two examples of such partnerships that provide data for evidence informed-policy.

‘The WHO collaborated with ministry to pilot-test a community health worker intervention as part of PHC in the country. These women workers were named Lady Health Workers. The pilot seemed so promising that the government decided to scale up even before the availability of final and conclusive proof of effectiveness’.—Health administrator (retired), federal

‘Countries in the world were keenly discussing their targets and plans for MDGs during late nineties. International partners like USAID and DFID proposed huge funding for MDG 4 and 5; as a result a new programme was launched to take care of MNCH in the country’.—Civil society representative

‘PDHS carried out every five years is an example of international support towards broadening the evidence base in Pakistan. Funded by the USAID, this survey is carried out every 5 years by reputable organizations who partner with local researchers; the next survey has been planned for 2017–18’.—Civil society representative

Process

The discussions about the process of policymaking involved primarily the federal and provincial levels in the two key time periods which were before and after the devolution of health to the provinces.

Ad-hocism

The majority of the respondents thought the technical-administrative-political nexus of policy is rudimentary and that policies are finalized in a non-systematic way. There is neither a culture of needs assessments nor one of paying attention to the available evidence. Even the budgetary allocations may not convey the policy intent as the budgets are often revised.While discussing current mechanisms at the provincial level, the participants did not see any change since 2012; the year health was devolved. Decisions are arbitrary, usually initiated from the office of the Health Secretary. Where political leadership is strong (e.g. chief minister of the Punjab province) it does play an active role however, the space for and contribution of technical information is minimal.

‘The policy decisions are reached in an unorganized, nearly chaotic way. There is usually no process or effort to be informed from the evidence. Decisions are usually delayed, reached by a few, at a time when there is no way out and no time to weigh the available options’.—Health administrator, federal

‘It is the Secretary’s office at the province and DHO at the district that are mainly responsible for development and implementation of policy decisions respectively’.—Program managers from provinces and regions

Provincial structures post-devolution

The province of Punjab has instituted the Punjab Strategic Planning Unit (PSPU), which works closely with the department of health (DOH) on data analysis, issues and challenges, strategic planning and new policies. The province also has established the Punjab Healthcare Commission, which is an autonomous body that aims to improve the quality, safety and efficiency of healthcare service delivery for all public and private establishments in the province.The situation however, is less favourable in other provinces and regions. In the Azad Jammu and Kashmir region (AJK), the PC-1 for a Health Systems Reforms Unit (HSRU) has been awaiting approval for a long time. The current system for data collection and management comprising the Health Management Information System (HMIS) and District Health Information System (DHIS) is being implemented in only five out of 30 districts. A similar situation was reported from Balochistan.

‘The PSPU and the Punjab Healthcare Commission are the steps towards making use of research and evidence into the development of policies that address local issues and problems, followed by their effective implementation and monitoring’.—Health official, Punjab

‘Out of the total 30 districts of the province, five do not report on HMIS. Those reporting usually submit incomplete data’.—Health official, Balochistan

Discussion

This study is the first systematic exploration of policy mechanisms in Pakistan that highlights the non-systematic, nearly ad-hoc process of health policy development and implementation in the country. Political actors seem to have the strongest influence on policymaking because of the power imparted to them through the centrality of the system. This centrality is part of the context, which comprises some additional features as well. Health, and therefore health policy, is not a priority in this context nor is the incorporation of evidence a norm. Weak capacities on both sides—the effective presentation of data in the case of researchers and understanding it for policy in the case of policymakers—are also part of this context. Post-devolution, the provinces are treading the same path when it comes to health policy and its implementation.

The fact that health policymaking is a complex process, that evidence alone cannot solve problems that are political in nature, and that policy may involve multiple social norms with different factors competing with each other, has been reported earlier (Greenhalgh and Russell 2006; Bosch-Capblanch ; Parkhurst 2016). The existence of this complex (actually chaotic) interplay in Pakistan is being documented for the first time. The country has a history of giving low priority to health. The manifestos of political parties, annual budget speeches in the parliament, and the oft-criticized low spending of 1% or less of the GDP (Nishtar et al.2013a, 2013b) on health need to be kept in mind, while understanding health policy and its implementation in the country.

An interesting finding from this study was that despite their role in policy, some participants perceived ‘others’ as policymakers and not themselves. They maintained the traditional top-down viewpoint according to which policy formulation is seen as a function of the cabinet and ministers while its implementation is an administrative or managerial process, disconnected from formulation. Some authors have argued that the two cannot be separated (Buse ). Implementation cannot be seen as a separate part of a sequential policy process in which political debate and decisions take place among politicians and civil servants, and managers and administrators at a lower level implement these decisions and generate data to feedback into the policy process (Walker and Gilson 2004; Koontz and Newig 2014).

We did not find provinces using evidence-informed policy following the devolution, while devolved policymaking has brought significant results in neighboring countries with similar context (Filmer ). For example, the Indian state of Kerala—a symbol of local wisdom solving local problems—has an annual per capita income of $1254 and an infant mortality rate of 31 per 1000 live births (Agrawal ), which is 40% lower than the Indian state of Punjab which has an income twice that of Kerala (Filmer ). The Indian states of Gujarat and Tamil Nadu also have made considerable improvements in their maternal and infant mortality through evidence-informed decisions at the state level (Krupp and Madhivanan 2009).

Any discourse about policy mechanisms is incomplete without discussing the process of agenda-setting. It is argued that governments themselves do this and setting up the agenda through party manifestos is common in democratic systems (Buse ). Much has been written about the role of the media in shaping the agenda (Kosicki 1993; Pan and Kosicki 1993). Some research (Harrabin ) also has shown that politicians are more likely to change their priorities based on media coverage compared to the evidence of what is in the public interest (Greenhalgh and Russell 2009; Parkhurst 2016). Our discussions with participants suggested that all of the above might be happening in Pakistan—something that health advocates should consider while strategizing their advocacy for policy.

Capacity emerged as a significant gap that needs to be addressed on both sides of the policy process—the generation of evidence and the policy formulation. Advocates of public health need to understand the mechanisms unravelled by our study and take earlier studies into account. Recommendations about good governance of evidence need to be adopted, including the appropriateness, transparency, accountability and contestability of data (Hawkins and Parkhurst 2015), as well as principles of interacting with policy including direct contact with researchers, timely availability of data, and synthesis of information in the form of summaries and recommendations (Innvaer ). Policymakers should be engaged in both conceptualizing and conducting research for their increased ownership as well as for their capacity to understand and utilize evidence for policy.

While this study carried out by local researchers from a lower middle-income country is important, a number of limitations need to be noted. No prior research on policy mechanisms in Pakistan was available that could guide this study carried out with minimal resources in a limited time. Owing to constraints, we could not include enough participants from the categories of politicians, private sector, public-private partnerships and global actors as has been suggested by scholars of this field (Walt ). Contacting senior government officials to be interviewees, getting appointments and then dealing with last-minute cancelations due to their urgent meetings, making maximum use of the available time, and not being able to tape-record discussions were some of the challenges encountered in carrying out this study. Such realities led to less than ideal solutions such as relying on hand-written notes for verbatim quotes and building a continuous story that was actually told in a fragmented manner.

Implications for future research include detailed case studies of legislative pieces related to public health (e.g. the promotion and protection of a breastfeeding ordinance or the banning of tobacco advertisements) that can be helpful in deepening the knowledge base. We also recommend further examination of party manifestos along with exploration of processes that precede these manifestos, as well those that roll out once the parties come into power after general elections. We feel this examination should synchronize with every election cycle and will help improve public health policy as well as strengthen true democracy in Pakistan. Impact evaluation of devolution of health to the provinces is another important area for research, which will help to understand both positive and negative outcomes from this seminal policy, enacted through the parliament.