A comprehensive model to evaluate implementation of the world health organization framework convention of tobacco control.

BACKGROUND: Iran is one of the countries that has ratified the World Health Organization Framework Convention of Tobacco Control (WHO-FCTC), and has implemented a series of tobacco control interventions including the Comprehensive Tobacco Control Law. Enforcement of this legislation and assessment of its outcome requires a dedicated evaluation system. This study aimed to develop a generic model to evaluate the implementation of the Comprehensive Tobacco Control Law in Iran that was provided based on WHO-FCTC articles.
MATERIALS AND METHODS: Using a grounded theory approach, qualitative data were collected from 265 subjects in individual interviews and focus group discussions with policymakers who designed the legislation, key stakeholders, and members of the target community. In addition, field observations data in supermarkets/shops, restaurants, teahouses and coffee shops were collected. Data were analyzed in two stages through conceptual theoretical coding.
FINDINGS: Overall, 617 open codes were extracted from the data into tables; 72 level-3 codes were retained from the level-2 code series. Using a Model Met paradigm, the relationships between the components of each paradigm were depicted graphically. The evaluation model entailed three levels, namely: short-term results, process evaluation and long-term results.
CONCLUSIONS: Central concept of the process of evaluation is that enforcing the law influences a variety of internal and environmental factors including legislative changes. These factors will be examined during the process evaluation and context evaluation. The current model can be applicable for providing FCTC evaluation tools across other jurisdictions.

INTRODUCTION

The tobacco epidemic is growing globally[1] resulting in a concurrent rise in smoking related morbidity and mortality.[2] Global action is essential because the mortality and morbidity caused by tobacco related disease increase health care cost in countries with limited resources.[3] Community health nursing is committed to maximize the potential of nurses, the largest group of healthcare professionals, in reducing adult and youth tobacco use, promoting cessation and actively protecting all people against tobacco smoke.[4] Direct foreign investment, free global trade and advertising, the affluence of the tobacco companies and the formation of transnational tobacco conglomerates have all contributed to the global rise in tobacco use.[5] On June 16 2003, the World Health Organization (WHO) convened a session in Geneva to develop a framework for tobacco control worldwide. The 38-article Framework Convention of Tobacco Control (FCTC) is the first international treaty with a mandate for public health. Signatories agreed to provide logistical and legislative support towards enforcement of the FCTC in their respective countries, with particular priority accorded to the protection of public health.[6]

To date, 168 countries have signed the FCTC. In addition to implementing interventions to meet FCTC’s stated goals, countries signing the FCTC are required to evaluate progress towards its goals in their communities and institutions. Evaluation is a systematic process to assess the level of implementation, as well as the short-and long-term outcomes of a program.[7] Specifically, the FCTC stipulates that signatories file periodic progress reports; government health authorities do this by completing annual report forms, which are analyzed and then summarized in reports in one of WHO’s eight official languages.[8] In most countries, enforcement of the FCTC has involved many challenges and requires approaches tailored to each country setting that are sensitive to cultural, economic and social circumstances. Similarly, the methods used to evaluate the implementation of the FCTC should take the particularities of local settings into account.

Iran signed the FCTC in 2006. Similar to all FCTC signatories, Iran implemented a series of tobacco control interventions within 3 years of signing the treaty, including the Comprehensive Tobacco Control Law. Enforcement of this legislation and assessment of its outcome requires a dedicated evaluation system that needs a multidisciplinary work including all health sectors. In this regard, nurses and nursing organizations are actively involved in developing and supporting other tobacco control centers[9] to promote prevention of tobacco for any use and to promote collaboration with other healthcare organizations, public health, and tobacco-control groups to strengthen tobacco control at all levels.[10] A community health nurse team in collaboration with Isfahan Cardiovascular Research Center, developed a broad model to evaluate the implementation of the Comprehensive Tobacco Control Law in Iran that is reported in this article.

MATERIALS AND METHODS

This study resulted from collaboration between the Isfahan Cardiovascular Research Center (a WHO Collaborating Center in the Eastern Mediterranean region), the Office of Environmental and Occupational Health of the Iranian Ministry of Health and Medical Education (MOHME), and FCTC policymakers. The first joint meeting of FCTC collaborators from the Office of MOHME and FCTC-related policymakers was convened in September 2007. A second meeting in October 2007 was attended by FCTC’s scientific collaborators at Isfahan Cardiovascular Research Center (ICRC). The FCTC goals and evaluation methodology which involved qualitative interview methods were explained to participants. All documentation predating the introduction of the Comprehensive Tobacco Control Law as well as tobacco-related legislation enforced after it, was compared with the FCTC articles. Existing national guidelines for enforcement of tobacco-related legislation were contrasted with each of the FCTC objectives. Qualitative methods were used to design a model for evaluating the implementation of the FCTC in Iran. A consensus was achieved in these meetings with respect to the main themes of interviews, questionnaires and related questions. The program collaborators identified three groups for interview in relation to implementation of the Comprehensive Tobacco Control Law in Iran, including policymakers involved in designing the legislation, stakeholders whose interests had to be considered in implementing the FCTC, and members of the community that law aimed to promote their health.

Data collection

Using an approach based on grounded theory, qualitative data on implementation of the tobacco control law were collected through field observation in supermarkets, shops, restaurants, teahouses and coffee shops. In addition, data were collected in individual and focus group interviews of adults aged 20 years or older; girls/boys aged 11-18 years from middle- and high school, male and female employees, and housewives. The questions used in the interviews were designed based on the FCTC articles. Sample size for the interviews was determined according to the principles of data saturation in qualitative research,[11] such that sampling and interviewing were discontinued when interview codes were repeated (which signified saturation). A total of 265 individuals were interviewed including 13 policymakers, 76 supermarket/shop workers, 22 owners of restaurants/ teahouse/ coffee shops, 8 tobacco farmers, 31 students, 34 office clerks (male and female), 27 university students, and 54 members of the community.

Data generation

The interviews with policy makers were unstructured and focused on the experiences of activities necessary for implementation of FCTC articles and the indicators for evaluation of these activities. Interviews with general population and tobacco sellers were structured and focused on FCTC implementation and the ways to control it. All interviews were recorded. Tobacco advertisements in stores were marked in checklists and interesting findings were stated in field notes.

Data analysis

According to the Glaser and Strauss method,[12] data were recorded and then they were written on paper; the interviews were read line by line and important sentences that were relevant to FCTC concepts were highlighted. In turn, the highlighted sentences were selected as meaningful keywords and were written in the paper borders. According to the Straus and Carbin method,[13] this stage is termed as open codes. We used constant comparison analysis, i.e. analysis was conducted simultaneously with interviews and open codes were extracted. From various interviews, 617 codes were extracted.

The open codes extracted from the data were grouped to 72 categories as external factors, such as tobacco advertisement, encouragement to cigarette smoking and tobacco production by government; and internal factors such as attractiveness of tobacco advertisements, difficulty and the tobacco quality. In the next step, the aforementioned 72 codes were revised and after merging similar codes, 5 pivotal variables were determined.

Using a Model Met paradigm, the relationships between components of each paradigm were displayed graphically based on researchers’ understanding of concepts (drawing from interviews), and meetings with colleagues and discussions about the interrelationship of components. These graphs depict how concepts relate to each other. This model was designed based on concepts and paradigms, following several meetings with program collaborators and discussions about the collected data and codes.

Concerning data accuracy and authenticity, regular trends, assimilation (simultaneous use of several research methods and data collection for verification), inspection and revision by colleagues, and revision by the participants were means of ensuring the validity and reliability of research and its findings.[14] Data analysis using Glaser’s method was performed in two stages through conceptual theoretical coding. The study was ethically approved by the Research Council of ICRC.

FINDINGS

The result of conceptual coding was extracted into tables which included 617 codes (i.e., open codes). It is worth noting that codes for each interview were extracted before conducting the next study. After reviewing code series pertinent to level 2, seventy-two level-3 codes were collected after summing up and assimilation of similar concepts and met paradigm by qualitative researchers (Table 1). After obtaining level-3 codes and Model Met paradigm, researchers made graphs to ascertain the relationship of the components of each paradigm. These graphs were made based on researchers’ understanding of concepts (drawing from interviews), and meetings with colleagues and discussions about the interrelationship of components. These graphs depict how concepts relate to each other. Figure 1 shows the components of the process of evaluation. The components of different factors in the model for the process of evaluation are depicted in Figure 2. Figure 3 presents the structure of the final model of evaluation provided by the current study. This model was designed based on concepts and paradigms, following several meetings with program collaborators and discussions about the collected data and codes.

Table 1

Extraxted evaluation concepts and model met paradigms from level-3 codes

Figure 1

Components of the implementation process in the model of evaluation

Figure 2

Components of various factors in the model of evaluation

Figure 3

Structure of the final evaluation model

The National Evaluating Committee

Among other goals was the suggestion of members of the National Evaluating Committee. Based on the obtained paradigms, which were discussed and evaluated in several meetings, the parties who benefited from implementation of the program were members of the National Evaluating Committee; they will be referred to as the headquarters structure and were representatives of the cabinet, the Health and Treatment Commission of the parliament, the MOHME, two non-governmental organizations involved with tobacco control, community members, the State Radio and Television, Ministry of Guidance, Ministry of Education, and Ministry of Agriculture (tasked with evaluating instances referred to their respective institution for implementation and presenting reports about implementation of these instances to the National Evaluating Committee), scientific societies such as the Society of Cardiologists, the Society of Pulmonologists and Cancer Specialists (to evaluate long-term outcomes and enforce the care system for tobacco-related diseases), the Ministry of Commerce and Trades Office, the Police and the Ministry of Economy and Finance.

Levels of evaluation

Evaluation indicators were defined based on level-3 codes, concepts, and ultimately the model paradigms. Given that the definition of evaluation entailed three levels (namely short-term results, process evaluation, and long-term results), the paradigms developed were assigned to these three levels. Therefore, the tool that will be designed for evaluation will be including these three levels. All level-3 codes of this program’s qualitative studies were considered as measurement indicators in these three levels.

The Comprehensive Tobacco Control Law, based on FCTC articles, targets community leaders as well as individual members of the community; hence the indicators target these groups. Moreover, in view of the FCTC goals and the Comprehensive Tobacco Control Law, the interventions for implementing FCTC were designed as educational strategies and change of behavior in individuals, as well as changes in the legal and environmental structure of the country. Hence, the indicators were determined in three areas, namely policymaking, environment and educational-behavioral. Given each set of indicators and different levels of evaluation, the frequency of conducting evaluations was set at once every year for short-term results, process of evaluation, and once every 5 years for long-term results. In addition, the feedback of outcomes would be presented as annual reports containing tables of indicators, to the evaluating committee of the Comprehensive Tobacco Control Law, the President’s Office, The Health and Treatment Commission of Iran’s Parliament, and representatives of non-government organizations. The 4 main pillars of evaluation can be studied by use of this model; these include context evaluation, process of evaluation, short-term outcome evaluation and long-term outcome evaluation. Levels of evaluation were discussed in the model as context evaluation, process evaluation, short-term outcome evaluation and long-term outcome evaluation. Program collaborators decided that process evaluation and short-term outcome evaluation would be conducted annually opinions varied between 3 and 5 years for evaluation of long-term outcome. In view of the effect of the tobacco control law on the prevalence and incidence of diseases, evaluation of long-term outcome would be conducted every 3 years. Evaluation of short-term outcome would be conducted annually and the results would be presented to the highest authorities of the health system; it can be integrated in the annual surveillance of risk factors conducted by the MOHME. As the process of evaluation constituted one of the pillars of evaluation, this committee had to convene monthly meetings to examine the process of implementation of the law and the outcomes of the process evaluation.

DISCUSSION

Huge health promotion intervention programs like the FCTC must be planned as a comprehensive community based program. All of the health policy makers and health sectors have to cooperate to implement these kinds of interventions. Community health nursing is one the main health promotion program.[4] Therefore, to achieve success in implementing the FCTC, nurses must work with other parts of health services to control and prevention smoking especially on health education and community work to treat smokers. Considering achievable, accessible, feasible and measurable goals are another important point to implement health promotion program.[1516]

According to article 3 of FCTC, this treaty aims to protect the present and future generations against health, environmental and economic harms arising from production and use of tobacco products. This goal may be achieved by designing a universal framework that would require the member states and signatories to enforce its articles. Articles 4 and 5 concern general methodology, indigenization and reporting by countries. In every country, these goals will have to be achieved according to the laws and individual circumstances. However, the legislated laws must be fundamentally geared towards achieving these goals. In other words, a country could be considered as having enforced the treaty which can demonstrate achievement of its goals through a proper system of evaluation. It is noteworthy to mention that the WHO questionnaire does not stipulate any external evaluation on the process whereby these questionnaires are completed.

Moreover, WHO has designed FCTC by focusing on its enforcement through laws that would be legislated in all countries; it has therefore paid less attention to evaluation of environmental changes and/or differences arising from cultural, economic and social characteristics of individual countries. The latter are two areas of weakness affecting this method of evaluation.

Tobacco use is not just a health concern, but rather it is related to economic and cultural issues in the community. Therefore, a host of organizations and ministries must be engaged towards implementation of the Comprehensive Tobacco Control Law; hence, Article 1 of this law concerns the formation of the Headquarters and its members. In the current national law, members of the headquarters include: the Health Minister (head of the headquarters), Minister of Culture and Islamic Guidance, Minister of Education, Minister of Commerce, the parliamentary Health and Treatment Commission (as supervisor), head of State Radio and Television (as supervisor), and representatives of tobacco-related NGOs. One of the main responsibilities of the headquarters is to develop guidelines concerning the definitions, criteria, and characteristics of advertising (Clause A, Article 2). As enforcement of laws in the community requires cultural preparation and acceptance, Articles 2, 3, and 5 of the Comprehensive Tobacco Control Law concern education, research and advertising. Clause B of Article 2 of the Law tasks the headquarters with developing and ratifying educational and research programs. Articles 3 and 5 deal with prohibition of tobacco advertising and the use of health labels, respectively. The Law takes heed of the nearly 15% prevalence of tobacco use in the community and the presence of tobacco addicts, hence it stipulates the development of smoking cessation clinics. Article 9 considers the integration of prevention, treatment, rehabilitation and smoking cessation counseling services into primary health services (PHC) as the responsibility of the Health Ministry. Other aspects of the Law deal with tobacco taxation, tobacco-related crimes, and regulations banning the importation, sale and consumption of tobacco products. In view of the articles of the Law and guidelines for enforcing it, which were sent to related organizations in 1997, it can be concluded that the Comprehensive Tobacco Control Law conforms to FCTC themes. In many instances, the Law features provisions for its enforcement. Articles 8, 9, and 18 of the Law explain the budgetary requirements of the Law. The need for evaluating the enforcement of the Law becomes even more evident in light of the fact design, evaluation and initiation of every comprehensive intervention program, especially one of national scope, must start with its implementation.[17] Six-monthly reporting of the headquarters’ performance to the government and the parliamentary Health Commission has been emphasized in appendix 2 of Article 1. In view of the arguments made earlier, the shortcomings of WHO questionnaires, and the need for indigenization of national tobacco control laws according to each country’s cultural, economic, and health requirements as per Article 4 of FCTC, there is an added need for identifying evaluation indicators and designing tools for evaluation based on Iran’s Comprehensive Tobacco Control Law. One of the important cultural differences between tobacco use in Iran and the West is hookah smoking. The increasing number of teahouses and hookah lounges and Iranian youths’ growing interest in hookah smoking as a pastime is one of the important indicators highlighted by policymakers, which needs to be evaluated in the target community. This is a prime example of a significant tobacco-related issue not addressed in the WHO questionnaire. Another example is the governmental nature of tobacco and cigarette production in Iran, which is considered as an important indicator for evaluation. In fact, the very government that is supposed to counter tobacco use, produces it. We will need to adopt a method for evaluating tobacco revenues, from farming to production and importation. The outcome of such evaluation will have to be made transparent for the government through presenting comparisons with prevalence of tobacco-related disease. The effect of the Tobacco Control Law on employment is another important indicator. The government’s main goal in many programs is creating opportunities for employment. In this study, one of the indicators of evaluation must be the reduction in employment and the possibility of providing alternative careers for those earning their livelihood in the tobacco industry. In addition, the nature of intersectoral cooperation, policy-making and supervision of ratified laws in Iran is different from those in other countries; it is important to take this difference into consideration when determining the indicators. The participation in this study, of the MOHME, the Office of Occupational and Environmental Health of this ministry, all members of the Headquarters, policymakers, stakeholders and community members, allowed the extraction of the required codes for designing a country-specific evaluation model. These codes, which were mentioned above in the results section, provide a foundation for determining indicators for context evaluation, process of evaluation, short- and long-term outcome evaluation:

Context Evaluation

Context items included: a) fundamental items as community, vulnerable groups, support systems, forces advocating tobacco, and beliefs against tobacco, as well as b) internal factors as follows: unpleasant cigarette smoke, tobacco quality, double-edged advertising, bad image of cigarette smoking and acceptability of smoking hookah.

Process Evaluation

It evaluated the following issues: a) policy; b) headquarters structure including government, parliament, sponsors and implementers; c) implementation process, which covers education and research, control and evaluation methods, links with the tobacco network, information system, inspection and regular and periodical surveillance of enforcement of articles of the Law; d) strategies of implementation and external factors comprising desirability of location for customers, encouraging smoking, tobacco production by the government; e) root factors as livelihood, profiteering groups, support system and the private sector, and supporters of profiteering groups, and f) intermediary factors consisting of prohibiting forces, taxation and facilitators.

Short-term Outcome Evaluation

It consists of a) tobacco network including centers of production, sale and supply of tobacco, existing locations in the community, potentially vulnerable locations, and tobacco product stores, and b) consequences of tobacco use encompass number of tobacco users and interest in tobacco.

Long-term outcome evaluation

It consists of the consequences of cigarette smoking, which are tobacco-related diseases and cigarette-related mortality.

“Policy” constitutes one of the principal concepts in the evaluation model. In this study, the concept of “policy” encompasses all tobacco-related legislation, including the processes and the path of production to consumption of tobacco products. This path includes all legislation concerning tobacco farming (support, supervision and control mechanisms at all stages of farming), imports (mechanism, amount of imports and laws related to supervision of operations and control of possible risks), distribution (rules concerning the legal and approved tobacco distribution networks, from the producer to the consumer), supervision and control (mechanisms for supervision of the legal production process, importation and consumption, as well as provisions for dealing with infringement) and sanctioning the implementation of the law’s articles. Control policies constitute yet another key element. Implementation of the program would be challenged in the absence of clear and unequivocal understanding of the existing policies relating to control, production, sale, distribution and importation of tobacco products. The indicators regarding the supply, demand and sale of tobacco products are useful in evaluating the implementation process of the program; hence, they are of special value in the evaluation program. Adequate funding is not always available for complete implementation of large-scale programs. Therefore, areas of spending need to be prioritized to achieve maximum efficacy. On a par with this evaluation model, other studies have also emphasized the central importance of policy-making, examining its relationship with the success achieved in tobacco control in the proposed models.[18]

The community’s understanding of tobacco and policies aimed at countering and controlling it will undoubtedly influence the success or failure of such programs in the context of the community; hence, taking account of “context” which derives mainly from the depths of the community is vital to models for implementation and evaluation of tobacco control programs.[19-21] The National Tobacco Control Law is a very large program, the enforcement of which is naturally beyond the capability of a single organization; what guarantees the enforcement of this law is the full cooperation of all the institutions involved in the program. Organizations and ministries such as the Ministry of Health, Treatment and Medical Education, Ministry of Culture and Islamic Guidance, Ministry of Education, Ministry of Commerce, Police, parliament and the state Radio and Television are vital components each with a significant role in implementation of this program. However, multifaceted operations involving several organizations undoubtedly require careful planning, and clear instructions on the tasks and responsibilities of each of the organizations in the relation to the law in questions. Such planning must delineate extend of authority, as well as responsibilities of every organization. In fact, transparency in defining the responsibilities and scope of activity of the individual components of the headquarters is essential if interference of their tasks is to be avoided and if each of them is to be held accountable in respect of their responsibilities. Thus, and in light of the findings, one of the important indicators for evaluating the enforcement of the Comprehensive Tobacco Control Law is defining procedures for intersectoral cooperation and assessing the degree of commitment shown by each member of the Headquarters towards their responsibilities.

The community, the vulnerable or at-risk groups, especially children and adolescents, tobacco control support systems, forces encouraging tobacco use, popular beliefs regarding tobacco use, psychological effects such as unpleasant smell or soothing effect of smoking, bad image of smoking, acceptability of hookah smoking as a pastime not an addiction, drawing tourists, and tobacco as a double-edged sword must be evaluated. The latter means increasing tobacco taxation/prices boosts tobacco smuggling; hence, the high profitability of tobacco smuggling affects the enforcement of the Tobacco Control Law as an internal factor. Thus, context evaluation must find defined border between profit-making from increased taxation and the harms of tobacco use. Many of these concepts have been collected by different organizations and ministries. Consequently, all of them will merely have to be gathered together by committees tasked with evaluation of the Comprehensive Tobacco Control Law. Process of evaluation is conducted after context evaluation and during enforcement of the Law. This indicates the polygonal nature of the concepts extracted in this study. Another concept which must be examined in the process of evaluation is the structure of the headquarters as the executive arm of the Law; i.e., manner of holding meetings, its decisions and ways of implementing them and finally, adopting related policies, themselves among the concepts outlined in the model. Given the goals of the Law in the country, achieving short-term and long-term outcomes requires an evaluation system based on evaluation of these outcomes. The concepts related to short-term outcomes include the tobacco network and consequences of tobacco use.

The components, which make up these concepts are durable environmental changes, as well as number of tobacco users and level of interest in tobacco use as a behavioral change. As outlined in the model, context and process of implementation will influence these concepts. It is important to consider the role of police in the process of enforcement, as well as their attitude towards enforcing the law. Only adults favoring stricter control on adolescents approve of police involvement in the supervision and control process. Adolescents’ view of police involvement in enforcing the Law is a negative one, and is among the important indicators during evaluation of the extent of enforcement of the Law at different levels.

CONCLUSION

In accordance with the developed model and its components, as well as the different levels of evaluation, we propose that national evaluation be conducted as context and the process evaluation. As outlined in our suggested model, the process of enforcing the law and the adopted strategies influence all types of factors as one of the concepts of the process of evaluation. Notably, some of these factors themselves are among environmental and legislative changes that will be examined during the process of evaluation, while others are among internal factors which will be examined in the context evaluation. Foremost in this evaluation are analysis of the present situation and the possibility of introducing change. Of notable importance in the extracted indicators is that the focus of policymakers is on long-term outcomes and process of implementation. The ultimate goal pursued by the Tobacco Control Law is reduction of the incidence of tobacco-related diseases and their associated morbidity, which is affected by all concepts in the model. Given the comprehensive vision embedded in the program’s central goals (i.e., reducing tobacco use and its associated mortality/morbidity), the policymakers have paid special attention to the process of enforcing the Law and intersectoral cooperation.