Feasibility and practicability studies of an evaluation index system for implementation of smoke-free laws and regulations of cities in China.

INTRODUCTION: The aim of the study is to adjust and improve the feasibility and practicability of an evaluation index system, developed for the implementation of smoke-free laws and regulations, by applying it to analyze and compare the contents and implementation of smoke-free laws and regulations in eight legislative cities in China.
METHODS: In regard to indicators of the established evaluation index system, data on tobacco control/smoke-free laws and regulations and their implementation in some legislative cities in China were collected, sorted, and compared, to examine and adjust the feasibility and practicality of the indicators.
RESULTS: The evaluation index system for implementation of smoke-free laws and regulations of cities in China has been adjusted to include 5 first-level indicators, including: 1) Simple, clear and executable legislation; 2) Careful planning and sufficient resources for law enforcement; 3) Participation of civil society; 4) Communication; and 5) Monitoring and evaluation of the implementation and impact of laws. There were 27 second-level indicators, including management department, scope of smoke-free places, requirements for non-smoking places etc. There were 27 third-level indicators, including construction of law enforcement, coordination network, work of coordination agencies, staffing situation etc.
CONCLUSIONS: The preliminary adjusted evaluation index system for the implementation of smoke-free laws and regulations of cities in China provides a reference and standardization for evaluating effectiveness of implementation of Chinese tobacco control laws and regulations.

INTRODUCTION

Creation of a smoke-free environment through the implementation of comprehensive smoke-free legislation is considered the only way to effectively protect people from the dangers of secondhand smoke[1]. Assessment of the implementation of tobacco control regulations, an important process to ensure effective implementation of smoke-free regulations, can identify existing problems and provide guidance and reference in upgrading tobacco control work. At present, there is not a well-recognized evaluation system for the implementation of China’s smoke-free regulations, nor a selection standard of evaluation methods and indicators, and the monitoring and evaluation capabilities in tobacco control at city-level are still weak, resulting in that evaluation results are not comparable among cities.

In 2019, Luo et al.[2] established an evaluation index system for the implementation of smoke-free laws and regulations of cities in China and assigned a weight to each indicator. The evaluation index system included 5 first-level indicators: 1) Simple, clear and executable legislation; 2) Careful planning and sufficient resources for law enforcement; 3) Participation of civil society; 4) Communication; and 5) Monitoring and evaluation of the implementation and impact of laws. There were 29 second-level indicators and 33 third-level indicators (Supplementary file).

This article develops a data collection table based on the established evaluation index system for the implementation of smoke-free laws and regulations of cities in China, to collect, sort and analyze the data of the legislative cities, and to analyze the content of the smoke-free laws of cities and the actual progress of smoke-free laws and regulations enforcement to improve the evaluation index system.

METHODS

Selection of research cites

From the cities that have implemented tobacco control laws and regulations in China, taking their actual work situation of tobacco control legislation, law enforcement and publicity into consideration, eight cities were selected as the research sites: Beijing, Shanghai, Shenzhen, Harbin, Hangzhou, Lanzhou, Qingdao, and Xining.

Data collection and sorting

Based on the evaluation index system determined by our previous research[2], and through discussions among the research team, a data collection form was designed, ‘Data Collection Form for the Implementation of Urban Smoke-Free Regulations (Trial)’, hereinafter referred to as the Data Collection Form (Trial).

The Data Collection Form (Trial) was sent to the main coordinating managers of urban tobacco control work by email/WeChat. Data on urban law enforcement, publicity and education, social participation, monitoring and evaluation etc., were collected, including related documents/notifications, law enforcement data, reports, etc. At the same time, other information was collected through the Internet, including the text of urban smoke-free regulations, urban population, and economic level, which mainly came from Baidu Encyclopedia, the official website of the City People’s Congress, the official website of the Bureau of Statistics etc. Part of the data comes from the report of the Chinese Center for Disease Control and Prevention.

After completion of the data collection, the data were organized and summarized one by one according to the evaluation index system. The problems found during the sorting and analysis process were recorded and discussed by the research team, and finally reasonable solutions were agreed upon.

According to the established evaluation index system, the status of smoke-free laws and regulations and their implementation in various cities were analyzed and the index system was adjusted.

Statistical analysis

We used Excel 2016 to establish a database for data collection results on the implementation of smoke-free laws and regulations.

Quality control

The Data Collection Form (Trial) was finalized after experts’ consultation and revision and after sending out the Data Collection Form (Trial) by email, ensuring that all city contacts received the email by phone or WeChat. After data collection, the data were collated by two professionals at the same time. If there was any inconsistency, it was resolved through discussions by the research team.

RESULTS

Contents of smoke-free laws and regulations

Regulatory level, scope of application, and administrative departments

The level of legislation for smoke-free regulations in the research cities was local. From the perspective of the scope of application, except for the Harbin City Regulations on Preventing the Harm of Secondhand Smoke Tobacco Smoke[3], which covers the central city, the rest covered the whole cities. From the perspective of administrative departments, the main management departments for smoke-free regulations in seven cities are the Patriotic Health Campaign Committee, the Health Promotion Committee, and the Smoking Control Leading Group and other institutions, while Hangzhou is the health administrative department.

Enforcement procedures and fines

From the perspective of enforcement procedures and fines, each city varies. The smoke-free laws of Shanghai and Lanzhou stipulate that both individuals and venues can be fined directly, while Beijing, Shenzhen, Harbin, Qingdao, and Xining, stipulate that individuals can be fined directly, whereas venues need to be warned and ordered to make rectifications within a time limit. Those who fail to make rectifications shall be fined. Hangzhou City stipulates that both individuals and venues need to be warned first, and those who refuse to make rectifications shall be fined (Table 1).

Table 1

Basic situation of smoke-free laws and regulations in eight cities

City	Scope of application	Administrative departments	Fine amount (RMB)
			Individual	Place
Beijing	The whole city	Patriotic Health Campaign Committee (Office)	50;200	2000–10000
Shanghai	The whole city	Health Promotion Committee (Office)	50–200	2000–10000 10000–30000 (The circumstances are serious)
Shenzhen	The whole city	Joint Conference on Tobacco Control Work (The office is located in the Municipal Health Commission)	50;200;500	10000–30000
Harbin	The central city	Tobacco Control Regulations Leading Group Work (Office)	200	2000–10000;1000–30000 (The circumstances are serious)
Hangzhou	The whole city	Health Administration Department	50	500–2000
Lanzhou	The whole city	Tobacco Control Leading Group (The office is located in the Municipal Health Commission)	50;200	2000–10000
Qingdao	The whole city	Patriotic Health Campaign Committee (Office)	200	1000–10000;10000–30000 (The circumstances are serious)
Xining	The whole city	Patriotic Health Campaign Committee (Office)	50;200	1000–30000

RMB: 1000 Chinese Renminbi about US$160.

The requirements of smoke-free laws and regulations on non-smoking venues

Judging from the requirements of the laws and regulations on non-smoking venues, all eight cities have imposed the necessary duties and statutory obligations on the managers or operators of the venues. However, from the perspective of specific requirements, there are differences: although all cities have requirements for ‘set up standard no-smoking signs’ and ‘dissuade illegal smokers’, the other items are not required. Only the ‘Beijing Regulations on Smoking Control’ made the requirement of ‘carry out tobacco control inspections and keep records’ on non-smoking venues (Table 2).

Table 2

The requirements of the smoke-free laws and regulations for non-smoking places in eight cities

City	No smoking management system	Set up standard no-smoking signs	No smoking utensils	No tobacco advertisement	Carry out tobacco control inspections and keep records	Discourage illegal smokers	Set up smoke control inspectors or supervisors	Publicity and education
Beijing	√	√	√	√	√	√		√
Shanghai		√	√			√	√	√
Shenzhen	√	√	√	√		√	√	√
Harbin	√	√	√			√	√
Hangzhou	√	√	√	√		√
Lanzhou	√	√	√	√		√	√	√
Qingdao	√	√	√	√		√
Xining	√	√	√			√	√

√ Means there is such a requirement.

Careful planning and sufficient resources for law and regulations enforcement

Personnel and law enforcement funding input in cities

Lanzhou and Harbin have full-time staff responsible for tobacco control coordination. Qingdao City has the highest proportion of law enforcement personnel directly involved in tobacco control law enforcement (24.54%), and Hangzhou City has the lowest (12.28%). The results show that none of the eight city law enforcement agencies has set up separate funds for tobacco control law enforcement, so it is impossible to calculate the investment in tobacco control law enforcement (Table 3).

Table 3

Status of staffing related to tobacco control in each city

City	Coordinating agency staff *		Law enforcement officers *
	Full-time	Part-time	Total number of departmental law enforcement officers	Number of people directly involved in tobacco control law enforcement	Proportion of persons directly involved in tobacco control law enforcement (%)
Beijing	0	10	1600	300	18.75
Shanghai	0	36	11207	2129	19.00
Shenzhen	0	15	5745	1000	17.41
Harbin	5	30	1733	329	18.98
Hangzhou	0	0	57	7	12.28
Lanzhou	3	8	8780	2193	24.98
Qingdao	0	3	2991	734	24.54
Xining	0	19	600	250	24.00

Both include cities and counties (districts) and all law enforcement agencies. Some data are estimated based on the number of jurisdictions; the number of law enforcement officers in Xining City includes only the Food and Drug Administration.

Documents related to the implementation of laws and regulations

All cities have issued plans or programs on tobacco control law enforcement in the form of documents, but the names and quality of the documents are different. At the beginning of each year, Beijing, Shanghai, Lanzhou etc. issued the tobacco control law enforcement work-plan for that year to clarify the key tasks of each law enforcement unit that year. However, from the perspective of law enforcement guidelines or detailed rules, only Shenzhen and Lanzhou currently have corresponding documents. Hangzhou City has issued guidance on the discretion of the punishment procedure. From the data collected so far, no other cities have formulated such documents.

Law enforcement assessment system

Judging from the data collected so far, except for Qingdao City, there are corresponding supervisions and inspections. However, the systems implemented in various cities are inconsistent. Beijing, Harbin, and Lanzhou, have formulated regular assessment systems. The assessment content involves the implementation of responsibilities of law enforcement agencies, the construction of tobacco control law enforcement work systems, and the construction of smoke-free environments in venues. The assessment methods include documents review, reporting meetings, on-site inspections, etc. The supervision and inspections conducted in Shanghai, Shenzhen, Hangzhou, and Xining, are only inspections of non-smoking venues and do not involve the implementation of the duties of law enforcement agencies or the government.

Law enforcement tools

Except Hangzhou and Lanzhou with lack of invoice and confiscated accounts, other cities are equipped with corresponding law enforcement tools (Table 4).

Table 4

Situation of law enforcement tools in eight cities

City	Tobacco control enforcement invoice or receipt	Tobacco control enforcement penalty account	Law enforcement equipment*
Beijing	√	√	√
Shanghai	√	√	√
Shenzhen	√	√	√
Harbin	√	√	√
Hangzhou			√
Lanzhou			√
Qingdao	√	√	√
Xining	√	√	√

√ Means there is such a tool.

Including law enforcement recorders etc.

Non-smoking sign posting rate and qualification rate

Judging from the current data collected, only Beijing, Shanghai, and Shenzhen, have carried out corresponding survey work, while other cities have not carried out relevant work. Judging from the results, the non-smoking signs are all used in the three cities, but Shanghai has not adopted qualification rate of the non-smoking signs.

By the end of our survey, the results of Shenzhen in 2018 were not released, so we only compare the results of 2016 and 2017. The posting rates in the three cities all exceeded 80%. The rate of non-smoking signs in Shanghai was the highest, exceeding 95% in both 2016 and 2017. In 2017, the qualification rate of non-smoking signs reached 93.3% in Beijing and 70.3% in Shenzhen (Table 5).

Table 5

Posting rate and qualification rate of non-smoking signs

City	Non-smoking sign posting rate or setting rate (%)			Qualification rate of non-smoking signs (%)
	2016	2017	2018	2016	2017	2018
Beijing	82.0	94.7	96.4	†	93.3	†
Shanghai	95.2	98.0	97.7	†	†	†
Shenzhen	81.5	86.8	†	59.1	70.3	†

† Indicators are not adopted or the survey in the data collection phase is not completed.

Law enforcement actions and fines

All cities have carried out tobacco control law enforcement activities. Beijing, Shenzhen, and Shanghai, have a relatively high number of sites inspected, with more than 50000 sites each year. In particular, Shanghai had more than 150000 sites in 2016 and more than 200000 sites in 2017 and 2018. In contrast, Harbin had much fewer conducted inspections, less than 7000 establishments in 2016–2018. From the point of view of punishments on establishments, Shenzhen had the largest number of warnings and orders for rectification every year, more than 5000 establishments. However, Shenzhen imposes fewer fines and fines on establishments, with less than 10 establishments every year; in 2017 and 2018, Shanghai had 1133 and 1671 establishments fined and penalized, the highest among the eight cities. From the perspective of the fines imposed on individuals, except Shenzhen and Beijing, other cities rarely impose fines on individuals; Shenzhen imposes fines on more than 20000 individuals who smoke illegally each year, which is the highest among the eight cities and is also much higher than Beijing’s annual fines on over 3000 individuals (Supplementary file).

Communication

Judging from the data collected so far, Shanghai has invested the most in publicity, with an annual investment of more than 3 million RMB (1000 Chinese Renminbi about US$160). Large-scale publicity activities have been held in Shanghai, Shenzhen, Lanzhou, and Xining, more often. From the perspective of communication methods, various cities have adopted various approaches for tobacco control communication. Beijing, Shanghai, and Shenzhen, conduct annual surveys to monitor the public’s awareness of tobacco hazards. However, the data collected so far is limited. In 2017, the awareness rates of tobacco hazards in Shanghai and Shenzhen were 69.6% and 68.0%. As for the core knowledge of smoke-free regulations, the indicators adopted by cities are inconsistent, including: awareness rate of smoke-free regulations, awareness rate of non-smoking venues, awareness rate of fines, awareness rate of complaint hotline, etc. The indicators adopted by each city are one or more of the above.

Monitoring and evaluation of the implementation and impact of laws

Judging from the data collected so far, from 2016 to 2018, only Beijing, Shanghai, and Shenzhen, have formed a regular monitoring and evaluation mechanism, which monitors and evaluates the implementation of laws and regulations every year, and releases the evaluation results to the public. From the perspective of the evaluation of the implementation of laws and regulations, only Beijing, Shanghai, and Shenzhen, have conducted surveys on illegal smoking rates, tobacco smoke concentrations in non-smoking venues, and support rates for smoke-free laws. It is only these three cities that have conducted by themselves ‘The survey on the smoking prevalence of adults over 15 years old’ in 2016–2018, while the survey conducted in 2013 in Harbin, Hangzhou, Lanzhou, and Qingdao, was conducted by the Tobacco Control Office of the Chinese Center for Disease Control and Prevention. From the perspective of the health effect evaluation of the implementation of smoke-free regulations, only Beijing conducted one, but the relevant results have not been published. The remaining seven cities have not carried out or reported relevant evaluations. However, it is good to know Beijing, Shanghai, Shenzhen, and Harbin, have established surveillance systems for the incidence or death of tobacco-related diseases.

Adjustment of the evaluation index system

On the basis of collating and analyzing the data of each city, the following adjustments were made to the indicator system:

Exclude ‘1.2 scope of application’, ‘1.10 advocacy clause’, ‘2.1.4 investments in law enforcement per 10000 people’, ‘2.2.4 pass rate of non-smoking labels’, ‘2.5.1 number of inspection sites per 10000 people’, ‘2.5.2 The number of warnings and orders for rectification per 10000 people’, ‘2.5.5 Amounts of fines per 10000 people’;

Improve the scores of the following indicators in the evaluation index system ‘1.6 Requirements for Non-Smoking Places’, ‘Establish a smoking management system’, ‘Set up standard no-smoking signs’, ‘No smoking utensils and no tobacco advertisements’, ‘Dissuade illegal smokers’ and ‘set up tobacco control inspectors or supervisors’;

The evaluation indicators of ‘2.1.3 Staffing Situation’ only includes the staff of the tobacco control coordination agency, excluding the law enforcement personnel;

The evaluation indicators of ‘4.3 Communication Methods’ should highlight the role of traditional media. It is recommended to score in accordance with ‘traditional media’, ‘new media’, and ‘other methods’, and increase the score of ‘traditional media’, or continue to explore more objectively quantitative indicators for evaluation;

The scores of the World No Tobacco Day communication activities are highlighted in the evaluation criteria of ‘4.4 Publicity Activities’;

‘4.5.1 Awareness rate of core knowledge of smoke-free laws and regulations of the public’ is replaced with ‘4.5.1 Awareness rate of smoke-free laws and regulations’;

‘4.5.2 Awareness rate of public tobacco hazards’ is replaced with ‘4.5.2 Awareness rate of stroke caused by smoking’.

See Supplementary file Appendix 3 for the adjusted evaluation index system for the implementation of smoke-free regulations.

DISCUSSION

Indicators of the contents of laws and regulations

At present, there are no national tobacco control regulations in China, and there are also large differences between urban regulations, such as the scope of smoke-free venues and fines. This difference will have an impact on the efficiency of the implementation of the regulations, so it is necessary to include indicators about the contents of the regulations in the evaluation index system.

The management requirements of no-smoking areas are the duties and statutory obligations that must be fulfilled by the operators and managers of non-smoking areas, and are of great significance to the creation of smoke-free environments. At present, all cities are advocating ‘Four Availables, No ash tray or tobacco ads, and Dissuasion’ but the requirements of each city are still different. In order to play an exemplary role, the evaluation clauses in ‘1.6 Requirements for No Smoking Places’ have been adjusted in proportion.

According to the evaluation criteria, the two indicators of ‘1.8 law enforcement procedures’ and ‘1.9 fines’ require experts to review the text of the regulations, so that there is a certain degree of subjectivity in the evaluation criteria. The fines (enforcement procedures and fines) in smoke-free regulations are the most important provisions that reflect the enforcement of regulations and enhance the deterrent effect of regulations. In the study of Ravara et al.[4], the main reason for taxi drivers to comply with smoke-free regulations is the mandatory fines of smoke-free regulations. Whether or not to be able to effectively punish violations is a prerequisite for deterrent regulations. The results of interviews with relevant personnel also show that the cancellation of warnings (for individuals) and the cancellation of orders for rectification (for venues) will increase the efficiency of law enforcement and greatly increase the deterrent effect of smoke-free regulations.

Therefore, experts should evaluate the ‘1.8 law enforcement procedures’ based on whether the punishment procedures are simple and can improve the efficiency of law enforcement. Too high or too low fines will affect the efficiency of law enforcement and the deterrent effect of regulations. Because the appropriateness of the fines is affected by various factors such as the city’s economic level, experts should evaluate the amount of ‘1.9 fines’ based on the experts’ practical experience, to determine whether the amount of punishment is appropriate and whether it is conducive to improving the efficiency of law enforcement.

Indicators of law enforcement

Because the multi-sector law enforcement model involves many units, it makes it difficult to estimate the number of law enforcement personnel. Some cities have not given the status of law enforcement personnel, so the data cannot be used for evaluation. The results of the study show that all cities have not increased the number of grassroots law enforcement personnel after the smoke-free legislation’s enactment, only adding the part of tobacco control law enforcement to the comprehensive law enforcement content of each law enforcement unit. The number of law enforcement personnel still depends on the original configuration. Therefore, the deployment of law enforcement personnel is not suitable for inclusion in the evaluation system of the implementation of smoke-free regulations. The staff of the coordinating agency is an important foundation for the smooth implementation of the city’s smoke-free regulations and should be used as an evaluation indicator. In summary, adjustments should be made to ‘2.1.3 Staffing Situation’.

Analysis of the data shows that the eight cities cannot clearly provide funding for tobacco control law enforcement. From the interviews, it was found that the law enforcement departments included tobacco control law enforcement in the work of comprehensive law enforcement, and the funding input was only a part of the comprehensive law enforcement funds. The tobacco control law enforcement funds were not established separately, so the relevant information could not be clearly provided. Therefore, the setting of this indicator does not conform to the actual situation of the city, and the indicator of ‘2.1.4 investment in law enforcement funds per 10000 people’ should be eliminated.

According to the indicator system and evaluation standards, the two indicators of ‘2.1.5 Law Enforcement Documents’ and ‘2.1.6 Law Enforcement Assessment System’ require experts to evaluate based on practical experience, which is subjective. According to WHO’s ‘Making cities smoke-free and Research Report on China's Urban Tobacco Control Law Enforcement’[4,5], the law enforcement documents should include law enforcement plans or programs, law enforcement guidelines or rules, and the content should cover the following parts: general introduction of smoke-free regulations; law enforcement coordination mechanism and organization settings; overall law enforcement strategy; complaint line management and complaint reporting feedback mechanism; law enforcement preparation; and monitoring and evaluation of law enforcement.

With regard to ‘2.2.3 Non-smoking sign posting rate or setting rate’ and ‘2.2.4 Non-smoking sign passing rate’, in Portugal it is required to post signs in service venues and taxis, and relevant research shows that these venues have higher rates of compliance with smoke-free regulations. In contrast, in some restaurants and educational institutions in India, where non-smoking signs are not posted, the compliance rate in these places is also low, indicating that the posting of non-smoking signs is an important part of the environmental construction of non-smoking venues[6-8]. Data analysis shows that some cities do not use the non-smoking sign qualification rate indicators, and the interview results show that most non-smoking signs in non-smoking venues are uniformly printed and issued by law enforcement units. Therefore, only the posting rate or setting rate of non-smoking signs can be used to evaluate the situation of non-smoking signs in venues. The deletion of the indicator ‘Non-smoking sign passing rate’ is suggested.

The law enforcement action and punishment result indicators are the process indicators to measure the output of the law enforcement departments of tobacco control, but there are five indicators in the current indicator system for evaluation, which is slightly cumbersome and not easy to implement the comprehensive evaluation. In order to make the indicator system more convenient, the indicators were deleted. Since the amounts of fines stipulated by the smoke-free laws of various cities varies greatly, in order to make the results more comparable, the number of individual penalty cases per 10000 people is used as an evaluation indicator.

Indicators of communication and education

The results of data collation and analysis show that there are differences among cities in terms of communication methods. When sorting and analyzing the data of each city, it is found that the cost and the effects of different communication methods are inconsistent, so according to the current evaluation standards (the same scores for various communication methods) are not suitable and need to be adjusted. According to ‘China Tobacco Control’[9], the current communication channels for tobacco control in China can be divided into two categories: traditional media and new media, and the influence of traditional media is still unmatched, so it should account for a large proportion of the score. According to the data of the cities, other methods such as posters are also used, and they also need to account for a certain proportion of the index score.

In regard to ‘4.5 Communication Effect’, under this second-level indicator, there are two third-level indicators: ‘4.5.1 Public knowledge rate of core knowledge of smoke-free regulations’ and ‘4.5.2 Public knowledge rate of tobacco harm knowledge’. The results of the study show that because the current surveys in cities are conducted independently, the indicators used are inconsistent, which makes horizontal evaluation impossible. Therefore, for the evaluation of publicity effects, a specific indicator should be selected for evaluation. According to previous studies[10], the awareness rate of smoke-free regulations is a commonly used indicator for smoke-free regulations. As for the awareness rate of tobacco hazards, higher awareness rate (lung cancer) or lower (sexual dysfunction) is not suitable as an effect indicator of horizontal comparison. It is recommended to choose the awareness rate of stroke caused by smoking as an indicator, which can better reflect the communication effect.

Limitations

The indicator system is a bit cumbersome, which may hinder the promotion and application of the next step, so it needs to be further adjusted in practice.

CONCLUSIONS

This study initially improves the indicator system by omitting the least effective second-level and third-level indicators from the aspects of feasibility and practicability, and proposes a more scientific and practical one. It serves as a base for the next step to establish a more feasible and practical evaluation index system for the implementation of smoke-free laws and regulations. Besides, it also provides a scientific basis for tobacco control legislation, law enforcement, and amendments in terms of what are essentials in practice. However, the indicator system still needs to be examined in practice and further improved. It is recommended to continue research, through practice, to further improve the comprehensive evaluation index system, and to establish a more effective data collection method or approach, and finally establish a comprehensive, scientific and simple evaluation system.

Click here for additional data file.