Selecting topic areas for developing quality standards in a resource-limited setting.

Variation in practices of and access to health promotion and disease prevention (P&P) across geographical areas have been studied in Thailand as well as other healthcare settings. The implementation of quality standards (QS)-a concise set of evidence-informed quality statements designed to drive and measure priority quality improvements-can be an option to solve the problem. This paper aims to provide an overview of the priority setting process of topic areas for developing QS and describes the criteria used. Topic selection consisted of an iterative process involving several steps and relevant stakeholders. Review of existing documents on the principles and criteria used for prioritising health technology assessment topics were performed. Problems with healthcare services were reviewed, and stakeholder consultation meetings were conducted to discuss current problems and comment on the proposed prioritisation criteria. Topics were then prioritised based on both empirical evidence derived from literature review and stakeholders' experiences through a deliberative process. Preterm birth, pre-eclampsia and postpartum haemorrhage were selected. The three health problems had significant disease burden; were prevalent among pregnant women in Thailand; led to high mortality and morbidity in mothers and children and caused variation in the practices and service uptake at health facilities. Having agreed-on criteria is one of the important elements of the priority setting process. The criteria should be discussed and refined with various stakeholders. Moreover, key stakeholders, especially the implementers of QS initiative, should be engaged in a constructive way and should be encouraged to actively participate and contribute significantly in the process.

Introduction

Quality improvement of healthcare services has been the main focus among decision makers and stakeholders in recent years. Different interventions have been developed to improve quality of care across primary healthcare and hospital settings. These interventions range from organisation-wide quality improvements to programme-specific or service-specific quality improvements and management function quality improvements.1 Evidence suggested that these initiatives could enhance capacity and practices of healthcare providers as well as improved health predictors and health outcomes of clients.1 Engagement of top-level leadership was reported as a key element in the success of quality improvement project and as a barrier when not present or when leadership changed.

The UK’s National Institute for Health and Clinical Excellence (NICE) implemented quality standards (QS) which is a concise set of evidence-informed quality statements designed to drive and measure priority quality improvements within a particular area of care.2 3 It is developed based on recommendations from clinical practice guidelines that were synthesised to formulate quality statements—thereby providing a pathway to achieving a high level of quality healthcare services. With the engagement of different stakeholders, policy-makers can use QS to ensure that high-quality healthcare services are being provided. Moreover, a range of information on the recommended services can be published for patients and the public to inform them about the quality of care they can expect to receive from their healthcare providers.

The first set of QS, published in 2010, included support for health and social care of dementia,4 stroke in adults5 and reducing the risk of venous thromboembolism in adults;6 since then, NICE has published more than 160 QS.

QS has also been initiated in resource-constrained settings. Given that Kerala, the Indian state, had a gap in quality of care for maternal health, the Kerala government collaborated with the international affairs division of NICE—so called NICE International and the Kerala Federation of Obstetrics and Gynaecology to develop QS on postpartum haemorrhage and hypertensive disorders of pregnancy; as a result, 10 QS were published in 2013.7 In 2014, the Ministry of Health of Vietnam, together with the Vietnam Health Economics Association, NICE International, and with funding support from the Rockefeller Foundation, worked to develop QS for improving the hospital management of strokes.8 Evidence has demonstrated the potential of QS in improving provider’s practices and enhancing processes of care through measuring and monitoring the quality of care.9–11 However, major impacts on health outcomes have not been found.12

In Thailand, the universal health coverage offers all citizens free access to 83 health promotion and disease prevention (P&P) services such as vaccinations, nutritional deficiency screening, hypertension screening, diabetes screening, oral health screening and care, tobacco and alcohol use risk screening, reproductive and mental health promotion, and home visits for chronic disease and bedridden patients.13 This P&P package targets five population groups—pregnant women, newborns and children aged under 5, older children and adolescents aged 6–24, adults aged 25–59 and elderly aged above 60. However, variation in delivery and access of these services across geographical areas have been reported,14–17 indicating the need for quality improvement. As such, QS might be an effective intervention to reduce this variation and improve quality of care. In response, an independent research unit of the Ministry of Public Health (MOPH), Thailand—the Health Intervention and Technology Assessment Program (HITAP)—was commissioned to facilitate the development of QS for P&P services. HITAP was supervised by the former NICE international, that is, the Institute of Global Health Innovation, Imperial College London, UK.

Given resource constraints in the development and implementation of QS, prioritisation was required to maximise its impact. Essentially, this meant that a specific area of care would be focused on. Moreover, as Thailand has never developed QS previously, this initiative can be seen as a demonstration exercise from which HITAP and stakeholders can draw lessons to inform the future’s policy and operation. Nevertheless, limited information was available on how the topic selection for QS development should be or was carried out in other settings.

This article aims to add to this rare literature by providing an overview of the priority setting process of topic areas for developing QS, especially in settings where limitations in resources and data are observed. The paper also describes the criteria used for this purpose. A combination of evidence together with a deliberative process allows priority setting to be undertaken in an empirical, rational and practical manner. It is also believed that this approach can enhance the legitimacy of decision-making by increasing transparency, the value of evidence synthesis and stakeholder engagement. The Thai experience discussed in this article will be helpful for policy-makers and practitioners in other low-resource settings.

The overall process of QS development

The development of QS should be a participatory and multidisciplinary exercise involving policy-makers and relevant stakeholders. It involves several stages including convening QS committee, selecting topic areas, defining the scope, selecting source documents, identifying relevant recommendations, prioritising recommendations, devising statements and measures, conducting budget impact analysis, convening a consultation with interested parties and present the final QS to the committee.2 In Thailand, the development process was informed by a guideline for developing QS in low- and middle-income countries and adapted to the country’s context. A technical working group was set up comprising a chair and topic experts, for example, doctors, nurses, academics and policy-makers; HITAP was the secretariat of this working group.

Topic prioritisation and selection

The priority setting process of topic areas in P&P was carried out in five steps (figure 1). Table 1 summarises the roles and composition of all consultation groups involved in the prioritisation process. Since QS development is new to Thailand, the first step consisted of using the availability of QS in other countries and the readiness of the Thai health database as two inclusion criteria to shortlist services. The examples of QS obtained from other countries were important because they served as prototypes for this development. Additionally, the accuracy and completeness of data within the database were also crucial for measuring the progress and outcomes of the services provided. The P&P services in the benefit package were categorised into five categories per beneficiaries’ groups. In total, 83 services were included and each service in the benefit package was reviewed against these two criteria. Table 2 presents the 21 services for all five categories that met the inclusion criteria.

Figure 1

Selection process of topics for quality standards development. Topics were selected through an iterative process. The first group of stakeholders were gathered to agree on criteria and select ANC as a priority area. The second group of stakeholders were invited to participate in a focus group discussion for the topic engagement process. The working group finalised the prioritised ANC problems for quality standards development. ANC, antenatal care; HITAP, Health Intervention and Technology Assessment Program.

Table 1

Characteristics of three consultation groups

	Stakeholders prioritising topic areas	Stakeholders in topic engagement process	Technical working group
Purposes	To comment on the criteria To prioritise areas in health promotion and disease prevention that need QS	To discuss problems around ANC services in Thailand To suggest issues in the selected area that need QS	To prioritise final ANC services for QS development To provide regular input to the QS To approve the final QS
Members	Five representatives from the NHSO Two representatives from the MOPH’s Strategy and Planning Division Four representatives from Department of Health Two representatives from Department of Mental Health Two representatives from Department of Medical Services One representative from Department of Disease Control One representative from Department of Medical Sciences One representative from the Healthcare Accreditation Institute Two representatives from Provincial Health Offices Six healthcare providers Four academics from the Faculty of Medicine Four representatives from civil society organisations	Six representatives from the NHSO Six representatives from the MOPH’s Bureaus of Reproductive Health, Health Promotion, and Dental Health One representative from the Healthcare Accreditation Institute Four academics from the Faculty of Medicine Seven healthcare providers from district and health promoting hospitals Three health volunteers One pregnant woman	Three obstetricians from university hospitals One obstetrician from Thailand’s Royal College of Obstetricians and Gynaecologists One obstetrician from a district hospital One nurse from a district hospital One representative from the NHSO Three representatives from the MOPH’s Bureaus of Reproductive Health, Health Promotion, and Dental Health One representative from the MOPH’s Institute of Medical Research and Technology Assessment

ANC, antenatal care; NHSO, National Health Security Office; MOPH, Ministry of Public Health; QS, quality standards.

Table 2

Primary selected health promotion and disease prevention services by age group

Pregnant women	0–5 years	6–24 years	25–59 years	60 years and above
Antenatal care Laboratory tests Oral health screening Postnatal care	Oral health screening Vaccination	Fluoride varnish Hypertension screening Reproductive health services and family planning Oral health screening Growth and nutrition assessment	Fluoride varnish Hypertension screening Reproductive health services and family planning Screening for diabetes Screening for tobacco use and alcohol use disorders	Fluoride varnish Cardiovascular risk screening Screening for mental illnesses Oral health screening

Subsequently, inputs for proposing the criteria to further narrow down the choice of services were determined. The research team reviewed international literature to find criteria used in setting the priority of topics for developing QS or related initiatives. The review focused on literature indicating criteria for setting the priority of interventions to improve quality of care as well as existing criteria for the development of the Thai benefit package. The NICE QS guideline recommended: (1) size of problems; (2) having a high impact on patient safety; (3) having a high-cost impact or high resource use and (4) current variations in service delivery or access to services as the criteria.18 19 Furthermore, a study using multiple criteria decision analysis to prioritise topics for health technology assessment for the development of the universal health coverage benefit package in Thailand recommended: (1) size of population affected by disease or health problem; (2) severity of disease; (3) variations in practice; (4) the effectiveness of health intervention; (5) economic impact on household expenditure and (6) equity and social implications as the criteria.20 Variations in service delivery or access to services were also suggested by a study on identifying topics for health technology assessment in Thailand and by the Agency for Healthcare Research and Quality.

From these findings, a set of appropriate and feasible criteria in the Thai context was then drafted by the research team. The criteria included: (1) size of the population affected by disease or health problem; (2) coverage of P&P services; (3) quality of service provision; (4) variations in service delivery or access to services and (5) effectiveness of health intervention. These criteria were proposed with scores ranging from 1 to 5 for each criterion.

Next, stakeholders were invited to a consultation meeting which convened in August 2017 to comment on the proposed criteria. As recommended in the guideline,2 stakeholders comprising policy-makers, healthcare practitioners, academics, service users, patient groups and members of civil society discussed the feasibility and appropriateness of the selected criteria. The composition of stakeholders is presented in table 1. These stakeholders were important in the implementation of P&P programme either making-decision and plan, delivering services, quality assurance or monitoring and evaluation. Moreover, service users, patient groups and members of civil society could offer insight into the challenges and obstacles of receiving P&P services. During the discussion, the following changes were suggested:

Criterion 1: Stakeholders commented that the size of the population affected by disease or health problem was not appropriate because it might overlook some health conditions that are less prevalent but have high severity, for example, rare diseases. Therefore, the burden of disease in terms of the loss of Disability-Adjusted Life Years (DALYs) seemed to be a better criterion to compare the effects of disease on the population. However, some concerns were raised on the availability of information for all diseases and health risks.

Criterion 2: Stakeholders did not agree with the second criterion as they viewed that the coverage of services depended on various factors other than the quality of healthcare provision. For example, a low uptake of diabetes screening may be the result of low risk perceived by service recipients or travel/transport issues to health facilities for people who reside in mountainous areas. As such, having QS would still not solve these issues. In addition, stakeholders pointed out that the second criterion was similar to the fourth criterion, that is, the variation of service delivery or accessibility, which might lead to double counting if these two are used together.

Criterion 3: Stakeholders agreed that the third criterion was important for this project but raised concerns on how to measure the quality of services provided because studies exploring the quality of P&P services were limited.

Criterion 4: Stakeholders agreed with this criterion and suggested removing variation in the services covered by the benefit package of the three health insurance schemes (the Universal Coverage Scheme, the Social Security Scheme and the Civil Servant Medical Benefit Scheme) since they use the same benefit package for P&P services. The criterion should instead focus on the variation in access to services across different subpopulations or geographical areas and the variation in providing such services.

Criterion 5: Stakeholders suggested to remove the fifth criterion on the effectiveness of health intervention because this criterion was already used during the development of benefit package of P&P services.

In addition to the proposed criteria, stakeholders raised concerns about the scoring algorithm as they anticipated that reliable data for each criterion may not be available for all services, resulting in low scores for those services. This may create biases in the priority setting results.

Given that the proposed criteria were deemed unready to be selected in this round coupled with the time constraint, a consensus-based approach was suggested to choose topics for further review. Problems around service delivery of the services in table 2 were discussed by stakeholders, and the antenatal care (ANC) package was ultimately selected. This was chosen as a priority area because ANC services still need to be improved as evidenced by two important national health indicators including the coverage of ANC uptake by pregnant women before the gestational age of twelve weeks (59% in 2016); similarly, only 26% of pregnant women with low risk pregnancies availed of ANC services at least five times in 2016.21 Furthermore, stakeholders suggested that the effectiveness of QS needed to be evaluated to inform policy-makers of its potential in improving quality of care in Thailand before expanding QS to other areas. Following this suggestion, ANC was deemed as the most suitable service because health outcomes from better ANC can be seen on a short-term basis.

Therefore, the team revisited the criteria based on stakeholders’ comments and proposed to use the revised version for the next round of priority setting. The revised criteria included: (1) current quality of service provision in Thailand by considering the readiness of infrastructure and human resources, process of care and health outcomes; (2) burden of disease as measured in terms of loss of DALY; (3) variation in service delivery and uptake across subpopulations or geographical areas and (4) availability of domestic or international guidelines and recommendations. The first three criteria attempt to explain why quality improvement is needed based on the current situation, and the fourth criterion indicates the feasibility of developing QS based on high-quality evidence. Concerning the availability of reliable information about DALY loss, and data on some process of care and health outcomes, for example, postnatal care, screening for mental illnesses, screening for tobacco use and alcohol use disorders, it was not feasible to score the service against each criterion at that time. Therefore, the team proposed to consider the criteria as a starting point for discussion and allow for deliberation to identify priority topics. Table 3 illustrates the revised criteria and their definitions as suggested by the consultation with key stakeholders.

Table 3

Criteria and definitions

Criteria	Definitions
1. Current quality of service provision	Quality of service provision is measured by considering the readiness of infrastructure and human resources, process of care and health outcomes. Related national statistics and figures will be referred to.
2. Burden of disease	The burden of disease is measured in terms of Disability-Adjusted Life Years loss.
3. Variation in service delivery and uptake	This criterion considered documents for variation in service delivery from the provider’s side and uptake of the services from clients. The variation can be seen across subpopulations or geographical areas.
4. Availability of domestic or international guidelines and recommendations.	Related domestic or international guidelines on the topic areas will be sought.

Topic engagement: identifying areas for quality improvement

Topic engagement is a process that aims to identify areas for quality improvement and highlight emerging areas of practice within the selected topic areas. Document reviews were undertaken initially to understand the current situation of ANC services in Thailand and its barriers from both providers’ and clients’ perspectives. This process was followed by two focus group discussions in September 2017 among stakeholders who have been working in the area of maternal and child care. The first group consisted of policy-makers, academics and healthcare providers in referral hospitals. The second group consisted of healthcare providers from district and health-promoting hospitals, health volunteers and a pregnant woman.

Discussions were guided by open-ended questions as informed by the review. The first group discussed issues related to service provision, administration, logistics and communication about ANC services. The second group focused on service provision, service utilisation and barriers to access.

Important issues in ANC services from the perspective of service providers and service recipients were summarised as follows:

Pregnant women were not concerned about or did not perceive the benefits of receiving ANC services within 12 weeks of their gestational age. They also reported a lack of proper information about ANC services.

Health practitioners reported that they used different practices in measuring the fundal height of pregnant women.

Pregnant women reported inadequate laboratory examinations, for example, complete blood count test, the tests for syphilis and HIV among couples of pregnant women and lack of information about the services and results of the examination. On the other hand, health practitioners indicated that the follow-up system for patients who had undergone an examination was inadequate.

It was observed that not all pregnant women received triferdine (iron+folic acid+iodine) supplements throughout their gestation periods even though Thai guidelines recommend triferdine for the entire pregnancy period.22

Health practitioners reported that non-standardised screening programmes for pregnancy-induced hypertension, preterm labour and gestational diabetes were delivered across the country. Additionally, there were also insufficient emergency referral services.

Health practitioners reported time constraints of staff and parents for attending parental schooling, lack of well-trained staff to provide services and lack of standard guidelines for service arrangements. This resulted in variation in providing parental schooling across different geographical areas.

All six issues were perceived as priority areas for improving quality of care. However, given the capacity of the research team and the availability of resources and time, developing QS was possible in only three areas. Therefore, these six issues needed to be prioritised again using the agreed-on criteria as described earlier. The prioritisation process in this stage was conducted by a technical working group comprising experts and representatives from policy-making bodies related to ANC programmes (table 1).

The technical working group suggested that this project should focus on important maternal and child health issues rather than focusing on a single intervention which may not be effective in reducing the burden of the problems; therefore, a package of interventions to mitigate the problems should be formulated. Following this approach, information related to the six issues were discussed. The group finally reached a consensus on selecting the topics of preterm labour, pre-eclampsia and postpartum haemorrhage for developing the first set of QS. These topics were selected for the following reasons:

The three conditions had the high burden of disease as indicated by the DALYs loss of 70 928 DALYs for preterm labour, 1346 DALYs for pre-eclampsia and 1318 DALYs for postpartum haemorrhage.23

The conditions were prevalent among pregnant women in Thailand—preterm birth rate was 12 per 100 births,24 25 the prevalence of pre-eclampsia was 5%–6%26 and postpartum haemorrhage was the cause of 184 maternal deaths from 2011 to 2015.27

These conditions were important problems that led to relatively high mortality and morbidity in mothers and children.27

Variation in the practices and service uptake were not documented but informed by the expert and local practitioner experiences.

Besides the criteria proposed, members of the working groups noted that interventions to tackle these issues already existed and could be carried out by healthcare providers at the primary care level.

Lessons learnt

For the priority setting process, the team first proposed a Multi-Criteria Decision Analysis (MCDA) with a scoring system suggested by previous studies.28 29 It was observed that refining the criteria, scoring and weighting needed more time, which was not practical in this phase of the project. Therefore, a consensus-based deliberative approach seemed to work better in this circumstance like the experiences from Vietnam and India.7 8 The team learnt that the criteria should be discussed and refined with various stakeholders before starting the priority setting process. This would provide the technical team an opportunity to explain the definitions and importance of the criteria. It might ensure that the valuable subcomponents within each criterion were acknowledged. It is recommended that the team should consult representatives of stakeholder groups about the clarity of the criteria and its definitions before drafting a document. The document should then be sent to stakeholders for reviews prior to the meeting. In addition, the availability of reliable evidence to inform each criterion is also important. Some information was not available for all health problems such as DALYs loss due to health problems or health risks and some processes of care and health outcomes. It would be costly and time-consuming to conduct studies on these issues for all health problems. Therefore, only available information together with expert opinions on these criteria was considered. For the next round of QS development, these two criteria, that is, current quality of service provision and burden of disease should be explored and revisited. Additionally, in the long run, a study on the criteria and feasibility of using MCDA for selecting topics for QS development might be needed as this approach offers a more comprehensive and systematic manner for making decisions. In the current situation, however, the combination of a scoring system (MCDA) and deliberative process for priority setting was suggested as the better approach for priority setting.30 This approach emphasised both the power of evidence and rational pursuit provided by the MCDA together with social values raised by stakeholders. As Thailand already had experiences using this approach for prioritising topics for the development of the benefit package,20 it may also have a role in the selection of topic areas for QS development in the future.

QS development should include key stakeholders from various disciplines in each activity. Altogether these stakeholders should provide policy, clinical, technical and administrative inputs that the QS needs. During the selection of topic areas under this project, the team learnt that participation of key stakeholders could enhance the discussions around the criteria, the selection of priority areas and setting the scope of the QS. The engagement of these key stakeholders made QS more acceptable. The process of engaging stakeholders began with setting a clear purpose of the consultation meetings. Then, the team conducted a stakeholder mapping by listing all relevant stakeholders and prioritising key stakeholders based on their responsibilities related to the P&P services. Stakeholders who are expecting to implement the QS initiative should be prioritised and reassure their significant inputs. During the process, all the stakeholders were encouraged to contribute to the discussions. For example, the team invited stakeholders from various disciplines and divided them into two groups. This ensured that all stakeholders would have an equal chance to share their views. It is also important to make the stakeholders understand the value that they will add to this process and reassure that their inputs will be carefully considered. Nevertheless, the participation in the meetings alone might not be enough to collect inputs from the stakeholders. More inputs can be collected using a more structured form for comments, for example, by conducting a survey or using an online platform with a semistructured questionnaire as the experience of NICE.

There are challenges when conducting similar initiatives in other settings. First, it takes time to gain mutual understanding among key stakeholders about the benefits and the differences between QS and clinical practice guidelines. QS is a new innovative intervention. Unlike clinical practice guidelines that cover the entire spectrum of care needed for solving health problems, QS statements focus on priority areas where quality improvement is needed. Second, it is not easy to find appropriate criteria for specific contexts from many existing criteria. Clear and well-defined criteria help achieve consensus in group discussions. Third, it is hard to ensure active participation and equal contributions of all stakeholders.

This study has a number of limitations. First, all criteria were weighted equally, which might not reflect the actual value of each criterion. Appropriate weightings of each criterion might be considered in the future. Second, the priority setting criteria developed under this project were not fully implemented in the selection of ANC problems because of time limitations and lack of reliable information. Third, the current study does not provide information on the entire process of QS development in Thailand as this will be the future work. Fourth, this study did not examine the impact of QS in practice. Therefore, an evaluative study is recommended to explore the effectiveness of QS in improving the quality of the P&P programme. The findings from such evaluative study can then be used to convince health authorities if they wish to expand the development of QS in other priority areas.