Identification of Remarkable Dimensions in Management and Evaluation of Non-Communicable Disease Programmes: A systematic review and meta-synthesis.

OBJECTIVES: This study aimed to identify indicators of proper programme development in the field of non-communicable diseases through the systematic review of existing literature.
METHODS: In this systematic review, a search was conducted through PubMed, Scopus, EMBASE, Web of Science, Google Scholar, Cochrane Library, ProQuest and grey literature. The search was limited to literature published between January 2000 and January 2019.
RESULTS: A total of 36 articles were found relevant to the study objectives. Data were obtained through these articles based on the context, input, process and product (CIPP) model. Seven subsets for context, five subsets for input, four subsets for process and six subsets for the product were identified.
CONCLUSION: According to CIPP subsets, indicators such as programme definition, appropriate organisational culture, structure and evaluation must be considered to develop an appropriate programme to improve health services. © Copyright 2021, Sultan Qaboos University Medical Journal, All Rights Reserved.

In recent decades, one of the leading global challenges in the health system has been noncommunicable diseases (NCDs).1 This results in poor health, economic loss, life loss, diminished quality of life and poor social development.2 NCDs are expected to account for over 70% of all deaths globally based on the prediction of the World Health Organization (WHO).3 It is estimated that 41 million people in countries with low-resource potential will die from NCDs mainly due to cardiovascular diseases (CVDs; 48%), cancers (21%), chronic respiratory diseases (12%) and diabetes (3%).3

The WHO has predicted that appropriate interventions will prevent the spread of NCDs.4 Coping strategies for NCDs include interventions to reduce risks to the general public. These interventions change health behaviours by providing interactions among health care providers and recipients of these services through legislation, financial and tax incentives and public health promotion.5 In this regard, organisations are responsible for managing and controlling NCDs. Despite these undertakings in the field of NCDs, the incidence and prevalence of these diseases have not yet decreased significantly.6

A systematic approach to prioritising, decision-making and managing and improving interventions and community-based strategies is necessary to develop a programme for managing NCDs considering the limited resources available in developing countries.7 Managerial bodies in the field of NCDs have to adopt a participatory approach relying on sound scientific evidence to merge information and set long-term plans.8 Developing a proper plan depends on setting goals and priorities and the way they are shaped to draw up a clear vision of the future. To this end, development plans are based on such factors as awareness of the external environment, the parent organisation and the necessary mechanisms.9 One of the important tasks within the process of programme development is determining the success rate of the programme goal achievement. Furthermore, to influence activities and actions, developing the programme requires continuous monitoring and evaluation and optimising the movement within the designated framework.10

Dimensions of programme management and evaluation include raising community awareness, improving indicators, performing special interventions to reduce risk factors for NCDs, conducting applied research, improving cooperation and participation and establishing a system to track and evaluate progress in the field of control and prevention of NCDs.11

Evaluation helps to compare actual situations with expected outcomes of managers and policymakers towards goals and objectives.12 Different assessment patterns can be applied in the health system, including time-cost models (such as half-life chart – score model), models of organisational excellence and self-evaluation (such as Deming model, quality audit model, Malcolm Baldridge quality model and organisation excellence model) and integrated models (such as smart system, performance charter model, modelling model, goal-based management model, Hoshin management system model, balanced scorecard and Donabedian framework).13 The context, input, process and product (CIPP) model is a comprehensive, functional model that considers all aspects of the programme or system.14 This evaluation model is one of the most recommended frameworks for the conception, design, implementation and assessment to improve decisions emphasising constructive evaluation during the evolutionary steps of a programme.11 The four components of this model include context, input, process and product. The model contributes to the programme evaluators by collecting regular information during development and implementation.15

Despite the prevalence of NCDs and the problems caused by the chronic, progressive and debilitating nature of the diseases and various responses from the health system to focus on prevention with NCD priority, programmes in this field have not been very successful in their implementation.6 To develop a suitable programme, aspects such as accessibility, quality, efficiency and equity, that lead to the promotion of health services and a healthier community, were considered. Therefore, this study aimed to identify significant dimensions of developing programmes for the management and evaluation of NCDs through a systematic review of existing literature.

Methods

The present study is a systematic review of existing literature that aimed to identify dimensions of management and evaluation of NCDs.

Two questions were chosen for the study: 1) What remarkable dimensions should be considered for the management of programmes for NCDs?; 2) What dimensions should be considered for the evaluation of programmes for NCDs?

To identify relevant studies, selected keywords were searched in PubMed, Scopus, EMBASE, Web of Science, Google Scholar, Cochrane Library and ProQuest. Grey literature, searched through Cochrane Library and ProQuest, pertained to the period from January 2000 to January 2019. After eliminating articles that met the exclusion criteria, to increase the sensitivity and to identify all relevant studies, the references were checked, which finally led to the inclusion of three more articles. Keywords used to search the relevant literature included ‘Evaluation OR assessment OR appraisal OR measurement OR audit OR accreditation’, ‘Developing programme’, ‘Noncommunicable disease OR health system’, ‘Framework OR model OR scheme OR pattern OR mechanism OR system’, ‘Non-communicable disease OR Diabetes OR Respiratory disease OR Heart disease OR Cancer Disease’ and ‘Management model OR Management pattern OR Management framework OR Management theory of diseases OR Disease management policy’.

The following inclusion criteria were used to select relevant studies: (1) Articles published between January 2000 and January 2019 in English; (2) studies that focused on the management and development of programmes in the NCD field; (3) studies related to the evaluation of the health and national plans for the evaluation and management of NCDs; and (4) studies that employed evaluating models for the health services. The following exclusion criteria were used to eliminate studies: (1) Studies published in other languages; (2) studies in the area of programme implementation; (3) performance evaluation studies unrelated to the research question; and (4) studies that were published before 2000.

The selected keywords were fed into the database search boxes and, in some databases, the search was limited to abstracts and titles. Search results of keywords were checked by a team member for the inclusion criteria; in cases of uncertainty, decisions were made based on consensus. After identifying unrelated articles and eliminating duplicated ones, the titles of the selected articles were imported into an Excel spreadsheet. The abstracts that met the inclusion criteria and were closely related to the aims of the present study were reviewed and unrelated ones were excluded. Closely related articles were included in the final step of analysis after obtaining a minimum quality score based on the ‘Strengthening the Reporting of Observational Studies in Epidemiology’ (STROBE) statement. Finally, the full text of selected articles was downloaded; in case an article was not for free access, it was purchased by the Kerman University of Medical Sciences Central Library.

Qualitative evaluation, selection of related articles and data collection were performed by two researchers separately (FA and LV). Selected articles were qualitatively evaluated by researchers using the STROBE checklist.16 Any disagreement between the two evaluators was referred to a third researcher (MA). The checklist consists of 22 different sections, following which various aspects of methodology were evaluated including sampling methods, measuring variables, statistical analysis, modifiers of confounders, validity and reliability issues of the tools used and study objectives. Since not all articles exactly adhered to the Strobe checklist, scores were calculated as a percentage and the acceptance threshold was considered to be a minimum score of 65%. Accordingly, studies that acquired the minimum score (65%) from the checklist questions were included and the related data extracted to perform the meta-synthesis.

For each study, data pertaining to the article title, first author, year of publication, place of the study and the prerequisites for programme development were extracted and manually entered into the Excel spreadsheet. Meta-synthesis was performed by collecting data from articles based on the dimensions of the CIPP model.

This study is a part of a PhD thesis in healthcare management at Kerman University of Medical Sciences that has not yet been completed. It was approved by the research ethics committee of Kerman University of Medical Sciences (Code: IR.KMU.REC.1399.272).

Results

A total of 6,633 articles were extracted during the initial search of electronic resources. Of these, 1,835 duplicated articles were removed, 4,798 relatively related articles retained and 3,035 studies excluded after evaluating the titles and abstracts; 1,763 potentially related studies remained and 706 articles were excluded after the restricting search. This restriction was done by setting a time limit, restricting keywords and searching only titles and abstracts. Subsequently, 1,057 studies were selected for text retrieval, following which 1,003 were removed; 54 studies were selected for the evaluation of the inclusion and exclusion criteria, following which 18 articles were set aside. Finally, 36 studies were selected for the analysis process [Figure 1].

Figure 1

Flow diagram of study selection.

Based on the above-described process, 36 articles were reviewed. Of these, five were national programmes. A total of 113 authors’ participation was recorded. Five articles were from the United States, four from South Africa, three from Iran and three from Bangladesh, published across 29 journals. The first paper was published in 2005 and the latest one in 2018; while the highest frequency of publication was in 2017 and 2018 with six articles each year.

Meta-synthesis was performed according to the CIPP model. First, the content of the articles and documents was analysed and all components related to management and evaluation of NCD programmes were placed based on the relationships among the four dimensions: context, input, process and product. Finally, seven sub-categories in the context dimension, five sub-categories in the input dimension, four sub-categories in the process dimension and six sub-categories in the product dimension were identified.

Meta-synthesis was conducted based on the CIPP model, relying on which programme development precondition was extracted. According to the above model, prerequisites for programme development comprise four dimensions: input, context, product and process [Table 1]. Subcategories and preconditions were taken into account for these four dimensions. It is noteworthy that dimensions, subcategories and prerequisites are related to the definitions of the CIPP model. In the context dimension, the relevant elements were placed as an attempt to identify problems, needs and opportunities in the field.17 Input was evaluated to specify and assess the existing capabilities, possible programme strategies for goal achievement, intended methods for implementing strategies, organisations and other resources, including budget and time.18

Table 1

Context, input, process and product dimensions necessary for plan developing

Dimension	Subset	Subcategory	Prerequisites
Context	Objective	Stages of development	Considering the process of forming a team, identifying dimensions of the programme, drafting, feedback to members and developing final goals
		Clarity	Considering realistic, clear trends in goal development
		Feasibility of implementation	Feasibility of implementation via considering adequate resources
		Coverage	Achieving health system goals, improving community health, reducing health inequality, identifying budgets and creating accountable systems
			International databases and targets, surveillance organisation, non-communicable disease control and prevention centres
	Programme	Development	Partnerships, clear policies, specific strategies and activities, stakeholders, outcomes, evaluation criterion and resource
		Coverage	Developing intervention programmes
			Developing an operational plan and roadmap
			Developing programmes at national and international levels
		Nature	Covering long-term, short-term and mid-term standards in the fields of home, school and workplace health
	Mission		Developing policies and documents aimed at capacity building, accessibility enhancement, cost reduction, careful monitoring and evaluation, multi-dimensionality, political commitment and prioritisation.
	Guidelines		Developing guidelines in the areas of managerial, economic, social, legal and goal-based protocols
	Technology	Information technology and tools	Application of multilateral information and communication technology, management and decision support system, metadata technology and data integration and media usable technology
		Technology in the field of people and culture and community	National and international collaborative innovative approaches
	Culture	Motivation	Attention to organisational incentives and creating a reward system to encourage initiatives and increase stakeholder confidence
		Culture	Investigating beliefs, ethnography and identifying the impact of cultural factors on risk factors
	Organisation	Accessibility	Access to diagnostic, treatment and essential medicines
			Geographical access
			Access to the right budget
		Prioritisation	Participatory prioritisation considering environmental, social and cultural, legal and political contexts
			Prioritisation through motivating, successful processes, stakeholder engagement and coordination
			Prioritisation through infrastructure identification and priority mechanisms (process, tool, method), use of evidence and public values
Input	Executive strategies		Developing international and national clinical guidelines to reduce disability, mortality, disease burden and appropriate design of intervention packages
			Developing comprehensive interdisciplinary guidelines to implement effective, integrated, sustainable and evidence-based policies, human resources mobilisation, research, monitoring and managing complex cases
			Developing comprehensive interdisciplinary guidelines to implement effective, integrated, sustainable and evidence-based policies, human resources mobilisation, research, monitoring and managing complex cases
			Developing strategies to improve municipal performance (commitment to create public services for recreational spaces and parks), transportation, education (promoting healthy programmes in healthy school food distribution), tax and food industry (use of native agricultural products and natural foods)
			Developing global marketing strategies, trade liberalisation, foreign investment and legal strategies to reduce demand (through price, tax)
	Resources	Human	Resource mobilisation and investment coordination to strengthen the human resources system
			Upgrading the capacity of specialist staff such as managers, physicians, nurses, health care staff and volunteers
		Equipment	Proper investment in equipment
		Financial	Economic and financial tools to fund equity in sectors
	Structure	Physical	Health service provider centres, provincial regional hospitals, educational consulting centres, diagnostic and pharmaceutical centres
		Programmatic	Redesigning the system with the aim of socialising and collaborating, accessing services, supporting clinical information, self-management, managing and supporting technology and equipment and creating an organisational structure consisting of different stakeholders
			Promoting capacity building in prevention and control in governmental and non-governmental service organisations
	Teaching		Increasing awareness through the authority delegated to local organisations to increase education and information for stakeholders and patients
			National and international training programmes for professionals, students, policymakers and clients and building up of a comprehensive participatory approach to developmental aid in education and knowledge, beliefs, attitudes and skills for individuals
	Learning		Developing scientific and evidence-based knowledge with the help of specialised public and private institutions to create a favourable environment for empowerment
Process	Responsibilities		Responsibility at the global, regional and national level for quality of service, patient safety and service effectiveness
			Proper accountability for noncommunicable diseases through collaboration commitment
		Individuals and stakeholders	Shared responsibility and proper communication among the stakeholders to develop the plan (stakeholders include community members, health professionals and non-health organizations, health and non-health associations, municipalities, the private sector, religious leaders, nongovernmental and local leaders and the education and training sector)
		Levels of service provision	Control mechanisms through communication at the service level
	Communications	Cross-sectoral communication	Communication at different levels of society, family and individuals, agriculture, environment, industry (coordination of administrative and non-administrative structure of tobacco laws, alcohol), education, economy, nutrition, media, urban planning, Ministry of Information (for information), collaboration between various departments of the Ministry of Health, religious organisations, trade unions, the insurance industry (private and public), universities, policymakers and community leaders
			Professional involvement of the national and international community in development, including engagement of community, health professionals, volunteers, advisory committees and health professionals
	Management		Strong leadership aimed at creating comprehensive policies (national and international) and monitoring activities
			Clear governance and leadership in every part of the programme
			Managing and leading services such as screening, diagnosis, treatment, follow-up, case management, referral, counselling, motivating, lifestyle changes, standard treatment protocol, team-based care, access to core data sets, supervision, patient empowerment, social participation and policy interventions
			Management in programme development to identify potential benefits, global coverage, multilateral actions, definition of multi-sectoral relationships (involvement of all aspects of private / public / industry / civil society / university), reduction of risk factors, coordination and cooperation
	Leadership		Level of service coverage, prevention in society, the time between diagnosis and treatment, survival rate and quality of life
Product	Benefits of change in the organisation		Use of scientific evidence and dissemination of information, goals, attention to multidisciplinary and stakeholder environments, fairness of integrated actions and essential interventions
			Appropriate interventions and accountability, promoting services based on available resources and infrastructure, resuscitation and reorganisation of health services, early diagnosis and integrated care, documentation and monitoring of activities, multi-sectoral participatory interventions, support and sensitisation of policy planners and programme managers and providers
	Health promotion		Development of operational plans, implementation strategies and roadmaps
			Promotion of fiscal policy through tax, subsidies and system to encourage healthy nutrition
			Increasing the role of the government as a supervisor of efficient and integrated prevention and control policies, allocating multilateral resources and plans and monitoring framework for plans and guidelines
	Resource use effectiveness assessment		Cost effectiveness of resources, National Health Reporting System (annually, monthly and periodically)
			Evaluation of programme results and programme performance based on international strategies
			Systematic evaluation and monitoring through review of the number of patients admitted, follow-up, treatment (improved, died, stopped treatment, stopped follow-up) and transfer to other health care centres
			Documentation and policy evaluation process
	People assessment		Feedback of evaluation results to contacts
			Assessment of trends and outcomes of interventions at the individual, social and environmental level
	Work progress assessment		Evaluation and monitoring of the priorities identified and work progress, integrated disease management and making the best decision
			Evaluation of facilities for diagnosis and treatment
			Assessment of cultural issues in prevention and control in society
			Evaluation and implementation of community-based activities and community initiatives
			Evaluating the impact of programmes on reducing disease risk factors
	Information system evaluation		Monitoring through NCD information and data
			Strengthen monitoring and evaluation in the area of education, research and regional strategies

Process evaluation is done to identify or predict executive problems in the field of NCDs and the desirability of the process of implementing these activities.18 In the output evaluation, the results were measured against the goals of the programme and the relationship between expectations and actual results. In this study, the findings were categorised based on the definitions of the four dimensions. The context dimension showed the subsets of the context for programme development. They include the objective (stages of development, clarity, feasibility of implementation and coverage), programme (development, coverage and nature), mission, guidelines, technology, culture and organisation. The input dimension includes executive strategies, resources (human, equipment and financial), structure (physical and programmatic), teaching and learning. The process dimension includes responsibilities, communications, management and leadership. The product dimension includes the benefits of change in the organisation, health promotion, resource use effectiveness assessment, people assessment, work progress assessment and information system evaluation.

Discussion

This study attempted to identify crucial and effective components for management and evaluation of the programmes in the field of NCDs and the creation of a responsive system. The findings were synthesised based on the CIPP model.

In the context dimension, factors such as the objective, development and coverage programme, mission, guidelines, technology, culture and organisation were studied. In this dimension, one of the prerequisites was to consider the process of developing programme objectives appropriately as well as checking clarity, feasibility of implementation and coverage. Peykari et al. stated that clear and appropriate goals should be considered in developing national plans. Referring to a national programme for the management and control of risk factors, they suggest that developing an operational plan and roadmap, as well as providing accountability through political commitment, is one of the success factors.1 The category of technology can be considered to be an appropriate mode in the development of the programme. In this regard, Sankaranarayanan et al. have indicated that most technological responses to address personnel shortages and problems of access to health services focus on NCD programmes. For example, they demonstrated the effectiveness of using cellphone applications to check patients’ compliance with medication regimens for managing diabetes and asthma.19

In the input section, some dimensions including executive strategies, resources, structure, teaching and learning have been presented. In developing NCD programmes, one of the aspects to be considered is the resource (e.g. human, equipment and financial). Nishtar suggests that a focus on human resources for health (HRH) in the context of NCD control could have been a lever for health system change after 2015, which led to the measures designed to improve health systems more broadly.20 HRH is also an important lever in the post-2015 efforts for the attainment of universal health coverage (UHC). Of all the health system resources, the most strategic ones are human, physical, technological and financial. Human resources can implement changes in the health system on a collective or individual basis.20 In programme development, providing an appropriate physical and programmatic structure is required, such as municipal commitment to creating public services, providing services centres and creating an organisational structure. In this regard, the Planning Commission of India (2003) states that, to achieve maximum impact on NCDs, programmes that are both horizontal (such as concerning public health and social welfare) and vertical (strategies focusing on populations such as controlling risk factors) should be integrated.21

The process dimension includes responsibilities, communications, management and leadership. One of the prerequisites for programme development is that communications and multi-sectoral collaborations are taken into account. The political statement of the United Nations (2012) stated that since NCDs resulted in higher costs and lower productivity for families, individuals and societies, coordination and multi-sector communications are essential to resolve the problems through promoting tools to identify and mobilise sufficient funds.2 Moreover, the Australian National Preventive Health Agency (2010) stated that cooperation in health matters was important and one should draw a distinction between processes designed for cross-sectional coordination and the official means of increasing involvement of different sectors.22

The management of programme development includes detecting, screening and treating the diseases and providing access to palliative care for people in need. Sarink et al. concluded that coronary artery disease (CAD) could be controlled by engaging the public in policy-making, organising, coordinating and financing.23 Public involvement can be evaluated in making policies by rules and regulation. Hence, a management framework comprising legislation, regulations and screening, as well as public education, seems necessary for preventing and controlling cardiovascular diseases.23

The product dimension includes the benefits of change in the organisation, health promotion, resource use effectiveness assessment and assessment of people, work progress and information systems. Monitoring through the information system and data on NCDs was found important in programme development. The strategic plan of Bangladesh (2007–2010) claimed that it would emphasise NCD surveillance mechanism through the National Health Information System which generates data and publish them in database form which provides access to information for policymakers, public health managers and researchers globally.24 One of the prerequisites in the product dimension is the issue of health promotion which can be met through appropriate interventions and accountability, promoting available resources and infrastructure, integrated care, documentation and monitoring of activities and multi-sectoral support of policymakers, programme managers and providers. In a study by Diem et al., the role of capacity building was established through a comprehensive approach to public health interventions such as creating processes of improvement, knowledge production, increasing need evaluation, examining the characteristics of the target population and identifying the resources and required procedures.25 Mondal et al. stated that, through appropriate approaches towards interdisciplinary leadership, the shift from treatment to prevention, guideline development on interdisciplinary communication education, community access and investment within the world framework have adopted a healthier approach to accountability.26

It is worth noting that in programme development, it is crucial to have common responsibility and proper communication among the stakeholders including community members, health professionals and non-health organisations, municipalities, the private sector, religious leaders, nongovernmental and local leaders and the Ministry of Education. Today, global health due to the prevalence and incidence of NCDs is in a state of crisis and the health system is trying to respond to this crisis.2 Proper planning and management are essential at all levels (primary, secondary and tertiary) in all types of organisations (public, private and non-profit). Therefore, to develop, implement and evaluate the appropriate programme, a framework and a collective approach should be considered in a comprehensive view of all effective factors to promote community health.

This study systematically considered the components of appropriate programme development. Although existing literature has already considered the dimensions of developing a programme in detail, a comprehensive review did not exist. Therefore, this study improves the current knowledge of this field by collecting and categorising all necessary dimensions for developing a programme.

In this article, data were collected based on the CIPP model. It is acknowledged that other models are available for this purpose; using any other model may change the results. Moreover, limited access to the full text of some articles and documents could be the reason that some studies have not been reviewed. The fact that less robust evidence was reviewed for developed national and strategic programmes could be regarded as another limitation of the present study. However, this study offered an opportunity to study substantial plans and valuable insights.

Conclusion

Certain dimensions are essential for programme development, including engaging all key stakeholders, providing required resources and addressing cultural issues. Furthermore, a proper development of programmes should take into account such issues as developing, implementing and evaluating programmes and all components such as context, input, process and product factors. This study thereby provides valuable information for policy-makers and planners to develop appropriate programmes in a sound and cost-effective manner.